20020428: nowrad data from adde.ucar.edu bad

>From: Leigh Orf <address@hidden>
>Organization: Department of Tesselating Kumquats
>Keywords: 200204290132.g3T1Wta03079 McIDAS ADDE

Leigh,

>How's it going? It's been a while, but I guess that's OK since that
>probably means things are working well on my side :)

Things keep moving along, thanks.  I hope all is well for you as well.

>One recent bit of strangeness: lately the RTNOWRAD data from
>adde.ucar.edu will display but it's clearly not the latest radar,
>although the date stamp on the bottom of the image says it is. RTNEXRAD
>data is fine. Is there something up with adde.ucar.edu as far as nowrad
>data is concerned?

Hmm...  The problem is that you should not be accessing this dataset.
The NOWrad (tm) imagery must be purchased from WSI Corp directly.  It
then gets fed point-to-point by LDMs in much the same manner that the
various other datastreams do.  More on this below

>Also - how would I go about ingesting nexrad/nowrad data locally to
>storm2.atms.unca.edu? I take it I'd (1) have to get permission from an
>upstream server and (2) put the magic perl regular expression in
>pqact.conf.

You will not be able to ingest NOWrad data from anyone but WSI.  Their
charge for the NOWrad composites is pretty hefty (last time I looked it
was on the order of $250 per month), but there is a different
solution.  As far as the NEXRAD Level III products, we can get you
setup with an upstream feed site for those products.  The thing you
must know about the NEXRAD data is that there is a LOT of it.  Consider
the volume of 159 sites each producing 15 products at rates of 5, 6 or
10 per hour.  Even though the products are zlib compressed, this makes
for a lot of data.  Of course, the LDM allows you to request a subset
of the radars, and that is what we recommend that you do.  So, your
first job is to decide which of the NEXRADs you will want to injest.
We will determine a feed site for you to get those products, and then
you will need to configure your LDM's pqact.conf file to FILE those
products into a directory structure that will make them easiest to use
(and scour).

>I notice the upper air soundings being served from storm2.atms.unca.edu
>still give that broken pipe message - any luck hunting that bug down?

I searched long and hard for any but in the McIDAS code, and I didn't
find any.  Given that the problem only occurs on Linux, and it only
appeared after the equivalent of the RedHat 6.0 release, we are of the
mind that the problem is some sort of very strange interaction with the
OS.  SSEC has been investigating the problem themselves of late (within
the past month or so), and I know that I could rewrite the server to
not chain ADDE service requests and get around the problem that way.
If SSEC can't come up with a fix (and this looks doubtful), I will be
forced to do the server rewrite -- not something I want to do given
that the code works on ever other OS I have tried.

>I
>have not upgraded the Linux kernel on storm2 since forever (it's
>running 2.2.18) - perhaps a newer kernel is in order?

I have tried every kernel that has become available since you first
reported this problem.  They all act the exact same way, so upgrading
will do no good here.

>I can switch over
>to adde.ucar.edu as an ADDE server, except that RTPTSRC also covers
>lightning data, which storm2 does serve but adde.ucar.edu does not! Too
>bad we can't have finer-grained control over products.

I agree completely.  Now, you could create a dataset that had everything
except the upper air data and serve that set from storm2 while continuing
to look at the upper air data from one of the many community servers,
but this will be less than satisfactory if you are using the current
MCGUI.  The reason for this is that MCGUI currently doesn't support access
to more than one POINT dataset.  This will change in the 7.9 release this
summer, but this does you no good right at the moment.

>I've bumped into various and sundry mcidas-x weirndesses over the past
>months, but by and large it's been pretty stable. Good job :)

Please pass along any weirdnesses you find.  This helps me find and fix
bugs and generally make things better.

>Oh, one more thing: is there a master list of ADDE servers, and what
>products they serve? It would be nice to have backup servers - and I'm
>willing to open storm2.atms.unca.edu up to the world too.

If you are using the MCGUI, then the list of servers is built in and
can be switched between easily.  Simply bring up the ADDE Client Routing
Config widget (click on the button just to the right of the button with
the Z), and then click on the IMAGE tab.  You will see all of the
IMAGE datasets supported by the MCGUI, and each one has a drop down
list of servers that are accessible to all users.  I can provide more
information on this if you like.

>>From address@hidden Sun Apr 28 20:04:09 2002

>Here is a screenshot of mcidas showing 'latest' nowrad data. It appears
>that the data may be recent but the mapping is messed up - note the
>radar 'ground clutter' signatures over the Atlantic. Thought this might
>help you figure this out. This was displayed using a custom load, centered
>over GSP with a LIN and ELE blowdown of -2. Changing the blowdowun/blowup
>numbers moves things around in a weird way - none of the mappings
>are right.

I looked at the GIF (tm) you sent and see that the navigation is indeed
off.  Since we are moving away from the for-pay WSI composites to
freely available ones that we are producing ourselves, I am not of the
mind to spend much, if any, time trying to troubleshoot the NOWrad
navigation problem.

In a couple of days, I will be makint the 7.805 update to McIDAS available
to the community.  In this release (and 7.804) the MCGUI switches to support
of the RTNOWRAD dataset to one called NEXRCOMP.  NEXRCOMP products are
contained in the FNEXRAD LDM feed, and are decoded using either the
ldm-mcidas 'pnga2area' or 'pngg2gini' decoders.  Both of these decoders
are available in a new release of the ldm-mcidas package:

http://www.unidata.ucar.edu/packages/mcidas/mcidd

This will be announced at the same time as the new McIDAS addendum and
the general availability of the composites in the FNEXRAD feed.
The example pqact.conf file distributed with the ldm-mcidas package
contains entries that will decode and file thes products.  One of the
things left to be done for the McIDAS addendum is a BATCH file that
will allow sites to setup ADDE service for the composites on their
own systems.

To get a quick idea of what I am talking about, try the following from
a McIDAS-X session:

DATALOC ADD NEXRCOMP atm.geo.nsf.gov
IMGDISP NEXRCOMP/1KN0R-NAT STA=TWX MAG=-5 EU=BREF24 REFRESH='EG;MAP H;BAR'

Find an interesting feature and look at it in full resolution.  As I
write this, there is some interesting weather streatching from Mississippi
through Alabama into Georgia.  So, my choosing 33:30 88 as a load center
works pretty well:

IMGDISP NEXRCOMP/1KN0R-NAT LATLON=33:30 88 MAG=1 EU=BREF24 REFRESH='EG;MAP 
H;BAR'

By the time you read this, the activity is bound to be somewhere else.
I leave it to you to pick an interesting location.

So, look for announcements later this week.  Also, if you are not on
the mcidas-x email list, you should subscribe so you will get the
notification of the addendum availability.

>Thanks,

Time to turn in for the night...

Tom