[IDV #TNW-894406]: IDV: Lightning data

Hi Dave,

Sorry for the slow reply... I eventually went skiing yesterday and was too
exhausted to check on support email after I got home and had dinner.  FYI
the skiing was great :-)

re:
> Thanks for the reply, even from out of the office. (You're famous for that. 
> :-) )

The nice thing about how we have support setup is we are able to address
questions from wherever we happen to be.  Yesterday, for instance, I was at
home conversing with Kevin Tyle at UAlbany regarding some problems that
we were seeing on the NLDN injection machine right up until I hopped in
my truck to head to Winter Park for an afternoon of fun in the snow.

re:
> Don might never escape from the legacy he's created with the IDV, even
> if he actually wants to.

Don is still actively involved in IDV development and use.  After all, the
IDV is his baby, and it is hard to let children go :-)

re: NLDN
> I didn't know that the data hadn't been available for a while, so it was
> just coincidence that when I asked Dave Knight  on January 19 to start
> feeding us the data, Albany happened to be on the verge of restoring the
> feed. The data started arriving on February 5, with several days missing 
> since then.

Hmm... you actually got good, usable data before this past Tuesday?  If the
answer is yes, I am, to say the least, shocked.  David and I had been trying
to get the old NLDN ingest system working at UAlbany, but we didn't have
any luck with what we were trying.  I bit the bullet last week and dove
in to rewriting the ingest code.  I got things running last Wednesday or 
Thursday
and then tweaked code over the weekend while running comparisons with the
USPLN data.  By Tuesday (Monday was a holiday for us, so it gave me time
to continue the comparisons), I felt confident enough to "flip the switch"
on the new process.  After doing that I asked David and Kevin Tyle if it
would be possible to fix some problems on the UAlbany machine that I had run
into (hostname was not fully qualified; the LDM running there was ancient;
the old NLDN ingest stuff needed to be turned off; etc).  The changing of
the hostname to be fully-qualified seemed to be related to a problem with
their name server which resulted in the data flow stopping at about 8Z
yesterday morning.  Kevin dove in and solved a series of problems and was
able to install LDM 6.8.1 sometime in the mid-afternoon (he ran into problems
with LDM-6.9.7 that Steve Emmerson is investigating now).  Since then the
data appears to have been flowing nicely... I just did a plot of all of the
data received today and compared it with the USPLN data and the comparison
looks very good over the southeastern U.S. where all the activity has been.

re: The fact that there is remaining work is the main reason that there has
not been an announcement made to the IDD community about the return of the data.

> OK. It probably doesn't hurt to know that some of us are already waiting 
> eagerly
> in the wings.

Hopefully the problems we were seeing on striker2 (the NLDN injection machine)
are now fixed enough that the data will continue flowing.  I will be continuing
to work on the reception process at UAlbany, but I intend to do that with no
noticeable impact on data reception to the end-users.
 
re: were you aware that there are two sources of lightning data

> Yes, before I contact Unidata support, I almost always pick through
> Unidata's Web pages and flounder around on my own trying to find an
> answer.

Excellent!

re:
> (Exception: comments and suggestions about IDV usability. Guess I haven't 
> quite
> realized that the IDV Steering Committee is no more, and haven't gotten out 
> of the
> habit. :-)  )

We are always looking for interested, involved users to provide guidance on
what features are important/need to be added or refined!

re: comparison of the global USPLN data and the NLDN data now flowing in the IDD

> I see what you mean (from the sample plot)--we look forward to getting those 
> data.

There will be an announcement after the necessary sign-up web pages have been
updated at Embry Riddle.  WSI is working with Chris Herbster to get this done,
and then we will update our lightning pages to reflect the new information.
Users will need to sign a new agreement with WSI/Embry Riddle to get the global
data, but the process will be easy and not much different from what they
had to do for the existing CONUS/North America USPLN data.

re: you were the first to notice (comment, actually) that the NLDN data was 
flowing

> As noted above, that was dumb luck.

I was expecting Gilbert to be the first one to notice. He may not have been
paying attention, however, because he receives the USPLN data and was the
first test site for the global USPLN data.

re: IDV procedure to use/display lightning data

> OK. I'll hold off on the other things that I've been intending to add to the
> IDV wish list (such as labels on latitude and longitude lines, or the option
> to add them).

OK.  I will be talking with Yuan and Julien today about the lightning use in
the IDV.  Hopefully, we will be able to send you a HOWTO soon.

re:
> When you say "we", should I conclude that part of your time now allocated to
> working on the IDV, or is that reading too much into it?

I help out with support where ever I can.

I became more involved in the IDV after becoming part of the UCAR Africa
Initiative and because of my role on the McIDAS Advisory Committee (MAC)
run by SSEC.  Unidata's part of the UCAR Africa Initiative is to make
available technologies we have developed/distribute to the project
personnel.  The packages of immediate interest are:  LDM, IDV, THREDDS, RAMADDA,
and the ADDE component of McIDAS.  As far as the MAC, I have served on this
advisory body since its inception.  Its charter is to advise SSEC developers
on things that are needed in the next-generation McIDAS, McIDAS-V.  If you
aren't aware of McIDAS-V, it is built on top of the IDV framework (which is,
in turn, built on top of VisAD which was developed by Bill Hibbard of SSEC)
and incorporates some tools that are important to the satellite meteorology
community (like the multi/hyper spectral analysis tool Hydra).

The upshot is that I use the IDV routinely for various tasks, so I can help
out a bit with the huge amount of support questions that are coming in from
the ever expanding IDV community of users.

We will contact you with the IDV procedure for displaying the NLDN and USPLN
lightning data.  The one thing that I can tell you is that the data will
need to be local files since serving either NLDN or USPLN data from a
remote server is not allowed by agreements with the lightning providers (Vaisala
and WSI, respectively).

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: TNW-894406
Department: Support IDV
Priority: Normal
Status: Closed