[LDM #BXH-661016]: NAM conduit data not arriving via chinkapin.cs.indiana.ed u LDM

Hi Felix,

re:
re: I assume that a downstream LDM is running on Chinkapin and is not
receiving NAM data from an upstream LDM somewhere.

> As I understand it, this is the problem.  We are indeed a downstream
> LDM server and we are not receiving our NAM "conduit" data, which is
> typically delivered via IDD/LDM.

OK.

re: What is the fully-qualified hostname on which the upstream LDM is
running?

> idd.cise-nsf.gov
> idd.unidata.ucar.edu

OK.  These toplevel IDD relay nodes both have all of the data available
in CONDUIT.  If the data you are requesting is available, they will
send them to you if you are allowed.

> We have determined that these are our upstream LDM servers by
> examining ldmd.conf.  These are the lines from ldmd.conf where these
> hostnames appear:
> 
> request WMO     ".*"    idd.cise-nsf.gov
> request WMO     ".*"    idd.unidata.ucar.edu
> request NNEXRAD ".*"    idd.cise-nsf.gov
> request NNEXRAD ".*"    idd.unidata.ucar.edu
> request CRAFT   ".*"    idd.cise-nsf.gov
> request CRAFT   ".*"    idd.unidata.ucar.edu
> request CONDUIT 
> "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)!" 
> idd.cise-nsf.gov
> request CONDUIT 
> "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)!" 
> idd.unidata.ucar.edu
> request NMC "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) 
> !(.*)!" idd.cise-nsf.gov
> request NMC "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) 
> !(.*)!" idd.unidata.ucar.edu

The request lines for 'NMC' are redundant:

NMC == GPSSRC|CONDUIT|FNEXRAD

You can remove them from your ~ldm/etc/ldmd.conf file.

NOTE:

- Remember that after making changes to ldmd.conf, you will need to stop
  and restart your LDM for the changes to take effect:

<as 'ldm'>
-- edit ~ldm/etc/ldmd.conf and remove the REQUEST lines for NMC
ldmadmin stop
ldmadmin start

re: Is there anything untoward in the LDM log file on Chinkapin?  The
log file is, typically, ~ldm/logs/ldmd.log.

> The log file is empty.

If ~ldm/logs/ldmd.log has zero length, it means that logging is not
setup/setup correctly on your machine.  The LDM log file is an invaluable
resource in troubleshooting problems, so this is something that needs fixing.

re: Is there anything untoward in the LDM log file on the upstream
host?

> I don't have access to these hosts presently.  Should I?

No.  Steve asked this to see if you had contacted the LDM administrator
on your upstream host(s).  Your reply indicated that we are the administrator
for both machines.  I can assure you that your machine is allowed to
request CONDUIT data on both of these toplevel relays.

re: What does the notifmye command do when executed by the LDM user
on Chinkapin:

> Note here that setting the NAM_PATTERN in bash seemed impossible since
> the pattern contains an exclamation mark, and bash won't let you escape
> exclamation marks normally (an escaped excl. mark still prints the
> backslash, which messes up the regex).  I used zsh to issue this
> command, but even then I had to add an additional space at the end of
> the NAM_PATTERN string.  This may have broken my regexp's
> functionality.  (?)

I think that your regular expression pattern is responsible for you
not receiving the data you want.

> UPSTREAM_HOST set to: idd.unidata.ucar.edu
> NAM_PATTERN set to:
> ^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)!
> 
> RESULT:
> 
> [ldm@chinkapin]~/ldmhome/etc% notifyme -vl- -h $UPSTREAM_HOST -f CONDUIT -o 
> 3600 -p $NAM_PATTERN
> Mar 05 18:20:58 notifyme[11687] NOTE: Starting Up: idd.unidata.ucar.edu: 
> 20090305172058.353 TS_ENDT {{CONDUIT,
> "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}}
> Mar 05 18:20:58 notifyme[11687] NOTE: LDM-5 desired product-class:
> 20090305172058.353 TS_ENDT {{CONDUIT, 
> "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}}
> Mar 05 18:20:58 notifyme[11687] INFO: Resolving idd.unidata.ucar.edu to
> 128.117.140.3 took 0.000269 seconds
> Mar 05 18:20:58 notifyme[11687] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK

Very good.  This demonstrates that you have been allowed to request CONDUIT data
from idd.unidata.ucar.edu.

Question:

- did you let this run for enough time to see if there were any products
  that match your pattern?

I ask this because I just ran the a similar 'notifyme' from my workstation
here at the UPC and got results:

% notifyme -vl- -f CONDUIT -h idd.unidata.ucar.edu -p 
"^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! " -o 
100000
Mar 06 20:44:11 notifyme[9131] NOTE: Starting Up: idd.unidata.ucar.edu: 
20090305165731.681 TS_ENDT {{CONDUIT,  
"^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}}
Mar 06 20:44:11 notifyme[9131] NOTE: LDM-5 desired product-class: 
20090305165731.681 TS_ENDT {{CONDUIT,  
"^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}}
Mar 06 20:44:11 notifyme[9131] INFO: Resolving idd.unidata.ucar.edu to 
128.117.140.3 took 0.000424 seconds
Mar 06 20:44:11 notifyme[9131] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK
Mar 06 20:44:16 notifyme[9131] INFO:     7645 20090306193739.826 CONDUIT 002  
data/nccf/com/nam/prod/nam.20090306/nam.t18z.awip3d00.tm00.grib2 
!grib2/ncep/NAM_84/#000/200903061800F000/VSBY/0 - NONE! 000002
Mar 06 20:44:16 notifyme[9131] INFO:    15420 20090306193739.827 CONDUIT 007  
data/nccf/com/nam/prod/nam.20090306/nam.t18z.awip3d00.tm00.grib2 
!grib2/ncep/NAM_84/#000/200903061800F000/AVOR/1000 Pa PRES! 000007
Mar 06 20:44:16 notifyme[9131] INFO:    10586 20090306193739.827 CONDUIT 012  
data/nccf/com/nam/prod/nam.20090306/nam.t18z.awip3d00.tm00.grib2 
!grib2/ncep/NAM_84/#000/200903061800F000/UREL/10 m HGHT! 000012
 ...

Notes:

- you can sorround the regular expression pattern in double quotes so that the
  exclamation points do not get interpreted by your shell

- I included the '-o 10000' flag to look back for products that match the
  specified pattern over the past 10000 seconds

> UPSTREAM_HOST set to: idd.cise-nsf.gov
> 
> RESULT:
> [ldm@chinkapin]~/ldmhome/etc% notifyme -vl- -h $UPSTREAM_HOST -f CONDUIT
> -o 3600 -p $NAM_PATTERN
> Mar 05 18:29:10 notifyme[11932] NOTE: Starting Up: idd.cise-nsf.gov:
> 20090305172910.568 TS_ENDT {{CONDUIT, 
> "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}}
> Mar 05 18:29:10 notifyme[11932] NOTE: LDM-5 desired product-class:
> 20090305172910.568 TS_ENDT {{CONDUIT, 
> "^data/nccf/com/nam/prod/nam.(.*)/nam.t(.*)z.awip3d(.*).tm(.*) !(.*)! "}}
> Mar 05 18:29:10 notifyme[11932] INFO: Resolving idd.cise-nsf.gov to
> 192.12.209.62 took 0.00027 seconds
> Mar 05 18:29:10 notifyme[11932] NOTE: NOTIFYME(idd.cise-nsf.gov): OK

Ditto for idd.cise-nsf.gov: your data request was allowed.

I took a look at the real time statistics that your machine is reporting
back to us:

Unidata HomePage
http://www.unidata.ucar.edu
  Tools -> IDD
  http://www.unidata.ucar.edu/software/idd
    Real-Time IDD statistics
    http://www.unidata.ucar.edu/software/idd/rtstats/
      Statistics by Host
      http://www.unidata.ucar.edu/cgi-bin/rtstats/siteindex

      chinkapin.cs.indiana.edu
      
http://www.unidata.ucar.edu/cgi-bin/rtstats/siteindex?chinkapin.cs.indiana.edu

From the last page, click on the CONDUIT 'volume' link to get a graphical
representation of the CONDUIT data you have received recently:

http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?CONDUIT+chinkapin.cs.indiana.edu

This plot shows that you _are_ getting CONDUIT data.  Since your CONDUIT request
is a smallish fraction of the full CONDUIT datastream, it is most likely that
you are receiving all of the data that you are requesting.  This, in turn,
implies that you are not _processing_ the data you are receiving.  The place
to look now is your LDM pattern-action file entry(ies).

Please send us the pattern-action file you are using to process the CONDUIT
data you are requesting.  In addition, please send us the ldmd.conf file
also.  It is likely that there is a simple mistake in your pattern action
file that is preventing the CONDUIT data you are receiving from being
processed.

Side comment:

I recommend that you upgrade your LDM installation to the latest version
avaiable: LDM-6.7.0.  The reason for this is that this version runs a
sanity check on all pattern-action files being used to see if there are
mistakes.  Ordinarily, mistakes would be logged into the ~ldm/logs/ldmd.log
file, but since your LDM logging is apparently not setup/setup correctly,
we have no way of seeing the error(s).

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: BXH-661016
Department: Support IDD
Priority: Normal
Status: Closed