20010323: Unidata application installation at Valparaiso (cont.)

>From: Simon Kissler <address@hidden>
>Organization: Valparaiso
>Keywords: 200103231414.f2NEEhL25449 McIDAS-X ADDE

Simon,

>thanks for your help. I look forward to whatever comes next with getting
>this working. :)

OK, I got back on aeolus a little while ago.  Here is what I have done
to "tune up" your installations of McIDAS and the LDM:

1) correct the crontab entry for scouring McIDAS data

   previous:

   00 21 * * * /home/mcidas/mcidas/bin/mcscour.sh

   changed to:
   00 21 * * * /home/mcidas/bin/mcscour.sh

   After the script ran at 21 local time, I verified that the scouring
   is now working.

2) transferred some updated source code (will be included in the
   next McIDAS addendum, but I figured I would update you now):

   cd ~mcidas/mcidas7.7/src
   ftp ftp.unidata.ucar.edu
     <user> anonymous
     <pass> address@hidden
     get addeinfo.pgm
     get frntdisp.pgm
     get uwgrid.pgm
     get uacross.pgm
     get mapurx.for
     cd  ../data
     lcd ~mcidas/data
     get GLOBAL.BAT
     get STNDB.SITE
     binary
     get AREA9000
     quit

   make mcx

   The McIDAS global topography file, AREA9000, will be useful in doing
   topographic image compositing.

3) setup creation of a couple of GRID files from grids decoded by McIDAS
   XCD.  These will be useful for the Unidata Fkey menu interface.  The
   setup was the addition of a couple invocations to 'mcidas' crontab
   file.

   Commented the crontab entries.

4) turned on compositing of GOES-East and GOES-West imagery.  This was
   done by:

   I)   Changing the read/write permissions on the copies of
        ROUTE.SYS and SYSKEY.TAB located in /var/data/mcidas.  'ldm'
        owned the files, and their permissions were set to rw-r-r when
        they should have been rw-rw-r.

   II)  RELeasing entries in the McIDAS routing table, ROUTE.SYS:

        <as 'mcidas'>
        cd workdata
        route.k REL CI CV CW
        route.k REL N1 N2 N3 N4 N5 N6 N7 N8

   III) <as 'ldm'>
        cd ~ldm/decoders
        <copy needed decoders from /usr/local/ldm-mcidas/bin>
        edit the newly copied batch.k in ~ldm/decoders to set
        McIDAS enviornment variables (just like xcd_run). 

        What is going here is that the ldm-mcidas decoders convert
        imagery in the Unidata-Wisconsin datastream into McIDAS AREA
        file format.  Each product has a product code entry in the
        McIDAS routing table.  The McIDAS routing table can specify
        that after an image is received and correctly decoded, a
        McIDAS BATCH file should be run.  The GOES-East/West VIS, IR,
        and H2O image entries in ROUTE.SYS are setup to do this.  Now,
        when one of these images comes in, a McIDAS BATCH file is
        run to composite them together (GOES-East and GOES-West, that
        is) and with topographic images.

5) updated the FOUS14 (NGM MOS) decoding.  This was needed as a result
   of updating the file STNDB.SITE - a number of FOUS14 stations were
   not correctly marked as FOUS14 stations in the master station
   database, STNDB.CORE.  STNDB.SITE entries are used to add/modify
   station entries for a site.  I am using it to correct for the FOUS14
   station omissions.

   To get the decoding for FOUS14 working with the new station information
   I had to:

   <as 'ldm'>
   ldmadmin stop

   <as 'mcidas'>
   cd /var/data/mcidas
   rm FOUS14.R*
   cd ~/workdata
   bildtext.k INIT FOUS14.RAP   FOUS14.RAT    600 2 C4   2 12 80 FOUS14   X 32
 
   The bildtext.k invocation rebuilt the file FOUS14.RAP.  This file is
   used in the FOUS14 decoding and in the rapid access to FOUS14 data
   (data listing).

   <as 'ldm'>
   ldmadmin start
  
   The decoded FOUS14 data will now be complete, and the lists of FOUS14
   data will work for all FOUS14 stations.

6) modified how the NEXRAD data you are ingesting gets filed.  The setup
   that Anne left you with is not easily used by McIDAS or GEMPAK (all
   files were being put in a single directory).

   I commented out the pqact.conf line for the NMC3 feed type (another
   name for NEXRAD) and entered a new one for NEXRAD.  The data is now
   being filed as follows:

   ~ldm/data/nexrad/ <station id> / <product_type>/ files

   Examples:

   ~ldm/data/nexrad/FTG/NTP/NTP_20010324_0356
   ~ldm/data/nexrad/UEX/N0R/N0R_20010324_0403

    etc.

   Given that you are requesting all of the NEXRAD products from all
   of the NEXRAD sites, I commented out the request for the NEXRAD
   floater sites in ~ldm/etc/ldmd.conf.  While I was in this file, I
   commented the various lines to let the reader know what is going on.

7) setup scouring of the NEXRAD data that you are getting:

   <as 'ldm'>
   cd /var/data/ldm/nexrad
   rm *.raw

   cd ~ldm/decoders
   ftp ftp.unidata.ucar.edu
      <user> anonymous
      <pass> address@hidden
      cd pub/ldm5
      binary
      get prune_nexrad.csh
      quit

   <edit prune_nexrad.csh and set PATH, KEEP, and KEEPLOCAL; PATH
   to include /home/ldm/decoders and KEEP and KEEPLOCAL to keep
   72 (at least 6 hours) of each kind of NEXRAD product for each
   NEXRAD station you are getting (and you happen to be getting ALL
   of the NEXRAD products for ALL of the NEXRAD stations!).  You may
   want to cut down on the number of products that you keep, and
   change LOCALSITE to the local radar site you want to keep KEEPLOCAL
   files for.>

8) setup McIDAS to be able to serve the RTNEXRAD data:

   <as 'mcidas'>
   cd workdata
   cp NNEXRAD.CFG VALPNEXR.CFG
   <edit VALPNEXR.CFG and set DIRMASK and FILEMASK; left IPMASK as it was>

   DIRMASK=/var/data/ldm/nexrad/NIDS/\ID/\TYPE
   FILEMASK=\TYPE_*
   IPMASK=*

A lot of the changes I made above were what you would have had to do
when upgrading to the next McIDAS addendum.  Again, I figured that
since I was already on your machine I would go ahead and make the
changes.

So, what's left?  Setup user accounts on client machines to run McIDAS
and access data from the remote ADDE server on aeolus.  To setup user
accounts on the client machines, you should follow the instructions
in:

Configuring Unidata McIDAS-X Accounts 
http://www.unidata.ucar.edu/packages/mcidas/770/mcx/config_mcx.html

Before logging off, I verified that I can access all of the datasets
through the remote ADDE server on aeolus.  The datasets that aeolus now
hosts are:

RTGRIDS    <- model output data stored in McIDAS GRID files

RTIMAGES   <- imagery in the Unidata-Wisconsin datastream (LDM MCIDAS feed
              type)

RTNEXRAD   <- NEXRAD Level III radar products (you are currently receiving
              all of the data for all of the NEXRADs!  You may want to
              rethink this if network bandwidth gets to be an issue
              (although things seem to be running very nicely right now).

RTPTSRC    <- POINT source data

RTWXTEXT   <- TEXT data

The way to see what kinds of "things" each dataset has is to interrogate
the remote ADDE server.  Here are a few example invocations that could
be run from a user account on a client in the user's McIDAS session Text
and Command window:

DATALOC ADD RTGRIDS  AEOLUS.VALPO.EDU
DATALOC ADD RTIMAGES AEOLUS.VALPO.EDU
DATALOC ADD RTNEXRAD AEOLUS.VALPO.EDU
DATALOC ADD RTPTSRC  AEOLUS.VALPO.EDU
DATALOC ADD RTWXTEXT AEOLUS.VALPO.EDU

DSINFO G RTGRIDS
DSINFO I RTIMAGES
DSINFO I RTNEXRAD
DSINFO P RTPTSRC
DSINFO T RTWXTEXT

SF 1
GRDLIST RTGRIDS/ALL.ALL FORM=FILE
GRDDISP RTGRIDS/ETA PAR=T LEV=SFC MAP=USACONF DAY=#Y TIME=12 FHOUR=0

SF 2
IMGDISP RTIMAGES/GE-IR STA=KSLC EU=IMAGE REFRESH='EG;MAP H'

SF 3
IMGDISP RTNEXRAD/N0R ID=FTG STA=FTG EU=BREF REFRESH='EG;MAP H;BAR'

SF 4
ERASE F
SFCPLOT T USACONF #H

WXTLIST APRO=FOUS14 WSTN=DEN
WXTLIST APRO=SFP WSTN=KBOU

After I create the next addendum, there will be a new Fkey menu that
will provide easy access to the datasets that are now being created.
Even though some work will need to be done to install the new addendum
bug fixes and upgrades, the great majority of the setup has been 
finished on aeolus.  The fact that I can use the aeolus remote ADDE
server for all of the data being decoded indicates that things are
running smoothly.

That's all for tonight...

Tom