20001127: McIDAS XCD decoding problem (cont.)

>From: "David B. Bukowski" <address@hidden>
>Organization: College of DuPage
>Keywords: 200011152022.eAFKMBo01525 McIDAS-XCD DECINFO.DAT xcd_run startxcd.k

Dave,

>I set an account on the machine for you.  please erase the password if
>you reply this message.

OK.

>ssh or secure shell client to port weather.cod.edu to port 443 then
>from weather ssh picard.cod.edu.

Got it.

>I do not have in.telnetd running on picard and strongly discourage
>telnet use on any of the boxes.  So please use an ssh client.  :)

We don't run in.telnetd anymore either.

>I did the steps you posted below and even again, talking with Gil along
>the way checking what he has on his machine (basically same setup on
>data location) and still failed at the MDXX files.  Everything else
>seems to be going in now such as AREA and *.IDX and *.XCD...

Well, I did find some things that I did not like, so I changed them:

<as ldm>

o stopped ldm
o edited .cshrc and removed all references to McIDAS things
    added MANPATH definition; logged out and then back in as 'ldm'
  verified PATH settings so that stuff in the decoders subdirectory
    would be found correctly
o edited etc/pqact.conf and changed xcd_run action for text stuff
    back to match the recommended setup; left the HRS stuff commented out
  removed a comment from the middle of the pnga2area action near the
    top of the file
  commented out the 'lwtoa3' action as it is obsolete
o fixed up MCDATA, MCPATH, etc. definitions in xcd_run
o copied batch.k from the ldm-mcidas src/decode directory to
  ~ldm/decoders; edited this script and set MCDATA, MCPATH, etc.
   in the same way that they are set in xcd_run
o instrumented decoders/xcd_run so that messages would be written
    to stdout
o ran xcd_run MONITOR by hand to see what it had to say for itself;
  it said that no XCD decoders were setup to be run.  Since startxcd.k
  thought that no XCD decoders were to be run, none were and so
  no MD files were created.  Fixed this while running as 'mcidas'
  (see below) and then uninstrumented xcd_run and restarted the LDM

<as mcidas>

o edited .cshrc and fixed the definitions for MCDATA, MCPATH, etc
  commented out extra defintions for MCDATA, MCPATH, etc. near the
  end of the file
o sourced .cshrc before proceeding
o removed all non AREA files from /home/data/mcidas
o verified REDIRECTions you setup; all look good
o noticed that there were no *.RAP files in /home/data/mcidas.  This
  might have been caused by you running BATCH XCDDEC.BAT before
  having setup REDIRECTions for the RAP files
o removed /home/data/mcidas/AREA0000; this is an illegal file name
  so it pointed to a situation that was not working correctly
o resetup the XCD decoding stuff:

  cd ~/workdata
  batch.k XCD.BAT
  batch.k XCDDEC.BAT
  
  after this, I do see the various *.RAP files in /home/data/mcidas
o since startxcd.k (run form the MONITOR action of xcd_run) was
  saying that no decoders were supposed to be active, I reset the
  decoding setup by successive runs of DECINFO:

  cd ~mcidas/workdata
  -- deactivate --
  decinfo.k SET DMSFC INACTIVE
  decinfo.k SET DMRAOB INACTIVE
  decinfo.k SET DMSYN INACTIVE
  decinfo.k SET DMMISC INACTIVE
  -- activate --
  decinfo.k SET DMSFC ACTIVE
  decinfo.k SET DMRAOB ACTIVE
  decinfo.k SET DMSYN ACTIVE
  decinfo.k SET DMMISC ACTIVE

  After doing this the restart of the LDM correctly started up the XCD
  decoders, and MD files are started being produced.  This was
  verified by an MDU LIST invocation:

  cd ~mcidas/workdata
  mdu.k LIST 1 100
  MD#  CREATED SCHM PROJ  NR   NC     ID   DESCRIPTION
 ----- ------- ---- ---- ---- ---- ------- -----------
     3 2000333 ISFC    0   72 4500 2000333 SAO/METAR data for   28 NOV 2000
    13 2000333 IRAB    0    8 1300 2000333 Mand. Level RAOB for 28 NOV 2000
    23 2000333 IRSG    0   16 6000 2000333 Sig.  Level RAOB for 28 NOV 2000
    32 2000333 ISHP    0   24 2000 2000332 SHIP/BUOY data for   27 NOV 2000
    33 2000333 ISHP    0   24 2000 2000333 SHIP/BUOY data for   28 NOV 2000
    52 2000333 SYN     0    8 6000 2000332 SYNOPTIC data for    27 NOV 2000
    53 2000333 SYN     0    8 6000 2000333 SYNOPTIC data for    28 NOV 2000
    63 2000333 PIRP    0   24 1500 2000333 PIREP/AIREP data for 28 NOV 2000
 -- END OF LISTING

o I noticed that the McIDAS file routing table, ROUTE.SYS, in /home/data/mcidas
  had gotten corrupted.  This was caused by the LDM pqact.conf entry
  for pnga2area decoding allowing decode of the CIMSS imagery that
  was added to the datastream back at the end of June.  The routing table
  had not, however, been update to handle the CIMSS products. The fix
  for this was done as the user 'mcidas' while the LDM was turned off:

  cd /home/data/mcidas
  rm ROUTE.SYS
  cp ~/workdata/ROUTE.SYS .
  cd ~/workdata
  ftp ftp.unidata.ucar.edu
     <user> anonymous
     <pass> full email address
     cd pub/mcidas
     get CIMSS.BAT
     quit
  batch.k CIMSS.BAT

  The CIMSS.BAT file added CIMSS entries to the McIDAS-X 7.6 version
  of ROUTE.SYS.  At this point, the LDM could be turned on, and the
  CIMSS products would be filed correctly.

o I also noticed that your system was not creating the GOES-East/West
  composite products (product codes CI, CV, and CW in the routing
  table).  This was due to those products being SUSpended in the
  routing table.  I decided to run their creating back on:

  cd ~/workdata
  route.k REL CI CV CW

  The compositing was doable by virtue of:

  o these routing table changes
  o my copying the ldm-mcidas/src/decode version of batch.k to
    ~ldm/decoders and editing it to set MCDATA, MCPATH, etc. in
    the same way as was done in ~ldm/decoders/xcd_run

o I also noticed that the image/topography composites are not being
  created, but I did NOT turn these on.  If you want these created,
  the procedure is the same as for the GOES-East/West composites above:

  cd ~mcidas/workdata
  route.k REL N1 N2 N3 N4 N5 N6 N7 N8

  I leave this up to you to do if you want.
   
** Wrap up **

The problem was that startxcd.k (the XCD data monitor/decoder
supervisory routine) did not "think" that decoders were setup to be run
(this information is set in the file ~mcidas/workdata/DECINFO.DAT).
After INACTIVEaging and reACTIVEating the data monitors for surface
(DMSFC), raob (DMRAOB), synoptic (DMSYN), and miscellaneous (DMMISC)
the expected MD files are being produced and are usable.

A couple of other notes:

o the default shell for the user 'mcadde' is currently set to be /bin/bash.
  This should be changed to be /bin/false (done by 'root') so that the
  'mcadde' account is not a login account.

o ADDE datasets have not been setup.  You would do this as the user
  'mcidas' as follows:

  cd workdata
  batch.k LSSERVE.BAT

  This is after you have setup LSSERVE.BAT as a copy of DSSERVE.BAT
  in the ~mcidas/data directory.

After making these changes, other machines can use picard as an ADDE
server (cool).

>Just the MDXX files are not being produced... I fixed the routing via
>LOCAL.NAM and did the steps with ldm turned off and everything else got
>fixed, except for the MDXX files :/  Well if you could figure it out
>please mail me what you did or what I missed.

Most things were correct.  The changes I made to the ~ldm/.cshrc file
were needed: we try to NOT define any McIDAS stuff in the LDM environment.
Instead, we define needed McIDAS things in scripts that are run in
the LDM environment (e.g., xcd_run, batch.k, etc.).

>Thanks again

You are welcome.

>p.s.  hope you had a good Thanksgiving w/e

It was grand, thanks.

Tom

>From address@hidden Tue Nov 28 07:40:00 2000
>Subject: Re: 20001127: McIDAS XCD decoding problem (cont.) 

Thank you very much Tom, your awesome :)
-dave

-------------------------------------------------------------------------------
David B. Bukowski               |email (work):          address@hidden
Network Analyst                 |email (personal):      address@hidden
College of Dupage               |pager:                 (630) 266-7775
Glen Ellyn, Illinois            |work phone:            (630) 942-2591
http://www.cod.edu/             |ICQ#:                  46516655
-------------------------------------------------------------------------------