20051209: McIDAS: Overlaying images

>From: "Corcoran, William T" <address@hidden>
>Organization: Missouri State University
>Keywords: 200512092242.jB9MgF7s024110 McIDAS

Hi Bill,

>My question for today is: "how can I effectively overlay radar data onto
>the topographic AREA bases?"

Ready...

>Here are the problems:
>
>1.     Since the end of November, the ADDE data set MDRTOPO is not
>being created on any of the servers that I can find.  I had been using
>ADDE.UCAR.

I just found and fixed the problem with MDRTOPO creation on
atm.geo.nsf.gov: The McIDAS ROUTE PostProcess BATCH file that was doing
the compositing was also creating a display for the NSF weather web
page, and the softlink to the directory to which GIF images were being
written was removed by someone other than myself.  Because the writing
of the GIF image (using FRMSAVE) was failing, none of the other actions
in the BATCH file were getting run.  I changed the REDIRECTion for .GIF
images in the 'mcidas' account on atm to point at the correct location
for the web images and then the compositing started working again.  I
then saw that the date and time in the resultant composite image was
not being updated with the date and time from the original MDR image.
I traced this down to a subtle change in how NORTEL runs.  Instead of
mucking with NOTEL, I found it was easier/quicker to make a change in
NORTMAPR (NORTMAPR calls NORTEL).  The most recently received MDR image
was successfully composited with the TOPOGRAPHY image (AREA9018) and
the time in the composite is the same as that in the MDR image.

I also logged onto adde.ucar.edu to see what is happening there, but I
havn't found the problem yet.  Because of this, I recommend that you
"point at" atm.geo.nsf.gov for the RTIMAGES dataset for now:

DATALOC ADD RTIMAGES atm.geo.nsf.gov

>For a couple of days (Dec 1), I was able to grab it from
>NIU, but even that one seems defunct now.

I'll take a look.

>BTW, all the RTIMAGES/combo
>products seem defunct (GEW, TOPO combos etc.)  These old products had a
>nice, easy to look at set of radar echoes.  The last one I could get is
>viewable as:
>http://cumulus.missouristate.edu/home/mcidas/data/mc/Radar_US.gif.

That composite is once again available on atm.geo.nsf.gov.  I will
be investigating the failure on other ADDE servers and fixing
problems as I find them.

>2.     For a day or two at the same time, the UCAR ADDE server was
>serving a similar image, but the radar echoes had a strong cross-hatch
>pattern that is visually unpleasing...sort of like Apple II graphics.
>An IMGDISP RTIMAGES/MDRTOPO should show you what I mean...Frame label
>says it is the topography image from 1996.
>http://stratus.missouristate.edu/home/mcidas/data/mc/Radar_US.gif

I am unable to look at this image.

>  has my current image now.

The bad date (1996) in the composite image is the thing I fixed in
NORTMAPR on atm.  The problem on adde.ucar.edu has me baffeled at the
moment -- the ROUTE PostProcess BATCH files are not being kicked off at
all.

>3.     I have used IMGCOPY, IMGREMAP, NORTHMAPR, NORTEL and can
>simulate the process a number of different ways using either MDR or
>NEXRCOMP stuff (thanks for pointing that out), I guess, but I always end
>up with the radar echoes being the ugly cross-hatch on the final
>product...it looks to arise from the NORTEL or NORTMAPR process.

NORTMAPR is used directly in the ROUTE PostProcess BATCH files that I
send out with the distribution.  Here is the infocation for the MDR
compositing:

REM Run NORTMAPR to remap the image and composite the remapped copy with a
REM a topography image
NORTMAPR 9018 %2 9993 N5 NONE 0 9018 MAPAREA=9974 DEV=NNN
          ^   ^   ^   ^    ^  ^  ^            ^    ^_____ run silently
          |   |   |   |    |  |  |            |__________ scratch area for 
bkgmap
          |   |   |   |    |  |  |_______________________ VIRT file with map
          |   |   |   |    |  |__________________________ output AREA number
          |   |   |   |    |_____________________________ band
          |   |   |   |__________________________________ routing product code
          |   |   |______________________________________ scratch area number
          |   |__________________________________________ MDR area number
          |______________________________________________ topo area number

An example invocation would be :

NORTMAPR 9018 207 9993 N5 NONE O 9018 MAPAREA=9974

This will composite the MDR radar echos in AREA0207 with the topography
image AREA9018.

I will be adding the fix I made to NORTMAPR to a v2005 addendum as soon
as I find and fix the problem on adde.ucar.edu (and NIU's machine).

>As of
>this time, I have just dropped back to copying a NEXRCOMP image to the
>local data set, then using the NORTMAPR command as in the MDR.BAT PP
>file.

There is another way.  See below.

>the radar echoes aren't as visually "pleasing" mapped on top of
>the TOPO underlayment as they are in the original data.

They never are since the NORTEL process only includes every Nth pixel
from the image to be composited.

>(Unless you can
>think of a decent background besides a black screen.)  I dunno...I'm
>dangerous with this stuff.  Just wondered if you'd care to suggest a
>path I can take here.  I'm afraid I might be bumping into stretch table
>issues.

Here is the other way:

- create a mini McIDAS session that supports 128 image levels
  and 32 graphic levels

- display an image you want the radar echos to overlay (e.g., the
  same TOPO image, an IR, etc.)

- use the FRMOLAY command to overlay image value ranges from a NEXRCOMP
  radar image

Try the following script:

----------------------- overlay.sh - begin --------------------------------
#!/bin/sh -f

#--------------------------------------------------------------------------
#
# Name:     mcrun.sh
#
# Purpose:  Bourne shell script used to run a sequence of McIDAS commands
#           outside of a McIDAS-X session.
#
# Syntax:   mcrun.sh
#
# Notes:    Environment variables that need to be set for McIDAS commands
#           to run outside of a McIDAS-X session:
#
#             MCHOME - the home directory for the user running mcrun.sh
#
#             MCDATA - the user's McIDAS-X working directory.  NOTE: this
#                      directory should also be the first directory
#                      specified in MCPATH.
#
#             MCPATH - colon separated list of directories that contain
#                      McIDAS data files, ancillary data files, and help
#                      files
#
#             PATH -   a colon separated list of directories that
#                      Unix will search when looking for executables.
#                      NOTE: the ~mcidas/bin directory should be first in
#                      the list of directories that will be searched.
#
#             LD_LIBRARY_PATH - the search path for sharable libraries;
#                      this should be the same search path as the one
#                      used by the McIDAS session
#
#             MCBATCHFILE.BAT - the name of the McIDAS BATCH file that
#                      you want to run.
#
#           This is a Bourne shell script.  All statements must conform to
#           Bourne shell scripting syntax.
#
#           The example PATH and LD_LIBRARY_PATH definitions are set
#           for Sun Solaris.  AIX and HP-UX users need to change
#           LD_LIBRARY_PATH everywhere in this file to what is appropriate
#           on their systems
#
#             AIX:  LD_LIBRARY_PATH -> LIBPATH
#             HPUX: LD_LIBRARY_PATH -> SHLIB_PATH
#
#
# History:  20000607 - Written for Unidata McIDAS-X, -XCD 7.70
#           20040720 - Changed example display to run MAP and BAR separately
#
#--------------------------------------------------------------------------

# Define macros needed for McIDAS-7.X environment
#
# MCHOME - set this to the HOME directory of the user McIDAS was installed as
# 
# MCDATA - this will be set according to who is running the script:
# 
#          user       MCDATA
#          ----------+---------------------
#          mcidas     $HOME/workdata
#          other      $HOME/mcidas/data
#
# MCPATH - this is a colon-separated list of directories McIDAS will search
#          when looking for anciallary data (e.g., map databases, enhancements,
#          stretch tables, etc.) and data (e.g., AREA, MDXX, GRID, and TEXT
#          files.
#
#          The $MCDATA directory should _always_ be the first MCPATH directory!
#
# MCLOG  - the name of the file to log output to; defaulted to
#          $MCDATA/mcrun.log
#
# MCTABLE_READ - a quoted, semi-colon-separated list of directories to search
#          for McIDAS-X DATALOCation information.  This is typically
#          MCTABLE_READ="$MCDATA/MCTABLE.TXT;$MCHOME/data/ADDESITE.TXT"
#          but can be expanded to suit the user's needs.
#
# PATH   - Unix shell search PATH with $MCHOME/bin being the first directory.
#
# LD_LIBRARY_PATH - most likely not needed, but put in for future use
#

SHELL=sh
export SHELL

#
# First, define MCHOME as the HOME directory for the user under which
# McIDAS-X is installed.
#

MCHOME=/home/mcidas

#
# Find out who is trying to run the script.  Don't allow 'root'!
#

case `id` in
'uid=0('*)
        echo "ERROR: cannot execute as user 'root'"
        exit 1
        ;;
uid=?'(mcidas)'*        |\
uid=??'(mcidas)'*       |\
uid=???'(mcidas)'*      |\
uid=????'(mcidas)'*     |\
uid=?????'(mcidas)'*    |\
uid=??????'(mcidas)'*   |\
uid=???????'(mcidas)'*  )
        MCDATA=$HOME/workdata
        ;;
*)
        MCDATA=$HOME/mcidas/data
        ;;
esac

#
# Set MCPATH using MCDATA and MCHOME
#

MCPATH=${MCDATA}:$MCHOME/data:$MCHOME/help

#
# Define MCLOG to be $MCDATA/mcrun.log
#

MCLOG=$MCDATA/mcrun.log

#
# Set MCTABLE_READ based on the existence of $MCDATA/MCTABLE.TXT.  Users
# should modify this section to match their setup if it does not follow
# the recommendations in the Unidata McIDAS-X online documentation
# for configuring user accounts.
#

if [ -f "${MCDATA}/MCTABLE.TXT" ]; then
  MCTABLE_READ="${MCDATA}/MCTABLE.TXT;${MCHOME}/data/ADDESITE.TXT"
else
  MCTABLE_READ="${MCHOME}/data/ADDESITE.TXT"
fi

#
# Setup PATH so that the McIDAS-X executables can be found
#

PATH=$MCHOME/bin:$PATH

#
# Set LD_LIBRARY_PATH to include all directories (other than those searched
# by default) that are needed to be searched to find shared libraries.
#

LD_LIBRARY_PATH=$MCHOME/lib:/usr/local/lib:/lib:/usr/lib:/usr/ucblib

#
# Send all textual output to the log file
#

exec 2>$MCLOG 1>&2

#
# Export the environment variables specified above
#

export MCHOME MCDATA MCLOG MCPATH MCTABLE_READ PATH LD_LIBRARY_PATH

#
# Now run 'mcenv' to create a McIDAS-X environment in which the McIDAS
# binary commands will run.
#
# It is the user's responsibility to put the commands they want to run
# in the following.
#

cd $MCDATA
mcenv -i 128 -g 32 -f 1@600x800 << EOF

# put McIDAS-X commands you want to run here, one command per line.

erase.k
imgdisp.k GINIEAST/GE4KIR STA=KSTL MAG=1 EU=IMAGE SF=YES
map.k H
bar.k
frmolay.k NEXRCOMP/1KN0R-NAT UNIT=BRIT 
RAN='3:85,86:96,97:107,108:119,120:130,131:141,142:153,154:164,165:175,176:187,188:198,199:209,210:221,222:232,233:243,244:255'
 INT=0 GU=RADAR16_32
frmsave.k 1 radarolay.gif

# done
exit

EOF

# Done
exit 0
----------------------- overlay.sh - end --------------------------------

Of course, you can change the background image to the topography image
of your choice.  All you need to do is IMGDISP the image you want making
sure to use the appropriate enhancement.

>So, it's kind a vague question, but if you look at the old image on
>cumulus vs. stratus the difference is obvious...

The question wasn't vague at all.  You now have a couple of options:

- use the MDR topo composites from atm.geo.nsf.gov (until I fix adde.ucar.edu)

- use the NORTMAPR invocation I showed above (better after I release
  the mod I made to NORTMAPR today)

- use IMGDISP and FRMOLAY to overlay radar echos on top of any image
  you choose.

>Thanks.

No worries.  Sorry for the problems...

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.

>From address@hidden  Mon Dec 12 07:41:48 2005

>Tom:

>Just a very short note.  THANKS!  Talk about support!!!

>You were unable to reach stratus.missouristate.edu because I've been
>powering-down when I go home at night.  Sorry.  I won't trust the
>hackers until I have a good backup of a stable, clean system and some
>confidence in my security.

>Thanks again.

>Bill

>Dr. William T. Corcoran  http://cumulus.smsu.edu
>(417) 836-5781 
>Dept. Geography, Geology, and Planning
>Missouri State University
>901 S. National
>Springfield, MO 65897