[McIDAS #NNA-627336]: Re: 20060626: survey of platform use for McIDAS (cont.)

Hi Kwan,

re: 
> I forgot to mention that the McIDAS2005d was compiled here
> using the gcc/g77 option.

Thanks for letting me know.

> Now I see you are working on McIDAS2006.  I think I can
> wait until the entire package is fully ready before
> installing it.

OK.

re:
> If you are not too busy working on the new release, I'd
> like to ask you about an abnormal behavior that has
> negatively affected the performance of McIDAS here at
> CCNY.  I have written shell scripts that run McIDAS
> commands, and they were put into crontab so that they can
> be executed hourly.  The purpose is to generate hourly
> .gif files of GOES images overlaid with RTPTSRC data for
> different geographical scenes.  The most complex scripts
> can take several minutes to finish.  Here is the problem.
> Early this month, there was a school-wide server
> maintainence which caused an outage of the internet
> connection for 30 hours or so.  The machine here that runs
> McIDAS (halo) stayed on for the entire outage period.
> What I found was that because of the outage, the cron
> jobs were backed up for 30 hours.  When the internet came
> back on, they were "fired off" and executed at the same
> time.  However, not all of the delayed cron jobs were able
> to finish entirely.  The results were unpredictable.
> Sometimes the cron jobs appeared to hang for 30 minutes
> before it could finish.  Even though they could finish,
> some of the .gif file showed unpredictable problems.  Some
> did not have the right image enhancements, some images
> were not remapped to the correct projection, and some did
> not have an image displayed at all.  Similar problems
> occurred about this time last year when a "Resource
> temporarily unavailable" error frequently occurred as
> McIDAS image commands were executed (e.g. IMGDISP,
> IMGREMAP).  (I did not see it happened this time though.)
> I tried to reboot the machine, cleaned the McIDAS memory
> objects, but could not make the error go away.  In fact,
> the problem slowly got worse.  But the turning point came
> when the McIDAS 2005 was released, and was installed.  The
> "Resource temporarily unavailable" error went away
> (although it appeared to take a couple of days to do
> that), and McIDAS appeared back to full function.  The
> same thing appeared to have happened when I decided to
> download and install the McIDAS 2005 update yesterday.  It
> appeared to have cured the problem!  But I don't
> understand it.  It seems to me it involves both the OS and
> McIDAS.  Do you have any explanations?

This behavior is puzzling.  My guess would be that more than one scripts ends 
up running
at the same time, and they are somehow intefering with each other.  This is 
most likely
the case when the network came back online after being down for 30 hours.

It would be wise to add some additional logic to your scripts to do some quick 
tests
of availability of various things before proceeding to actual processing:

- system resources (memory, disk, CPU, etc.)
- network connectivity
- McIDAS ADDE server

I don't have any quick and easy things for you to implement.  I would approach 
the problem
by taking one script and make it smarter and smarter to the point that it will 
gracefully
exit when needed.

Another approach -- which will take a lot of understand of how McIDAS really 
works -- is
to have a running McIDAS session which the scripts attach to and do their work. 
 This would
be tricky to get right, so I would only try it after other approaches fail.

Finally, McIDAS has a scheduler facility that can be used instead of Unix's 
cron.  The
scheduler makes running McIDAS procedures (BATCH and MCBASI scripts) easier, 
but it
requires that a McIDAS session be running all of the time.  The nice thing 
about this
approach is that each invocation of a time-based script does not need to create 
new
shared memory segments, so it would be less demanding on your computer's 
resources.
Please take a look at the scheduler facility (SKE, SKU, etc.) to get some ideas.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: NNA-627336
Department: Support McIDAS
Priority: Normal
Status: Closed