20030529: McIDAS installation at UFRJ (cont.)

>From:  David Garrana Coelho <address@hidden>
>Organization:  UFRJ
>Keywords:  200305280404.h4S44lLd024244 McIDAS ADDE IDD-Brazil

Hi David,

re: GOES-12 scanning schedules

>       I dont know exactly, but I may guess, since you are in summer
>season right now, that eventually you(you meaning ppl who are in charge of
>this) stop the normal GOES scan in the 10S or 20S latitude (cutting of
>Rio, which is at 23S), and use the satellite "bandwidth" to get some high
>res info of some weather pattern who may be of interest in North America.
>Am I wrong?

You may be correct.  I guess the information I wanted to make you aware
of is that there is not always South American sectors taken at the same
time as the Norh American sectors, and some of the North American
sectors extend down only to just south of Belem (NH) while others
extend down to Rio (NHE).

re: start McIDAS session with mcidas, not mcidasx

>       Uber Doh! Lol
>       I tried this way "mcidas -config" and its ok, but no matter how
>many times I configure it to start automatically MCGUI, if I type only
>"mcidas" it starts the text interface. The window size configuration works
>well, though.

I just took a look and saw that the configuration defaults file,
~mcidas/.mcidasrc was missing the line that would start the MCGUI.  I
don't know why this got deleted; it must be a bug in the GUI startup
routine.

>       Tried all the available options, only bug I found so far is on
>Meteosat7 group, which use the last loop made to fill a loop of it. (If
>you make a loop of Meteosat, the last 2 or 3 images are from the last loop
>you did).

I checked into this and find that the reason for this unexpected/wanted
action is that the server for the ME7 dataset only has one image.
It is a known weakeness of the MCGUI that the loop bounds for an image
are set to the number of images that were desired to be loaded and
not the number actually loaded.  This needs to be fixed.

>Tried observation overlays also. I used the GOESEAST group most
>of the time (because SAMERICA lagged out in time)

I inadvertantly made a typo in the transfer script this morning that
stopped the new sectors for being transferred.  It just goes to show
that I should not make any changes right before heading out of the
door to an all day meeting.  I corrected the problem when I got home
this evening, so the images will be current tomorrow.

>and it went as fast as
>SAMERICA to load, except when I asked for a full resolution VIS sector
>loop of my modeling domain, which took 2~3 mins, tops.

I am suprised by this.  The images served locally should load very
fast.  How fast the images will load from a remote site depends
on the network connection to the remote site and the loading of the
server.  The server for the GOESEAST dataset is nice and fast, so
your only limitation today would have been the network connection.
Your observation is, however, shows me that you appreciate what
we are doing by making interesting data sets available on demand
over the net.

>       Based on experimentation, there is no apparent lag between
>displaying GOESEAST or SAMERICA 4km res images. We maybe (to save time on
>cut jobs on the machine who does it) should try to save locally entire
>GOESEAST images here (or if you think our bandwidth can't hold it, in the
>dedicated server to come) and only use cut jobs for VIS 1km res images.
>Now that MCGUI is working properly, it's very easy to cut/blowdown/blowup
>images.

We can test transferring the entire sectors that are available on the
server for GOESEAST.  If we do this, there will be no need to receive
the RTIMAGES sectors from the IDD since they are essentially the North
American portions of the full sectors anyway, but are available every
half hour when the RTIMAGES images are only being sent every hour.
I always say, the more data the better!

OK, I just set this up on brisa, so we can see the results tomorrow:

1) commented out the request for the North American GOES-12 RTIMAGES
   image sectors from ~ldm/etc/ldmd.conf and then stopped and restarted
   the LDM.  I kept in the IDD ingest of the global Mollweide IR and
   WV images and the Antarctic IR composites.

2) changed the ADDE requests for all IR and 4 km VIS images from
   Southern sectors to full scans.  I left the high resolution
   VIS sectors the same.

>       Now the questions raised section: (:=)

Oh no! ;-)

>       1) Do you think processor/memory size is a factor to be considered
>in the dedicated server to come? I am planning to use it as LDM host,
>MCIDAS host, WWW server (for our lab page and products) and data server
>(gradsdods) of our model outputs.(Internet Network bandwidth is 100Mbits
>on the place where we putting the dedicated server) Please reply with
>suggestions of processor speed and memory size, if needed.=)

Yes, I do think that processor and memory size are factors to be
considered.  If I were setting up a server that will be responsible for
what you list above AND relaying the data to downstream sites, I would
try to get as fast a system with as much memory as possible.  I would
say that a fast dual processor (e.g., 2x2.8 or 2x3.0 Ghz P4) system
with 1 GB -- or more -- of RAM would be good.  And, let's not forget
about disk space.  The more disk you have, the more data you can keep
online.  Just today, we saw system like the one I outlined here for US
$1851, and it has a 250 GB hard disk.  I don't know what the PC prices
are like in Rio, but they may be similar, or, at least, not a lot
more.

As far as operating systems go, one of our university sites has had
exceptional good luck with a dual 1 Ghz system with 4 GB of RAM running
FreeBSD (we see the same thing in our office, by the way).  They are
using this machine for extensive ADDE serving, web hosting, and LDM
services (not relaying, but that is a relatively low overhead).  Our
experience is that FreeBSD is measurably faster than Linux on the exact
same hardware.  It is also more secure than Linux.

>       2) There is any restriction to publish images on the net with
>products generated by MCIDAS? (putting a small logo of our lab in the
>corner along with one of unidata, if needed, and putting a "disclaimer" on
>page telling where the data came from and restrictions of use (if any))

No, there are no restrictions.  We would appreciate it if you reference
your participation in the Unidata community and our funding agency, the
US National Science Foundation.  We don't expect this on the images
themselves (it is nice, however), but we really do like to see it
somewhere on our sites' web pages.  Basically, we sincerely hope that
sites will see their participation in Unidata as a benefit to their
eduational and research endeavors.

>       The questions below only apply if the first answer below is no
>(dont wanna to take more of your time than needed:=)
>
>       3) There is a printable version of MCIDAS manual (pdf or ps)
>where we can search for answers to the questions below?

Yes, but the printable files do not include the Unidata additions to
the manual.  You can get the printable portions from the passworded
Unidata McIDAS FTP site:

machine: ftp.unidata.ucar.edu
user: umcidas
pass: XXXXXX
directory: unix/2002/docs/...

>       4) We plan to keep backups of all data received (observations and
>Imagery). How we should do it? Save the GEMPAK or MCIDAS files?
>(Wondering about file sizes)

As far as imagery goes, both GEMPAK and McIDAS each can use the McIDAS
AREA format.  This is the format of the images that are being saved on
your system right now.  As far as decoded data goes, I would say that
it would depend on which package you use most freuently.  FYI, I will
be working on ADDE servers that will be able to use data decoded into
GEMPAK point (e.g., surface, upper air, etc.) and gridded formats.
This code is in planning, so I can't tell you when it will be ready to
go.

>Where they are kept?

Right now, the imagery is being saved on brisa in subdirectories under
~ldm/data/gempak/images/sat.  The POINT data you were accessing
was all coming over the net from the ADDE server we operate at
the National Science Foundation offices in Arlington, Virginia in
the US.

>How we access old
>imagery/data thru MCGUI? (i.e. choose the time frame)

Time selection is being added to the MCGUI.  It is not available
in the current version (a major limitation).

>       5) Temperature Observation Data is available only in F for
>overlays. There is a way we can change it for Celsius? (Which files we
>should modify?)

I will have to check if an option in a configuration file works
correctly for the current versions of the point data plot routines.  I
think that it does, but I have not tried it for quite some time so I am
reluctant to say anything until I verify.  McIDAS can create
displays in both US and metric units.  The question is whether or
not the MCGUI's execution of those routines will work with the
simple configuration file change.

>       6) I saw some nice topo/IR topo/VIS composite images on Unidata
>site. There is a way to do the same for our products using the available
>TOPO images?

Yes, the creation of the image/topo composites can be done easily
locally.  The process is more than I want to go into in this email
(it is 21:30 now, and I am running out of gas :(), but it is
straightforward.

>       7) How do we add/create map overlays for brazilian state
>boundaries?

I need to locate a database that contains the lat,lon values of
Brazilian states and then create a McIDAS map outline file.  I will
check to see if GEMPAK already has a file that I can use, and if it
does, the creation of the McIDAS file takes only a couple of minutes.

>       I think thats more than enough for now. Our teachers here asked me
>to thank ya once more for your work on MCIDAS installation. All of them
>were (literally) drooling over the monitor when I showed them MCIDAS in
>action this morning. Wild ideas abound. :-)

I am delighted to hear that the data and software will be of use in
your research and educational activities.  Supporting these activities
are what we are funded to do, and my personal interest has always been
extending the sevices we provide to US universities to our collegues in
Central and South America and beyond.

More tomorrow.  Cheers,

Tom
--
+-----------------------------------------------------------------------------+
* Tom Yoksas                                             UCAR Unidata Program *
* (303) 497-8642 (last resort)                                  P.O. Box 3000 *
* address@hidden                                   Boulder, CO 80307 *
* Unidata WWW Service                             http://www.unidata.ucar.edu/*
+-----------------------------------------------------------------------------+