19990927: ADDE performance

>From: Mark Tucker <address@hidden>
>Organization: Lyndon State
>Keywords: 199909271950.NAA15960 McIDAS ADDE performance

Mark and MUGsters,

I am CCing this note about ADDE performance to SSEC for their review.

>We finally put ADDE to the test today in the classroom and ran into a
>performance problem.  There were about 20 students running contours on
>upper air data via the F-key menu at the same time.  "mcserv" did not seem
>to be able to keep up with the requests.  Some machines took several
>minutes to complete the plot.

The pervormance will look like 20 people doing data access on the machine
at the same time.  This is the first real feedback I have gotten about
an ADDE stress test.  I will relay the information to SSEC for their
edification, so any information you can provide will be greatly
appreciated.

>There were _many_ entries in the log files for the mcserv requests:
>
>Sep 27 14:31:53 cirrus tcplogd: mcserv connection attempt from
>metlab05.lsc.vsc.edu [155.42.21.119]
>Sep 27 14:32:03 cirrus tcplogd: mcserv connection attempt from
>metlab05.lsc.vsc.edu [155.42.21.119]
>Sep 27 14:32:13 cirrus tcplogd: mcserv connection attempt from
>metlab05.lsc.vsc.edu [155.42.21.119]
>Sep 27 14:32:26 cirrus tcplogd: mcserv connection attempt from
 ...

>Our server is a PII-400 with 512MB RAM.  The server and lab are all
>connected with 100mb ethernet and were not under heavy load at the time.
>Are there any changes I can make to mcserv that would speed up the
>response times?

I don't think so, but I will need to get some guidance from SSEC.

>I will be upgrading Cirrus with a second CPU later next
>week but I'd like to have this resolved before then if possible.

OK, I am CCing this message to SSEC to see if there are any performance
tuning parameters that should be adjusted; I don't think that there
are but...

>Also, I am planning on upgrading our server's install of McIdas to 7.6
>either later this week or next week.  Will support be available around
>that time frame (should things break down on me)?

Not really.  I will be involved in the User Committee meeting on both
Thursday and Friday.  I leave for Madison, WI on Sunday night to attend
the annual McIDAS User's Group meeting.  I will try to check in with
email, but I can't promise as high a response level as I would like.  I
can advise you to go ahead and build 7.60 and test it in place.  This
will at least lessen the number of things to do before switchover.

The thing that you will have to worry about, however, is the switch
over in versions of XCD.  Depending on which version of McIDAS you are
upgrading from (i.e.  7.4 or 7.5), you will run into different MD file
schemas.  This means that existing output data files for the day will
have to be deleted, or that you will need to switchover as close to 0Z
as possible (so that the new day's files will not yet be created with
different DAY formats internally.

I will be back from WI a week from Thursday after midday (the plane
lands in Denver in the morning).  Is there any chance you can hold
off on the upgrade until then?

Tom

>From address@hidden  Tue Sep 28 08:36:40 1999

Actually, the performance is now _much_ worse than last year when we had
~15 users requesting data through redirections to NFS mounted drives.
Our entire lab has been upgraded so this is especially surprising.  Last
year we were running McIdas 7.1 on Pentium 133's, 32mb RAM with 10 base-T,
as opposed to this year's lab is using McIdas 7.6 on P400's with 128mb RAM
and 100mbit networking.  All of our other applications are noticeably
faster, Garp in particular.
   When a McIdas session is run individually, the ADDE server's response
is quite fast.  I was hoping that there was a way of spawning addintional
processes or threading to manage multiple requests.  These types of loads
are very typical of our usage here and currently it is almost unusable in
a classroom situation.

There are a couple of other labs scheduled this week that should make
similar usage of the ADDE server.  I will be watching things more closely
to see if I can find any other factors that may be causing or
contributing to the bottle neck.  If there is anything in particular you'd
like me to take note of, let me know.
 
> I will be back from WI a week from Thursday after midday (the plane
> lands in Denver in the morning).  Is there any chance you can hold
> off on the upgrade until then?

I can wait until late next week.  In fact, that may work out well for us. 
I believe we have a long weekend coming up then, so I can afford to have
a little data loss during that time if necessary.  I have built McIdas 7.6
successfully for our lab machines, so I'm confident about the that portion
of the install.  My main conserns are the XCD decoding, which you
mentioned, and ADDE serving.  I'll do some more reading until then.

Thanks.

Mark Tucker
Information Technology
Lyndon State College
address@hidden



>From address@hidden  Tue Sep 28 14:33:01 1999

On Mon, 27 Sep 1999, Unidata Support wrote:

> The pervormance will look like 20 people doing data access on the machine
> at the same time.  This is the first real feedback I have gotten about
> an ADDE stress test.  I will relay the information to SSEC for their
> edification, so any information you can provide will be greatly
> appreciated.

We had some additional usage of McIdas today and I kept a closer watch on
the server during the requests.  Again, it responded very slowly to the
requests.  What I did notice was that there were no associated prcesses
coming to the top of the process list using "top".  Overall CPU usage did 
not max so I do not believe that other processes were slowing things
down.  I watched the network utilization and it was busy but nowhere near
capacity.  
The more I study this the less sense it makes.  I hope this is helpful.

Mark Tucker
Information Technology
Lyndon State College
address@hidden