20021206: McIDAS: Directory overflow

>From: Michael Keables <address@hidden>
>Organization: DU
>Keywords: 200212062000.gB6K0e402340 McIDAS-XCD mcscour.sh

Hi Mike,

>The /data directory on cyclone is full again...I've gone through it but
>can't seem to find out where all of the space went. Part of the problem is I
>don't know what the typical file sizes should be for the various data
>products.
>
>Would you please take a look at cyclone when you get a chance and see what's
>going on?

Sure.  I jumped onto cyclone and started looking in /data/ldm/mcidas.
Sure enough, the files were eating up all of the filesystem (8 GB).
The first thing I did was to manually delete three really old .XCD
files.  This freed up a few hundred MB of space, but it was not the
problem.  The next thing I did by hand was to delete all GRID* files
that were of zero length.

Next, I became 'ldm' and took a look at the crontab to see what was
being run for data scouring.  Crontab shows that ~ldm/decoders/mcscour.sh
is to be run, so I took a look in it.  The first thing I noted was that
mcscour.sh had not been updated to scour all of the grids that
McIDAS-XCD produces.  Since the McIDAS GRID files are huge, this as
probly your problem.

I edited ~ldm/decoders/mcscour.sh and:

changed:

qrtmdg.k GRID 5001 6000 1

to:

qrtmdg.k GRID 5001 6400 1

I then ran mcscour.sh by hand:

<as 'ldm'>
~ldm/decoders/mcscour.sh

After this ran, you now have a lot (like 6 GB) of free space in /data.
So, it looks like the problem was that the GRID scouring entry in
mcscour.sh needed updating to match the GRID files that the v2002 XCD
GRIB decoder produces.

>Thanks!

No worries.  You should be OK now.  Please keep an eye on your disk
space for a couple of days to make sure that things are working
correctly.  In particular, you should look at the mcscour.sh log file:

~mcidas/workdata/scour.log

This will tell you when mcscour.sh is run from cron (by the file's time
stamp), and what it deleted.

Tom