[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20011025: McIDAS problems at Cornell
- Subject: 20011025: McIDAS problems at Cornell
- Date: Fri, 26 Oct 2001 12:28:59 -0600
>From: Venkat Manakkal <address@hidden>
>Organization: Cornell
>Keywords: 200110252100.f9PL0V109149 McIDAS-X
Venkat,
I was set to respond to your first inquiry when I saw that you had
found and solved your problem. I am replying to the message just so
that your experience can be added to the body of McIDAS information
that is available to all online.
>Hi there (I've heard that I should talk to Tom....)
>
>I'm the new sys admin at the Cornell EAS department. I came in here to get
>the Linux boxes straightened out. The day I joined, the lab machine running
>LDM and McIDAS was hacked into (visibility.cit.cornell.edu) , and I fixed
>that and it is back in action (somewhat: the LDM data feeds in, but McIDAS
>cannot see the data that has come in since the machine got hacked into).
This actually does not sound like the problem you found and fixed.
>I also set out to duplicate the configuration with a fresh install of
>everything from scratch. Starting from installing RH7.1, hardening the
>install to getting LDM, XCD, Gempak and McIDAS to run took a few days.
This was probably because you were new to the various packages. I
think you will find that maintenance and/or fresh reinstalls will go a
lot more quickly in the future now that you have been to the various
ins and outs of the various packages.
>I have the machine (sealevel.cit.cornell.edu) running pulling data with the
>same pqact.conf as on visibility. However, I have the same problem - how do
>I get McIDAS to see the new area and data files that are being pulled in? I
>have tried playing with LOCAL.NAM but to no luck.
For completeness, the you can set this up in two different ways:
1) setup the 'mcidas' account on a primary machine and configure it to
be able to access all of the data. After doing this, setup the
ADDE remote server on the same box (setup is done for user 'mcadde',
but the work is done in the 'mcidas' account), and then have all users
"point" (via McIDAS DATALOC commands) to that server for their data.
2) setup each account that will be using McIDAS to be able to dirctly
access McIDAS data files. The procedure is the same as for the 'mcidas'
account, but it has to be repeated for each user.
The basic difference is that option 1) concentrates the setup activities
to the user 'mcidas'. This makes it the setup for other users extremely
simple. In fact, a complete setup of the 'mcidas' account would totally
eliminate setup for everything except how the other user wants their
session to look (number and size of frames, etc.). We can pursue this
issue further if you like; please let me know.
>I have detailed install notes in the attached text file. I suppose a quick
>phone conversation sometime tomorrow would be a good start.
>
>My number at Cornell is 607-255-5589, although my voice mail is still not
>reset. I can call you folks on my cell phone if necessary.
Our preferred method of communication is by email. The primary reason
for this is that all email exchanges are logged into our inquiry
tracking system which is then indexed nightly and made available for
online searches by any interested party.
>Best regards,
>
>>From address@hidden Fri Oct 26 10:01:28 2001
>>Subject: All set
>
>Hello there,
>
>Its fixed now, and we are able to view our data feeds. As user mcidas, when
>I tried the config by invoking mcidas the script would fail, so I had tried
>it by calling mcidasx directly which did not actually configure things.
Good thinking!
>Apparently after looking at the source of the mcidas script that invokes
>mcidasx, I realized that for some reason there was a .mcidasrc file in the
>home directory that was present with nothing in it (size 0, don't know why
>it was there, I did not create it).
This is a new one for me!
>One removed, mcidas could be invoked
>normally and all the configs worked out. Thanks for your anticipated help!
Wonderful. Did you notice in the 7.8 distribution that one can run:
mcidas config
and then select automatic startup of the MCGUI AND save configuration
changes (changes are saved to ~/.mcidasrc)? When ever a user wants to
reconfigure, s/he only needs to run 'mcidas config' and make sure that
the changes are saved. From that point on, running 'mcidas' will
startup up the environment that they specified. The reason I am
hitting on this is that the MCGUI is the interface that I am
concentrating my efforts on. The old Fkey menu interface is being
mothballed (i.e., no new development), and the command line only
approach puts most users off of McIDAS.
>Also, I plan to repeat the entire configuration process on another hard
>drive, make notes again and make it available (this time with RH 7.2). I'll
>send those along when I have them. (This will have to wait for a few days).
Please let me know how this goes. We are just loading RedHat 7.2 on a
test system, so other's experiences will be useful.
>Best regards,
>
>---Venkat.
>
>
>RH 7.1 netboot.img install from
>www.ibiblio.org/pub/Linux/distributions/redhat/7.1/en/os/i386
>text
>IP nos:
>
>IP: 132.236.186.24
>NM: 255.255.255.128
>GW: 132.236.186.1
>NS: 132.236.56.250
>
>add partitions
>
>swap 512MB
>/boot 50 mb
>/ 250MB
>/var 600MB
>/tmp 1000MB
>/usr 3000MB
>/home rest 11947MB
>
>format all partitions
>
>defaults,
>security high,
>generic 3 button ps2 (default)
>
>root pass, add venkat (sys admin)
>package selection:
>
>* Printer
>* X Windows System
>* Gnome
>* KDE
>* Mail/www/news tools
>* Networked Workstation
>* Authoring/Pub
>* Emacs
>* Kernel Dev
>* Utilities
>
>Don't bother with individual packages.
>
>Finds correct card: Matrox
>
>Proceeds to format, install... install size 1289MB
>
>At end of install, choose X settings: 1280x1024 24 Bit color, use
>graphical login
>
>Hardening install:
>
>Hardening RH7.1:
>
>remove (using /usr/sbin/setup) following system services:
>autofs
>sendmail
>lpd (since lpd was hacked on another system)
>
>xinetd has not been installed. /etc/xinetd.d empty
>
>Install openssh-sever
>
> rpm -i
> http://www.ibiblio.org/pub/Linux/distributions/redhat/7.1/en/os/i386/Re\dHat/
> RPMS/openssh-server-2.5.2p2-5.i386.rpm
>
>Edit /etc/ssh/sshd_config:
>
>Change PermitRootLogin to no
>
> /etc/init.d/sshd start
>
>Install /etc/sysconfig/ipchains with the following:
>
>------
># Firewall configuration written by lokkit
># Manual customization of this file is not recommended.
># Note: ifup-post will punch the current nameservers through the
># firewall; such entries will *not* be listed here.
>:input ACCEPT
>:forward ACCEPT
>:output ACCEPT
># Allow all traffic on lo
>-A input -s 0/0 -d 0/0 -i lo -j ACCEPT
>
># Allow local subnet traffic
>-A input -s 132.236.186.0/24 -p tcp -j ACCEPT
>-A input -s 132.236.186.0/24 -p udp -j ACCEPT
>
># Allow dns servers
>-A input -s 132.236.56.250 53 -d 0/0 -p udp -j ACCEPT
>-A input -s 128.253.180.2 53 -d 0/0 -p udp -j ACCEPT
>-A input -s 132.236.56.250 53 -d 0/0 -p tcp -j ACCEPT
>-A input -s 128.253.180.2 53 -d 0/0 -p tcp -j ACCEPT
>
># Reject and log any other tcp and udp attempts
>-A input -s 0/0 -d 0/0 -p tcp -y -j REJECT -l
>-A input -s 0/0 -d 0/0 -p udp -j REJECT -l
>
>--------
>
>Add ipchains rules:
>
>[root@sealevel /root]# /etc/init.d/ipchains start
>Flushing all current rules and user defined chains: [ OK ]
>Clearing all current rules and user defined chains: [ OK ]
>Applying ipchains firewall rules: [ OK ]
>
>Create /etc/sysconfig/ip_protect with the following contents
>(courtesy http://www.linux-firewall-tools.com/linux/)
>
>-------------
>
>#!/bin/bash
># Enable TCP SYN Cookie Protection
>echo 1 > /proc/sys/net/ipv4/tcp_syncookies
>
># Enable always defragging Protection
>echo 1 > /proc/sys/net/ipv4/ip_always_defrag
>
># Enable broadcast echo Protection
>echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
>
># Enable bad error message Protection
>echo 1 > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses
>
># Enable IP spoofing protection
># turn on Source Address Verification
>for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
> echo 1 > $f
>done
>
># Disable ICMP Redirect Acceptance
>for f in /proc/sys/net/ipv4/conf/*/accept_redirects; do
> echo 0 > $f
>done
>
>for f in /proc/sys/net/ipv4/conf/*/send_redirects; do
> echo 0 > $f
>done
>
># Disable Source Routed Packets
>for f in /proc/sys/net/ipv4/conf/*/accept_source_route; do
> echo 0 > $f
>done
>
># Log Spoofed Packets, Source Routed Packets, Redirect Packets
>for f in /proc/sys/net/ipv4/conf/*/log_martians; do
> echo 1 > $f
>done
>
>-------------
>
>[root@sealevel /root]# chmod 600 /etc/sysconfig/ipchains
>[root@sealevel /root]# chmod 700 /etc/sysconfig/ip_protect
>
>
>In /etc/init.d/ipchains, add the folowing around line number 60
>(between ipchains -Z and echo -n $"Applying ipchains firewall rules:
>")
>
> action $"Running /etc/sysconfig/ip_protect:"
> /etc/sysconfig/ip_protect
>
>to run ip_protect at the right moment during startup.
>
>
>
>---------------
>
>RH 7.1 is now installed and secure. Proceeding to install McIDAS.
> From http://www.unidata.ucar.edu/packages/mcidas/780/index.html
>
>add this line to your /etc/rc.d/rc.local file:
>
> echo shared_memory_size > /proc/sys/kernel/shmmax
>
> (where shared_memory_size is the amount of shared memory you
> want to declare in bytes) and reboot.
>
> Note: since this will be the last thing run before your system
> comes up, your McIDAS session won't get a larger shared memory
> segment if it's started during
> the boot process).
>
>In this case echo 680000000 > /proc/sys/kernel/shmmax (256MB mem +
>512 MB swap, leaving ~100MB free)
>
>Then:
>[root@sealevel /root]# echo 680000000 > /proc/sys/kernel/shmmax
>
>---------------
>
>Installing McIDAS
>
>[root@sealevel /root]# groupadd -g 1000 mcidas
>[root@sealevel /root]# useradd -g mcidas mcidas
>[root@sealevel /root]# useradd -g mcidas -d /home/mcidas mcadde
>[root@sealevel /root]# useradd -g mcidas ldm
>
>-----------
>
>Then add the following to bottom of /home/mcidas/.bash_profile
>
># Bourne/Korn shell environment variable definitions for the user
> 'mcidas'
>
># umask
>umask 002
>
># MCHOME and McINST_ROOT
>MCHOME=$HOME
>McINST_ROOT=$MCHOME
>
># McIDAS environment variables
>MCDATA=$MCHOME/workdata
>MCPATH=${MCDATA}:$MCHOME/data:$MCHOME/help
>MCGUI=$MCHOME/bin
>MCTABLE_READ="${MCDATA}/MCTABLE.TXT;$MCHOME/data/ADDESITE.TXT"
>MCTABLE_WRITE="$MCHOME/data/ADDESITE.TXT"
>XCD_disp_file=$MCDATA/DECOSTAT.DAT
>PATH=${MCGUI}:$PATH
>export MCHOME McINST_ROOT MCDATA MCPATH MCGUI MCTABLE_READ
>MCTABLE_WRITE XCD_disp_file PATH
>
>
>-----------
>
>create /root/installs where all ldm, mcidas and gempak install files
>are kept.
>
>[root@sealevel /root]# mkdir /home/installs
>[root@sealevel /root]# mkdir /home/installs/ldm
>[root@sealevel /root]# mkdir /home/installs/mcidas
>[root@sealevel /root]# mkdir /home/installs/gempak
>
>
>download all files to these directories.
>
>then
>
>install ldm
>
>su ldm
>cd
>[ldm@sealevel ldm]$ tar zxvf /home/install/ldm/ldm-5.1.4.tar.Z
>[ldm@sealevel ldm]$ cd ldm-5.1.4/
>[ldm@sealevel ldm-5.1.4]$ cd src/
>
>Directions are at:
>http://www.unidata.ucar.edu/packages/ldm/ldmSourceInstallList.html
>
>Add LDMHOME=/home/ldm to~ldm/.bash_profile
>
>run export LDMHOME=/home/ldm
>
>run in /home/ldm/ldm-5.1.4/src ./configure
>
>[ldm@sealevel src]$ ./configure
>
>[root@sealevel src]# make install
>
>[root@sealevel src]# make install_setuids
>
>[ldm@sealevel src]$ cp pqact/pqact.conf ~ldm/etc
>[ldm@sealevel src]$ cp server/ldmd.conf ~ldm/etc
>[ldm@sealevel src]$ cp scripts/netcheck.conf ~ldm/etc
>[ldm@sealevel src]$ cp scour/scour.conf ~ldm/etc
>
>Replace "@hostname@" with fqdn in /home/ldm/ldm-5.1.4/bin/ldmadmin:
>
>$hostname = "sealevel.cit.cornell.edu";
>
>[ldm@sealevel ldm]$ ln -s ldm-5.1.4/ runtime
>
>[ldm@sealevel ldm]$ mkdir logs
>[ldm@sealevel ldm]$ touch logs/ldmd.log
>
>as root
>
>[root@sealevel bin]# cd /bin
>[root@sealevel /bin]# ln -s /home/ldm/runtime/bin/ldmadmin
>
>as ldm
>
>[ldm@sealevel ldm]$ ln -s ldm-5.1.4/bin
>
>[ldm@sealevel ldm]$ mkdir data
>[ldm@sealevel ldm]$ ldmadmin mkqueue
>
>[ldm@sealevel ldm]$ ldmadmin start
>starting the LDM server...
>
>_____________ LDM install done.
>
>su mcidas
>cd
>
> cp /home/install/mcidas/* .
>
>Logout and login as mcidas (reqd so that the .bash_profile runs
>
>[mcidas@sealevel mcidas]$ ./mcinstall
>
>[mcidas@sealevel src]$ cd mcidas7.8/src
>
>Install f2c
>
>[root@sealevel src]# rpm -i http://www.ibiblio.org/pub/Linux/distributions/red
> hat/7.1/en/powertools/i386/RedHat/RPMS/f2c-20000510-5.i386.rpm
>
>from:
>http://www.unidata.ucar.edu/packages/mcidas/780/mcx/warnings_mcx.html#gcc
>
>after your distribution is unpacked edit mcidas7.8/src/makefile and:
>
> change:
> VENDOR= -vendor
>
> to:
> VENDOR= -gcc
>
>[root@sealevel src]# ln -s /usr/lib/libf2c.a /usr/local/lib/libf2c.a
>
>add
>
>LD_LIBRARY_PATH=/usr/local/lib:/lib:/usr/lib to ~mcidas/.bash_profile
>
>Test McIDAS using the info on:
>http://www.unidata.ucar.edu/packages/mcidas/780/index.html
>
>[mcidas@sealevel src]$ cd ~mcidas/mcidas7.8/src
>[mcidas@sealevel src]$ make install.mcxall
>
>mcidas is now installed.
>
>Installing the ADDE services:
>
>The files to create in /etc/xinetd.d are:
>
>/etc/xinetd.d/mcserv:
>
># mcservsh (port 500)
>service mcserv
>{
> log_type = SYSLOG authpriv debug
> flags = REUSE
> socket_type = stream
> protocol = tcp
> wait = no
> port = 500
> user = mcadde
> server = /home/mcidas/bin/mcservsh
> server_args = -H /home/mcidas
> log_on_success += USERID HOST DURATION
> log_on_failure += USERID HOST RECORD
>}
>
>/etc/xinetd.d/mccompress:
>
># mccompress (port 503)
>service mccompress
>{
> log_type = SYSLOG authpriv debug
> flags = REUSE
> socket_type = stream
> protocol = tcp
> wait = no
> user = mcadde
> port = 503
> server = /home/mcidas/bin/mcservsh
> server_args = -H /home/mcidas
> log_on_success += USERID HOST DURATION
> log_on_failure += USERID HOST
>}
>
>Install xinetd:
>
>rpm -i http://www.ibiblio.org/pub/Linux/distributions/redhat/7.1/en/os/i386/Re
> dHat/RPMS/xinetd-2.1.8.9pre14-6.i386.rpm
>
>[root@sealevel mcidas]# /etc/init.d/xinetd start
>
>
>GEMPAK install.
>
>Gempak has been installed in /home/gempak (user was created, although
>not required)
>
>Modify the PATH line in /etc/skel/.bash_profile
>
>PATH=$PATH:$HOME/bin;/home/gempak/bin/linux;/home/gempak/scripts
>
>
>Also, add the McIDAS config stuff to skel:
>
>---------------
># .bash_profile
>
># Get the aliases and functions
>if [ -f ~/.bashrc ]; then
> . ~/.bashrc
>fi
>
># User specific environment and startup programs
>
>PATH=$PATH:$HOME/bin:/home/gempak/bin/linux:/home/gempak/scripts
>BASH_ENV=$HOME/.bashrc
>
>unset USERNAME
>
># umask
>umask 002
>
># MCHOME and McINST_ROOT
>MCHOME=/home/mcidas
>McINST_ROOT=$MCHOME
>
># McIDAS environment variables
>MCDATA=$MCHOME/workdata
>MCPATH=${MCDATA}:$MCHOME/data:$MCHOME/help
>MCGUI=$MCHOME/bin
>MCTABLE_READ="${MCDATA}/MCTABLE.TXT;$MCHOME/data/ADDESITE.TXT"
>MCTABLE_WRITE="$MCHOME/data/ADDESITE.TXT"
>XCD_disp_file=$MCDATA/DECOSTAT.DAT
>PATH=${MCGUI}:$PATH
>export BASH_ENV MCHOME McINST_ROOT MCDATA MCPATH MCGUI MCTABLE_READ
>MCTABLE_WRITE XCD_disp_file PATH
>
>
>
>---------------
>
>Getting LDM to work like on visibility... (is this a good thing?)
>
>... at least it is a baseline for now.
>
>Create ~ldm/etc files based on those on visibility replacing /var/data with /h
> ome/ldm/data
>
>run ldm... complains about failed writes.
>
>Created /home/ldm/data/gempak/logs as user ldm
>
>
>created directories under /home/ldm/data/gempak copying directory structure
>from visibility.
>
>In the /var/data/gempak directory on visibility, extracted directory structure
>using
>
>du | cut -f 2 | sort > newdirs.txt
>
>moved file to sealevel.
>
>created script mk-dirs
>with the following contents
>
>#!/bin/bash
>for i in $(cat newdirs.txt); do
> mkdir $i
>done
>
>runs fine.
>
>discovered a mkdirs.mak on ~ldm/etc on visibility that is supposed to
>do a similar thing but did not run as nicely.
>
>running pqact.conf with only the pnga2area - had to copy the following files
>to stop it from complaining:
>
>
>
>[ldm@sealevel mcidas]$ scp root@visibility:~ldm/data/mcidas/ROUTE.* .
>[ldm@sealevel mcidas]$ scp root@visibility:~ldm/data/mcidas/SYSKEY.TAB .
>
>Also copied /home/gempak/tables and ~ldm/decoders over to sealevel.
>
>Also had to add ~ldm/decoders to PATH for user ldm.
>
>OK... trying to get xcd_run to work discovered email on support list that is
>quoted below:
>
>--------------
>From address@hidden Wed Sep 15 19:35:16 1999
> by unidata.ucar.edu (8.8.8/8.8.8) with SMTP id TAA23333;
> Wed, 15 Sep 1999 19:35:16 -0600 (MDT)
>Message-Id: <address@hidden>
>To: Michael Keables <address@hidden>
>cc: address@hidden, address@hidden
>Subject: 19990915: LDM: Installation Problems
>In-reply-to: Your message of "Wed, 15 Sep 1999 14:35:01 MDT."
>Date: Wed, 15 Sep 1999 19:35:15 -0600
>From: Unidata Support <address@hidden>
>
>>From: Michael Keables <address@hidden>
>>Organization: DU
>>Keywords: 199909152035.OAA16060 LDM ldm-mcidas
>
>Mike,
>
>I saw your email and decided to answer since Robb was taking some
>vacation time today.
>
>>I'm trying to install the LDM on a Sun Ultra (Solaris 7) aka
>>cyclone.natnet.du.edu. I've gone through the checklists on the web but am
>>still unable to get the LDM to run properly.
>>
>>When I issue ldmadmin start & a process kicks off but nothing happens.
>
>OK, when this happens, you should follow the recommendations at the
>top of the file ~ldm/etc/ldmd.conf:
>
>####
>#
># This is the main configuration file for the LDM server. All lines that start
># with a "#" sign are comments.
>#
># To debug an LDM that hangs on start up, run the following from LDM home:
># % bin/rpc.ldmd -vl - -q data/ldm.pq etc/ldmd.conf
>#
># If the LDM still hangs, comment out all lines in this file, try again.
>
>In doing so, you could have found out that things would work after you
>commented out the following line from ldmd.conf:
>
>#exec "pqsurf"
>
>The reason that this line needed to be commented out was that the
>queue that pqsurf needs did not exist. The great majority of sites
>do not use pqsurf anymore, so I left this line commented out.
>
>This was not all that was needed, however. I also found that you had
>two entries for NDLN data. This was a typo from somewhere else since
>the datasteam is NLDN (D and L transposed). Also, the problem with doing:
>
>request NLDN ".*" cirrus.al.noaa.gov
>
>is that the NLDN (lightning) data is not one of the ones that gets
>relayed from upstream sites. All lightning data is sent point-to-point
>from SUNY Albany. If you havn't already done so, you must request
>(email) a feed of the NLDN data from:
>
> Name David Knight (p)
> Institution State Univ. of NY-Albany
> Department Earth and Atmospheric Sciences
> Street Address #1 1400 Washington Ave.
> Street Address #2 Earth Sci. Bldg. ES 228
>City, State, Zip, Co Albany NY 12222
> Phone: 518 442-4204 Fax: 518 442-4494
> Email Address address@hidden
>
>
>Dave will enable your machine to receive the data. Once he has done
>this, you will need to change the request NLDN line from what it is in
>~ldm/etc/ldmd.conf to:
>
>request NLDN ".*" striker.atmos.albany.edu
>
>For convenience, I did this for you but left the line commented out:
>
>#request NLDN ".*" striker.atmos.albany.edu
>
When Dave has setup stuff up, all you need to do is uncomment the line
>and stop and restart the LDM.
>
>>After a few minutes I get the following emai:
>>
>>Sep 15 19:54:40 UTC cyclone.natnet.du.edu : start_ldm: Server not started
>>or registered.
>>
>>A ps -eaf | grep ldm yields:
>>
>>cyclone% ps -ef | grep ldm
>> ldm 22126 22125 0 14:00:00 ? 0:00 /usr/local/bin/perl
>>bin/ldmfail -p cirrus.al.noaa.gov -f cirrus.al.noaa.gov
>> ldm 22125 176 0 14:00:00 ? 0:00 sh -c bin/ldmfail -p
>>cirrus.al.noaa.gov -f cirrus.al.noaa.gov
>> ldm 22135 22126 0 14:00:00 ? 0:00 /usr/local/bin/perl
>>/usr/local/ldm/bin/ldmadmin start
>> ldm 22265 22259 0 14:07:16 pts/4 0:00 -csh
>>
>>Issuing notifyme shows that I have access to cirrus.al.noaa.gov (albeit I
>>don't have a failover host yet so ldmd.primary and ldmd.failover are the
>>same as ldmd.conf.)
>
>It is great that you used notifyme to see if you have access to your
>upstream feed site. This was the exact right thing to try! It is
>also great that you used 'ldmadmin pqactcheck' to check the pqact.conf
>file entries. See below.
>
>>I have deduced the following:
>>
>>1. there is a problem with the following statement in pqact.conf (which I
>>downloaded from the web):
>>
>>cyclone% ldmadmin pqactcheck
>>Sep 15 20:25:57 pqact[22391]: feedtype error at line 14: unknown feed name
>>in feedtype expression: "MCIDAS ^(LWTOA3 .*)"
>
>Right. I looked at your pqact.conf file and found that all of the
>entries had spaces where tabs were called for. The most likely cause
>for this was someone cutting and pasting example pqact.conf entries
>into the file; true? I edited pqact.conf and changed all of the spaces
>to tabs where needed. I then ran 'ldmadmin pqactcheck' to discover
>other lines that had problems. There were three lines that are very
>long had line breaks in them. They looked like:
>
>#WSI ^NEX/(...)/(BREF1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])(
> [0
>-6][0-9])
>
>This entry was coming out as two lines instead of one. This was also
>probably due to cutting and pasting.
>
>>2. ldm/decoders is empty ... I assumed that I needed to download the
>>decoders from the web but I get a permissions violation when I try to
>>download decoders.tar.Z
>
>The other things that were missing from ~ldm/decoders were the ldm-mcidas
>decoders. Binary versions of these can be FTPed from ftp.unidata.ucar.edu
>from the pub/binary/sunos_5.7-sparc directory. I did this for you:
>
>cd ~ldm
>ftp ftp.unidata.ucar.edu
> <anonymous>
> <your email address>
> cd pub/binary/sunos_5.7-sparc
> bin
> get ldm-mcidas.tar.Z
> quit
>zcat ldm-mcidas-tar.Z | tar xvf -
>
>I then copied the ldm-mcidas decoders to the ~ldm/decoders directory:
>
>cp ldm-mcidas-7.6.1/bin/* decoders
>
>After that, I went into the decoders directory and setup the McIDAS
>ROUTE PostProcessing script, batch.k, to match your setup. This
>script allows such things as composite images to be produced. The
>editing job consisted of nothing more than changing the definition
>of MCHOME from /home/mcidas to /export/home/mcidas. What remains
>to be done is to enable the compositing by running ROUTE from
>a McIDAS-X session running as the user 'mcidas'. I didn't do this
>for you since we need to talk about where you have setup data
>file storage. More on this below.
>
>While in the decoders directory, I copied the file xcd_run from the
> McIDAS distribution:
>
>cp ~mcidas/mcidas7.6/src/xcd_run .
>
>I then edited xcd_run (another Bourne shell script) and changed MCHOME
>in the same way that was done for batch.k
>
>Finally, since the FSL2 wind profiler decoder needs the McIDAS SCHEMA
>file in the directory in which the decoder wants to create output
>MD files, I copied it there. I also copied ROUTE.SYS and SYSKEY.TAB
>since they will be used/updated by the ldm-mcidas and XCD decoders:
>
>cd /var/data/mcidas
>cp ~mcidas/data/SCHEMA .
>cp ~mcidas/data/SYSKEY.TAB .
>cp ~mcidas/workdata/ROUTE.SYS .
>
>After that, I was ready to start the LDM:
>
>ldmadmin start
>ldmadmin tail
>
>The LDM started up and began receiving data from the upstream feed site.
>Here is the contents of the /var/data/mcidas directory as I write this:
>
>cyclone% cd /var/data/mcidas
>cyclone% ls
>AREA0060 AREA0140 AREA0170 AREA0205 ROUTE.SYS
>AREA0120 AREA0150 AREA0191 AREA0210 SCHEMA
>AREA0130 AREA0160 AREA0200 MDXX0099 SYSKEY.TAB
>
>You can see that a number of images have already been received and decoded
>and that one MD file has been created. That MD file, 99, was created
>from the FSL2 6-minute profiler data.
>
>>Please advise on how to get out of the mess I'm in.
>
>OK, now we need to talk about where the data are currently going:
>/var/data/mcidas. I did a quick check of disk space on your machine:
>
>cyclone% df -k
>Filesystem kbytes used avail capacity Mounted on
>/proc 0 0 0 0% /proc
>/dev/dsk/c0t0d0s0 96455 40768 46042 47% /
>/dev/dsk/c0t0d0s6 877790 665487 150858 82% /usr
>fd 0 0 0 0% /dev/fd
>/dev/dsk/c0t0d0s1 413639 77771 294505 21% /var
>/dev/dsk/c0t1d0s7 8509324 339070 8085161 5% /export/home
>/dev/dsk/c0t0d0s5 3007086 917816 2029129 32% /opt
>/dev/dsk/c0t0d0s7 4031022 140916 3849796 4% /usr/local
>swap 657744 112 657632 1% /tmp
>
>and see that /var is very small (total of 400 MB to begin with). This
>will not be enough room to store McIDAS or GEMPAK data in. The most
>likely candidate for data storage is /export/home since it has
>8 GB of disk available. I ordinarily don't recommend that data files
>be kept in /home (or /export/home in your case), but it would be painful
>to go back and repartition your disk. Is there more disk in the system
>that hasn't been mounted? If so, and if it is on the order of at
>least 2-3 GB, then that is where the data should go.
>
>In the meantime, in order to exercise the LDM and ldm-mcidas decoders,
>I left things running.
>
>On the McIDAS side of things, I see that you have not yet setup XCD.
>Once you have done this (I would have done so for you, but there
>is not enough disk space in /var/data), turning on XCD decoding
>will be as simple as editing ~ldm/etc/pqact.conf and uncommenting
>out the lines for XCD processing:
>
># Entries for XCD decoders
>#DDPLUS|IDS ^.* PIPE
># xcd_run DDS
>#HRS ^.* PIPE
># xcd_run HRS
>
>(remove the '#' signs at the beginning of the lines while making sure
>to NOT turn them into spaces).
>
>Let's touch base on your setup tomorrow.
>
>>Thanks in advance.
>
>You are welcome.
>
>Tom
>
>
>------------------------
>
>Things learned: TABS not equal to Spaces. (seen that somewhere)
>
>Also, copying of the SYSKEY.TAB and ROUTE.SYS are validated. [will copy from
>original sources again, see below]
>
>Attemting to reconstruct decoders from downloaded gempak and mcidas files.
>
>Created new-decoders with
>
>mkdir new-decoders
>cd new-decoders/
>cp ~gempak/bin/linux/* .
>cd /home/mcidas/mcidas7.8/src/
>cp ./batch.k ~/new-decoders/
>cp ./xcd_run ~/new-decoders/
>cp /home/mcidas/workdata/uwgrid.sh ~/new-decoders/
>cp /home/mcidas/mcidas7.8/data/mcscour.sh ~/new-decoders/
>
>Note:
>
>from /home mcidas
>diff ./workdata/mcscour.sh ./mcidas7.8/data/mcscour.sh
>diff ./workdata/uwgrid.sh ./mcidas7.8/data/uwgrid.sh
>
>showed that they were identical.
>
>
>edit the xcd_run file and modify path as follows:
>
>-----------
># in this example that McIDAS-X was installed in /home/mcidas.
>
>MCHOME=/home/mcidas
>MCDATA=$MCHOME/workdata
>MCGUI=$MCHOME/bin
>MCLOG=$MCDATA/XCD_START.LOG
>MCPATH=${MCDATA}:$MCHOME/data:$MCHOME/help:/home/ldm/data/mcidas:/home/mcidas/
> bin
>
>
># Setup PATH so that the McIDAS-X executables can be found
>
>PATH=${MCGUI}:/usr/bin:/usr/sbin:/usr/local/bin:/bin:/home/mcidas/bin
># Set LD_LIBRARY_PATH to include all directories (other than those searched
># by default) that are needed to be searched to find shared libraries.
># For this example, I assume that the shared Fortran library is located
># in /opt/SUNWspro/lib
>
>LD_LIBRARY_PATH=/usr/local/lib:/lib:/usr/lib:.:
>
># Export needed environment variables
>export MCPATH PATH LD_LIBRARY_PATH
>-----------
>
>Added export MCHOME MCDATA MCLOG MCPATH PATH LD_LIBRARY_PATH
>
>to the bottom of path definitions (as was done in other file) in mcscour.sh
>
>[ldm@sealevel mcidas]$ cd mcidas7.8/src/
>[ldm@sealevel src]$ cp ingebin.k ~/decoders/
>[ldm@sealevel src]$ cp ingetext.k ~/decoders/
>
>
>-----------
>
>Oct 22 2001
>
>Added crontab to ldm: scour stuff must run periodically or we get a full FS.
>
>[ldm@sealevel decoders]$ crontab -l
># DO NOT EDIT THIS FILE - edit the master and reinstall.
># (/tmp/crontab.5536 installed on Mon Oct 22 10:47:19 2001)
># (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)
>0 0 * * * /home/ldm/bin/ldmadmin newlog
>15 1,3,5,7,9,11,13,15,17,19,21,23 * * * /home/ldm/bin/ldmadmin scour >>/dev/nu
> ll 2>&1
>0 1,3,5,7,9,11,13,15,17,19,21,23 * * * /home/ldm/decoders/mcscour.sh >>/dev/n
> ull 2>&1
>0 23 * * * /home/ldm/bin/clean_WMO
>
>Figured out that I did not install McIDAS XCD! duh!
>
>
>[mcidas@sealevel mcidas]$ cd mcidas7.8/src/
>[mcidas@sealevel src]$ make xcd
>[mcidas@sealevel src]$ make install.xcd
>
>For using decoders:
>
>http://www.unidata.ucar.edu/packages/mcidas/780/mcx/xcd_start.html
>
>
>[ldm@sealevel decoders]$ cp ~mcidas/mcidas7.8/src/xcd_run ~ldm/decoders
>
>
>
>[ldm@sealevel etc]$ cp ~mcidas/workdata/mcscour.sh ~ldm/decoders
>
>Edit ldmd.conf make sure exec lines are
>
>exec "pqexpire"
>exec "xcd_run MONITOR"
>exec "pqbinstats"
>exec "pqact"
>exec "pqsurf"
>
>
>get decoders ftp://ftp.unidata.ucar.edu/pub/decoders/decoders.tar.Z
>
>get
>[root@sealevel src]# export CPP_UDUNITS=-I/home/install/udunits/udunits-1.11.7
> /src/lib
>[root@sealevel src]# export LD_UDUNITS="-L/home/install/udunits/udunits-1.11.7
> /src/lib -ludunits"
>[root@sealevel src]# export CPP_NETCDF=-I/home/install/netcdf/netcdf-3.5.0/src
> /libsrc
>[root@sealevel src]# export LD_NETCDF="-L/home/install/netcdf/netcdf-3.5.0/src
> /libsrc -lnetcdf"
>
>abandoned.
>---
>
>[ldm@sealevel decoders]$ cp /home/install/ldm-mcidas/ldm-mcidas-7.6.4/bin/* .
>
>
>In ~ldm/data/mcidas:
>
>cp ~mcidas/data/SCHEMA .
cp ~mcidas/data/SYSKEY.TAB .
>cp ~mcidas/workdata/ROUTE.SYS .
>
>then in ~ldm/decoders
>
>[ldm@sealevel decoders]$ cp ~gempak/bin/linux/dc* .
>[ldm@sealevel decoders]$ cp ~gempak/bin/linux/fi* .
>
>
>Then config XCD using
>http://www.unidata.ucar.edu/packages/mcidas/780/mcx/xcd_config.html
>
>run mcidasx instead of mcidas (as user mcidas).
>
>
>
>OK. got the thing to run. in /home/mcidas/workdata. using above config,
>setting up xcd_run to run in the above directory. will reboot to check
Tom Yoksas
>From address@hidden Fri Oct 26 13:21:46 2001
>Subject: Re: 20011025: McIDAS problems at Cornell
>Cc: address@hidden
Hi Tom,
re: This actually does not sound like the problem you found and fixed.
Yes, my wording was ambiguous, - the problem I found and fixed was the
backdoor that was put on the machine so that the hackers could get in and
run IRC and search for other sites to exploit. However since the machine
froze and rebooted it lost some configs that must not been committed to
startup scripts, and since I did not put the machine together, I had no
idea of what that was. Putting ipchains on the machine (which should have
been there from the start, but was not) gave it some measure of safety.
Btw, the hackers used the lpd driver (see advisories for BSD based lpd
daemons which are run on every flavor of UNIX (linux & BSD included)) which
has a buffer overflow that allowed them to get root access - If you run lpd
(without the *latest* patch) and do not have any firewalling, watch out!
They were then running an ssh based back door, installed in the /dev directory.
I would also like to note (for the sake of others who might read your
previous reply), that my notes are a little ambiguous towards the end - I
had trouble figuring out how to get the minimum set of decoder files in the
ldm/decoders directory that would work with the pqact.conf that I had from
the other machine. What ended up working was roughly something like this
1) copy the dc* files from the gempack/bin/linux directory that I needed
for the Gempak decoders that were required by my pqact.conf
2) the xcd_run from the mcidas install - take care that $PATH as well as
other environment variables are correct - I got a number of write errors
running the HRS and DDS decoders until it was all fixed, including
appropriate group write permissions, follow the fine manual.
3) the pnga2area and others from the ldm-mcidas.tar.Z file
4) other stuff - mcscour.sh etc: Here is a listing of what I have in
~ldm/decoders
area2png dcgrib dcmosm dcredbook dctrop lwfile uwgrid.sh
batch.k dcgrib2 dcmsfc dcredbook_gf dcuair lwtmd2 xcd_run
cdftomd dchrcn dcncon dcredbook_ps dcwarn lwtoa3 xcd_run~
dcacars dcisig dcncprof dcredbook_vg dcwatch mcscour.sh
dcacft dclsfc dcnldn dcscd dcwtch nids2area
dcairm dcmetr dcnmos dcshef fips nldn2md
dcamos dcmmos dcprof dcstorm gunrv2 pnga2area
dcffg dcmosa dcreanal dcsvrl ldm_pipe_exec proftomd
So - if I've not mentioned them above they came either from the
gempak/bin/linux or from somewhere in ~mcidas.
I have to do this once more for my own satisfaction as I mentioned before
and come up with a list to get McIDAS, LDM and the Gempak decoders working
together from scratch.
Best regards,
---Venkat.
Venkat Manakkal
President,
Manakkal & Associates, Inc.
http://www.manakkal.com/
Tel: 607-546-7300 Fax: 509-471-5693 Cell: 603-321-9870