Re: 20051115:missing HDS parameters

Wen,

The NEXRCOMP radar composite is distributed in the FNEXRAD data feed.
The GINI format products I create are current in the data feed. You
can check your ldmd.conf to see if you are requesting the data
from your upstream, and chek the rtstats pages to make sure your
upstream site is receiving the data.

The TAMU site has current GINI products in the 1km/n0r, 2km/n1p and
4km/NTP directories. The pqact.gempak_nexrad file has the FNEXRAD
pattern for these products.

The directories on the TAMU site that are old are the AREA file
directories for products that Tom Yoksas was creating from the GINI
composites mentioned above. My guess is that Tom's program did not get
ported over when the adde.ucar.edu server was changed. But, these are
not original data products, just reformating of the original GINI format
FNEXRAD products, so all the data is still available in the original
format. I'll let Tom know his AREA files are not currently flowing.

Steve Chiswell
Unidata User Support


On Thu, 2005-11-17 at 08:45, Lu wrote:
> Steve,
> 
> Thank you very much for your instructions. I reorganized our pqact conf
> partly and will reorganize more. Things get better. Model data come in
> normally, but will watch more.
> Here is another question for you: we don't receive NEXRCOMP data now. I
> have not changed pqact for it for a while. I noticed that some places,
> such as http://mesonet.tamu.edu/products/IMAGES/SAT/NEXRCOMP, have the
> data until 19-oct-2005. What is the problem? I appreciate it if you could
> advise me on that matter.
> Thanks,
> 
> Wen
> 
> 
> On Tue, 15 Nov 2005, Steve Chiswell wrote:
> 
> > Wen,
> >
> > If you are missing decoded fields in your file, and your data volume
> > and latency for the GDS feed checks out, then the likely problem is that
> > you are not processing the data out of the queue by pqact before
> > it gets overwritten by new data.
> >
> > What is the size of your queue?
> > You are receiving approximately 1.2GB an hour on average, and up
> > to 1.8GB per hour during the hours of maximum model data. If your
> > queue is smaller than 2GB, that would be the first suspect.
> >
> > The pqmon program from LDM will tell you what the age of the oldest
> > product is in your queue. That combined with the amount of IO
> > you may have with the level2 radar feed might cause the pqact processing
> > to be slower than is needed.
> >
> > Are you running separate pqact processes for the GEMPAK decoders apart
> > from other pqact processing of the Level2 radar? The gen_pqact.csh
> > script I provide in $NAWIPS/ldm/etc asks whether you want a single
> > or multiple pqact files, and will generate separate pqact files
> > and provide you with the suggested ldmd.conf entries as needed.
> >
> > Steve Chiswell
> > Unidata User Support
> >
> >
> >
> >
> > On Tue, 2005-11-15 at 14:38, Jeff Weber wrote:
> > > Hi Wen,
> > >
> > > I suspect it is a table or reg exp issue, as you are running a fairly old
> > > version of GEMPAK...
> > >
> > > The latest version is 5.8.4
> > >
> > > I will pass this by our GEMPAK developer to make sure there are not other
> > > issues at play.
> > >
> > > I checked the volume of your data ingest on the HDS (HRS) feed and it
> > > matched identically with ours, so the data is making it to your queue, how
> > > it is being processed from there is where a problem may exist.
> > >
> > > Cheers,
> > >
> > > Jeff
> > > ---------------------------------------------------------------------
> > > Jeff Weber                                    address@hidden        :
> > > Unidata Program Center                        PH:303-497-8676        :
> > > University Corp for Atmospheric Research      3300 Mitchell Ln       :
> > > http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 80307-3000  :
> > > ---------------------------------------------------------------------
> > >
> > > On Tue, 15 Nov 2005, Lu wrote:
> > >
> > > >
> > > > Hi Jeff,
> > > >
> > > > I believe it is version 5.6
> > > >
> > > > Wen
> > > >
> > > >
> > > > On Tue, 15 Nov 2005, Jeff Weber wrote:
> > > >
> > > > > Hi Wen,
> > > > >
> > > > > What version of GEMPAK are you running?
> > > > >
> > > > > Thank you,
> > > > >
> > > > > Jeff
> > > > > ---------------------------------------------------------------------
> > > > > Jeff Weber                                    address@hidden        :
> > > > > Unidata Program Center                        PH:303-497-8676        :
> > > > > University Corp for Atmospheric Research      3300 Mitchell Ln       :
> > > > > http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 80307-3000  :
> > > > > ---------------------------------------------------------------------
> > > > >
> > > > > On Mon, 14 Nov 2005, Lu wrote:
> > > > >
> > > > > >
> > > > > > Hi Jeff,
> > > > > >
> > > > > > Let's specify to talk about 211 grids.
> > > > > > We receive files for
> > > > > >
> > > > > > ETA, 2 files daily at hours 00 and 12
> > > > > > RUC, 24 files, one file per hour
> > > > > > NGM, 2 files daily at hours 00 and 12
> > > > > > NOGAPS, 4 files a day at hours 00,06,12,18, without specifing the 
> > > > > > grid
> > > > > >
> > > > > > Also let's display pressure at level 500 of images Nov 13 and 14, 
> > > > > > errors
> > > > > > for:
> > > > > >
> > > > > > ETA (41 images in all)
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0000F012 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0000F042 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0000F054 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F018 @500 %PRES cannot 
> > > > > > be found
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F030 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F042 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F012 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F024 @500 %PRES cannot 
> > > > > > be found
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F054 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/1200F030 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/1200F036 @500 %PRES cannot 
> > > > > > be found.
> > > > > >
> > > > > > RUC ( 72 images for Nov 13, 60 till 1900UTC
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/0700F003 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0200F003 @500 %PRES cannot 
> > > > > > be found.
> > > > > >
> > > > > > NGM ( 36 images in all)
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051113/1200F036 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/0000F018 @500 %PRES cannot 
> > > > > > be found.
> > > > > > GEMPAK: [DG -7]  Input grid TMPC ^051114/1200F042 @500 %PRES cannot 
> > > > > > be found.
> > > > > >
> > > > > >
> > > > > > NOGAPS (124 images Nov 13,
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F051 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F075 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F081 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F120 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/0000F132 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/0600F078 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/0600F090 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F051 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F075 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F081 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F084 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F093 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F132 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1200F144 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1800F078 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051113/1800F090 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > >
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F051 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F066 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F069 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F072 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F075 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F078 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F081 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F108 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0000F132 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0600F078 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/0600F090 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F051 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F075 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F081 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F090 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F093 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > > GEMPAK: [DG -7]  Input grid WND ^051114/1200F114 @500 %PRES in AVOR 
> > > > > > cannot be found.
> > > > > >
> > > > > > Attached please find are our files of ldmm.conf, pqact.conf, 
> > > > > > pqact.gempak,
> > > > > > and pqact.nexrad
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Wen
> > > > > >
> > > > > >
> > > > > > On Mon, 14 Nov 2005, Jeff Weber wrote:
> > > > > >
> > > > > > > Hi Wen,
> > > > > > >
> > > > > > > What grids (model numbers) of the ETA (NAM), RUC, NGM, MRF (GFS), 
> > > > > > > and
> > > > > > > NOGAPS are you referring to?
> > > > > > >
> > > > > > > Are these all the 211 (CONUS) grids?
> > > > > > >
> > > > > > > Some models have 6 hour runs, while other are every 12...and 
> > > > > > > NOGAPS, is
> > > > > > > spotty with some parameters..
> > > > > > >
> > > > > > > No worries about rtstats, that will not affect your data 
> > > > > > > reception, but
> > > > > > > thank you for the heads up...I will check on that.
> > > > > > >
> > > > > > > Please send me your ldmd.conf if you can, and your pqact.conf 
> > > > > > > files.
> > > > > > >
> > > > > > > Also, if you could be more specific about what parameters are 
> > > > > > > missing and
> > > > > > > when, and if there is a repeatable pattern to these data gaps, so 
> > > > > > > we can
> > > > > > > determine if in fact you are missing data, or if it is in fact a 
> > > > > > > reality
> > > > > > > in data availability.
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Jeff
> > > > > > > ---------------------------------------------------------------------
> > > > > > > Jeff Weber                                    address@hidden      
> > > > > > >   :
> > > > > > > Unidata Program Center                        PH:303-497-8676     
> > > > > > >    :
> > > > > > > University Corp for Atmospheric Research      3300 Mitchell Ln    
> > > > > > >    :
> > > > > > > http://www.unidata.ucar.edu/staff/jweber      Boulder,Co 
> > > > > > > 80307-3000  :
> > > > > > > ---------------------------------------------------------------------
> > > > > > >
> > > > > > > On Mon, 14 Nov 2005, Lu wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Hi Jeff,
> > > > > > > >
> > > > > > > > I have a couple quentions for you.
> > > > > > > > Recently, quite many model images of ETA, RUC, NGM, MRF, and 
> > > > > > > > NOGAPS
> > > > > > > > are missing from our data streaming. We received whole files of 
> > > > > > > > these
> > > > > > > > models, but we could not display all images of a type (eg. @500 
> > > > > > > > %pres)
> > > > > > > > in its entire time sequence: always missing some of them.
> > > > > > > > Our ldmd.log file shows that we could not connect
> > > > > > > > rtstats.unidata.ucar.edu. We could not figure it out. Was it 
> > > > > > > > the machine
> > > > > > > > down sometimes?
> > > > > > > > I gratefully appreciate it if you could provide your 
> > > > > > > > suggestions.
> > > > > > > >
> > > > > > > > Wen
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >