[GEMPAK #VOF-422461]: Problems with merged data and decoder

Arlene,

I clipped out the obs you had below, and decoded and have:

 GEMPAK-SFLIST>r
 PARM = TMPC;SKYC                                                               
 

    STN    YYMMDD/HHMM      TMPC     SKYC
   OAKB    060114/0100     -3.00 -9999.00
   OAKB    060114/0200     -3.00 -9999.00
   OAKB    060114/0300     -2.00 -9999.00
   OAKB    060114/0400      0.00     6.00
   OAKB    060114/0500      2.00 -9999.00
   OAKB    060114/0600      3.00 -9999.00
   OAKB    060114/0700      4.00     8.00
   OAKB    060114/0800      5.00     8.00
   OAKB    060114/0840      5.00     6.00
   OAKB    060114/0900      4.00     6.00
   OAKB    060114/0920      3.00     6.00
   OAKB    060114/1000      2.00     8.00
   OAKB    060114/1100      1.00     8.00
   OAKB    060114/1120      1.00     8.00
   OAKB    060114/1140      1.00     8.00
   OAKB    060114/1200      1.00     8.00
   OAKB    060114/1300      0.00     8.00
   OAKB    060114/1320      0.00     8.00
   OAKB    060114/1400      0.00     8.00
   OAKB    060114/1500      0.00     8.00
   OAKB    060114/1600      0.00     8.00
   OAKB    060114/1700      0.00     8.00
   OAKB    060114/1800      0.00     8.00
   OAKB    060114/1900      0.00     8.00
   OAKB    060114/2000      0.00     8.00
   OAKB    060114/2100      0.00     8.00
   OAKB    060114/2200      0.00     8.00
   OAKB    060114/2300      0.00     8.00


I clearly get times that you don't. So, either your input .csv file isn't what 
I'm
expecting based on the info below, or your file is maxed out on times-
which should be impossible given that we are creating daily files with
72 entries.

1) Verify that all you data times in your decoded data file are for the 14th
with DATTIM=list in sflist.

2) assuming you don't have more than the 1 day's times in that file, would
you send me your .csv file that you are using with the cvs2gem.csh script I 
provided 
since the sample set I have to try is the OKBK XLS file you had sent previously.

As for pressure, the decoder expects the report to provide altimeter setting in 
hectopascals if the
number is Qxxxx or inches if it is Axxxx. The sea level pressure is expected to 
use the SLP
identifier, and the altimeter setting is used to guess at the hundreds digit 
(if altimeter
is missing, then the SLP is assumed to be in the range 980-1049 mb).

Steve Chiswell
Unidata User Support


> Hi Steve,
> 
> Thanks for checking the sample data and providing feedback on
> the decoder.  I will review my two problems in order.
> 
> 1) The pressure and altimeter observations
> -----------------------------------------
> I understand that I should be able to use the PANY to access whatever
> pressure observations the non-US stations report.
> However, I am still confused about what is actually reported.
> I thought that for international METARs the "Q" group represents
> the mean sea-level pressure.
> I looked at the WMO Ttable of codes and other sites where questions are
> asked about the "Q" code group, which reports QNH.
> From what I understand (and remember from my long ago forecaster days)
> this group represents sea level pressure except for regional requirements
> like the US where the group is replaced by the "A" + altimeter setting in
> in inches of mercury.  In that case, the sea level pressure
> is reported in the "RMK" section of the US METAR.
> Here's what the UK Civil Aviation authority says in their FAQ:
> 
> 2.6 Pressure
> Which pressure is reported in the METAR ? the airfield threshold pressure
> or the mean sea level pressure?
> 2.6.1 Care should be taken to report the aerodrome QNH (mean sea level 
> pressure)
> in the METAR
> 
> QNH: Atmospheric pressure at nautical height.  Q-code designation for
> atmospheric pressure at mean sea level.
> QFE: Atmospheric Pressure at Aerodrome Elevation (or at runway threshold)
> 
> 2) Missing times from the GEMPAK files
> ------------------------------------------
> Here's data from the ".csv" text file for Kabul, OAKB:
> 
> OAKB 140050Z 320014KT 3000 BR NSC M03/M04 Q1018
> OAKB 140150Z 32003KT 2000 BR NSC M03/M03 Q1019
> OAKB 140250Z 00000KT 2000 BR NSC M02/M03 Q1020
> OAKB 140350Z 29001KT 1500 HZ FEW100 BKN160 M00/M02 Q1020
> OAKB 140450Z 34002KT 1800 HZ NSC 02/M02 Q1021
> OAKB 140550Z 31002KT 2000 HZ NSC 03/M01 Q1020
> OAKB 140650Z 14002KT 1800 HZ FEW013 OVC100 04/M01 Q1019
> OAKB 140750Z 32004KT 2000 HZ FEW011 OVC100 05/M01 Q1018
> OAKB 140830Z 31003KT 5000 HZ SCT045 BKN060 05/M01 Q1018
> OAKB 140850Z 03003KT 7000 -RA FEW030 SCT040 BKN050 04/M01 Q1018
> OAKB 140915Z 15004KT 4000 BR -RA SCT020 SCT030 BKN035 03/M00 Q1018
> OAKB 140950Z 21005KT 2000 BR -RASN FEW007 SCT015 OVC025 02/00 Q1018
> OAKB 141050Z 27004KT 1500 BR -SNRA FEW005 SCT010 OVC020 01/01 Q1017
> OAKB 141110Z 29003KT 1200 BR -RASN SCT004 BKN015 OVC020 01/01 Q1017
> OAKB 141140Z 30003KT 0700 FG -SN FEW005 SCT015 OVC020 01/01 Q1017
> OAKB 141150Z 30002KT 0700 FG -SN FEW005 SCT015 OVC020 01/01 Q1017
> OAKB 141200Z 29003KT 0700 FG SN FEW004 SCT010 OVC020 01/01 Q1017
> OAKB 141250Z 00000KT 0500 FG SN FEW001 SCT010 OVC015 00/00 Q1016
> OAKB 141320Z 30005KT 1000 BR SN FEW001 SCT010 OVC015 00/00 Q1017
> OAKB 141350Z 33004KT 1000 BR SN FEW007 SCT015 OVC020 00/00 Q1016
> OAKB 141450Z 27004KT 1500 SN FEW005 BKN015 OVC020 00/00 Q1017
> OAKB 141550Z 00000KT 1500 -SN BKN013 OVC020 00/00 Q1016
> OAKB 141650Z 27003KT 2000 -RASN BKN007 OVC020 00/00 Q1016
> OAKB 141750Z 03003KT 1200 SN SCT006 OVC013 00/00 Q1016
> OAKB 141850Z 00000KT 1500 SN BKN005 OVC016 00/00 Q1015
> OAKB 141950Z 04005KT 1000 SN BKN005 OVC012 00/00 Q1014
> OAKB 142050Z 09005KT 1000 SN OVC010 00/00 Q1014
> OAKB 142150Z 00000KT 1200 SN BKN006 OVC015 00/00 Q1014
> OAKB 142250Z 36004KT 1500 SN SCT005 OVC015 00/00 Q1013
> OAKB 142350Z 04004KT 1200 SN SCT005 OVC012 00/00 Q1013
> 
> The GEMPAK file shows only three hours of data.  I've listed only
> two parameters in the example but it is the same three times for all
> parameters.
> 
> GEMPAK-SFLIST>list
> SFFILE   = 20060114_sao.gem
> AREA     = @oakb
> DATTIM   = all
> SFPARM   = tmpc;skyc
> OUTPUT   = t
> IDNTYP   = STID
> GEMPAK-SFLIST>r
> PARM = TMPC;SKYC
> 
> STN    YYMMDD/HHMM      TMPC     SKYC
> OAKB    060114/0000      0.00     8.00
> OAKB    060114/0100     -3.00 -9999.00
> OAKB    060114/2300      0.00     8.00
> 
> Have you any words of advice about what we can do to
> solve this problem?
> 
> Thank you very much for your help.
> 
> Arlene Laing
> 
> 
> On Fri, May 19, 2006 at 10:48:54AM -0600, Unidata GEMPAK Support wrote:
> > Arlene,
> >
> > In your data below, both stations are decoded. The GEMPAK dcmetr decoder 
> > understands both
> > units of millibars and of inches.
> >
> > The OKBK station does not report a reduced sea level pressure, so that will 
> > be -9999
> > in the decoded data. The station does report an altimeter setting as "Q1011"
> > which is 29.85 inches as you see stored as ALTI (if you plot ALTM you will 
> > see 1011).
> > Note that altimeter and reduced sea level pressure are 2 different 
> > variables!
> >
> > The KQSA station does report sea level pressure and altimeter setting, so 
> > both will
> > be in the decoded file (eg the section of the metar A3018 RMK SLP265 )
> > as you see in your decoded data 1026.50 and 30.18.
> >
> > So, I'm interpreting your statement that something is "missing" as you are
> > trying to plot a map at a time when PSML doesn't exist at a station. If you 
> > want to
> > allow the station to plot either PMSL if it exists, or ALTM otherwise,
> > you can use the PANY parameter as shown in "phelp sfparm":
> >
> >    PANY - Returns PMSL, if avaliable, if not, returns ALTM
> >
> > If you have a different example of s station that wasn't decoded, we can 
> > look at that, but from
> > what you have presented I'm assuming you are trying to merge ALTM and PMSL.
> >
> > Steve Chiswell
> > Unidata User Support
> >
> >
> >
> > > Hi Steve,
> > >
> > > Thanks to your help, we converted our extra station data into GEMPAK 
> > > format
> > > and merged the new data with our global archive data files.
> > > However, we have encountered a problem with the GEMPAK data files.
> > > After examining some recent plots, I realized that some
> > > of the data was missing from the plots.  I went back to the original files
> > > and found the data.  However, those data did not show up in
> > > the GEMPAK files that were created using the script that you provided
> > > (csv2gem.csh)
> > > For some stations, the GEMPAK files are missing several hours of data.
> > > For example, for Kabul, only 2 of 24 observation times show up in GEMPAK.
> > > For other stations, only some of the data are missing.
> > > I think that the latter problem arises because the data files contain
> > > both US military and WMO station data.  For example, the decoder 
> > > recognizes
> > > pressure in inches of Hg but not millibars and so some stations do not
> > > have any pressure data listed in the GEMPAK files.
> > >
> > > Do you have any recommendations for how to resolve this problem so
> > > that we can plot all of the data on a single map?  I checked through the
> > > email archives and found an example of someone trying to plot Canadian 
> > > station
> > > data with US data but could not follow everything well enough for 
> > > application
> > > to this problem.
> > >
> > > Here are examples from Kuwait (WMO) and Bagram (US Mil):
> > >
> > > From kuwait.csv
> > > -----------------
> > > 20060111,1000,180,9,,9000,,,,,,,,,,19,12,29.85,,OKBK 111000Z 18009KT 
> > > CAVOK 19/12 Q1011 NOSIG;
> > >
> > > From GEMPAK
> > > -----------
> > > GEMPAK-SFLIST>dattim=060111/1000
> > > GEMPAK-SFLIST>r
> > > PARM = 
> > > PMSL;ALTI;TMPC;DWPC;SKNT;DRCT;GUST;WNUM;CHC1;CHC2;CHC3;VSBY;P03D;P03I;
> > > MSUN;SNOW;WEQS;P24I;TDXC;TDNC;P03C;CTYL;CTYM;CTYH;P06I;T6XC;T6NC;CEIL;
> > > P01I;PWSP;PWDR
> > >
> > > STN    YYMMDD/HHMM      PMSL     ALTI     TMPC     DWPC     SKNT     DRCT
> > > GUST     WNUM     CHC1     CHC2     CHC3     VSBY
> > > P03D     P03I     MSUN     SNOW     WEQS     P24I
> > > TDXC     TDNC     P03C     CTYL     CTYM     CTYH
> > > P06I     T6XC     T6NC     CEIL     P01I     PWSP
> > > PWDR
> > > OKBK    060111/1000  -9999.00    29.85    19.00    12.00     9.00   180.00
> > > -9999.00 -9999.00 -9999.00 -9999.00 -9999.00     6.21
> > > -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
> > > -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
> > > -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
> > > -9999.00
> > >
> > >
> > > From bagram.csv
> > > --------------
> > > 20060111,1000,220,5,,9000,HZ,FEW,100,SCT,200,,,,,11,-2,30.18,,KQSA 
> > > 110955Z 22005KT 9000 HZ FEW100 SCT200 11/M02 A3018 RMK SLP265 WND DATA 
> > > ESTMD ALSTG/SLP ESTMD;
> > >
> > > From GEMPAK
> > > ----------
> > > GEMPAK-SFLIST>dattim=060111/1000
> > > GEMPAK-SFLIST>r
> > > PARM = 
> > > PMSL;ALTI;TMPC;DWPC;SKNT;DRCT;GUST;WNUM;CHC1;CHC2;CHC3;VSBY;P03D;P03I;
> > > MSUN;SNOW;WEQS;P24I;TDXC;TDNC;P03C;CTYL;CTYM;CTYH;P06I;T6XC;T6NC;CEIL;
> > > P01I
> > >
> > > STN    YYMMDD/HHMM      PMSL     ALTI     TMPC     DWPC     SKNT     DRCT
> > > GUST     WNUM     CHC1     CHC2     CHC3     VSBY
> > > P03D     P03I     MSUN     SNOW     WEQS     P24I
> > > TDXC     TDNC     P03C     CTYL     CTYM     CTYH
> > > P06I     T6XC     T6NC     CEIL     P01I
> > > KQSA    060111/1000   1026.50    30.18    11.00    -2.00     5.00   220.00
> > > -9999.00     6.00  1006.00  2002.00 -9999.00     5.59
> > > -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
> > > -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
> > > -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
> > >
> > >
> > > Thanks in advance,
> > > Arlene Laing
> > > MMM/COMET
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: VOF-422461
> > Department: Support GEMPAK
> > Priority: Normal
> > Status: Closed
> >
> 
> 


Ticket Details
===================
Ticket ID: VOF-422461
Department: Support GEMPAK
Priority: Normal
Status: Closed