Re: 20050907: trapped exception in Grib code hides failure

eric,

the problem was the X2 var in bit unpacking rountine can be a missing
value if all bits are set to 1's.  it's not apparent from the docs about
this case.  so now the ds.mint.bin file can be decoded correctly.  i need
to check the other files.  the ds.sky.bin file with proj 10 needs to have
the proj 10 code implemented. do you know of any java coding of proj 10?
if not, how about c code.  it will probably be awhile before i have the
time to look at proj 10.

robb...



On Tue, 13 Sep 2005, Eric Davies wrote:

> Hi Robb,
>
>     glad to hear you've been able to repeat the problem. Prehaps a bug in
> the decompression.
>
>     One simple fix I would suggest though is not masking the exception.
>     Its one thing if the code chokes and you find out about it, its another
> if it does so and your program is left in the dark about it.
>
> At 11:39 AM 9/13/2005, you wrote:
>
> >hi eric,
> >
> >yes, i got the example files and been working on decoding them. the files
> >use spatial differencing that i haven't coded yet so i in the process of
> >implementing it. there's a small bug in my program that i haven't
> >been able to find.  the ds.mint.bin file decodes correctly up to data
> >point 12896 out of 37,887 and then the values  are distorted. i
> >reviewed the code, compared it to the NCEP grib2 decoder, etc. actually
> >i've spent 3 days on it.  at this point, i have to let it set for a couple
> >days before returning to it.
> >
> >robb...
> >
> >
> >
> >
> >
> >On Tue, 13 Sep 2005, Eric Davies wrote:
> >
> > > Hi Robb,
> > >
> > > I've been out of the office for a few days, and returned to a crashed disk
> > > (we lost power) so I thought I'd better check directly to see if the
> > > example grib files were useful. Did they reveal any interesting issues?
> > >
> > > Eric.
> > > At 03:34 PM 9/7/2005, you wrote:
> > > >Eric,
> > > >
> > > >is it possible to make the file available on the web or if it's
> > > >already available, tell me where so i can download the example.
> > > >
> > > >thanks,
> > > >robb...
> > >
> > > I've placed them on our web server where you should be able to grab them.
> > > The urls are:
> > > http://www.barrodale.com/example_bad_gribs/ds.mint.bin
> > > http://www.barrodale.com/example_bad_gribs/ds.pop12.bin
> > > http://www.barrodale.com/example_bad_gribs/ds.rhm.bin
> > > http://www.barrodale.com/example_bad_gribs/ds.snow.bin
> > >
> > > There is another one that causes a different problem, something about
> > > projection 10 not supported. Appears to be a mercator projection from the
> > > .gbx file it produces.
> > > http://www.barrodale.com/example_bad_gribs/ds.sky.bin
> > >
> > > Eric.
> > >
> > >
> > >
> > >
> > > >On Wed, 7 Sep 2005, Unidata Support wrote:
> > > >
> > > > > >To: address@hidden
> > > > > >From: Eric Davies <address@hidden>
> > > > > >Subject: trapped exception in Grib code hides failure
> > > > > >Organization: Barrodale Services
> > > > > >Keywords: 200509071707.j87H7Ajo024161
> > > > >
> > > > > I'm trying to read a grib file I downloaded from NOA using the
> > > > > toolsUI-2.2.10.jar .
> > > > > The grib file has a single variable with dimensions (Time, Y,X)
> > > > > Through the netcdf interface, I'm trying to read in one time step at a
> > > > time.
> > > > > The first time step succeeds, the subsequent fail due to the below
> > > > > ArrayIndexOutOfBounds exception.
> > > > >
> > > > > java.lang.ArrayIndexOutOfBoundsException: 49608
> > > > >          at
> > > > >
> > > >
> > ucar.grib.grib2.Grib2DataSection.complexUnpacking(Grib2DataSection.java:313)
> > > > >          at
> > > > ucar.grib.grib2.Grib2DataSection.<init>(Grib2DataSection.java:114)
> > > > >          at ucar.grib.grib2.Grib2Data.getData(Grib2Data.java:84)
> > > > >          at
> > > > >
> > > >
> > ucar.nc2.iosp.grib.Grib2ServiceProvider._readData(Grib2ServiceProvider.java:97)
> > > > >          at
> > > > >
> > ucar.nc2.iosp.grib.GribServiceProvider.readXY(GribServiceProvider.java:211)
> > > > >          at
> > > > >
> > > >
> > ucar.nc2.iosp.grib.GribServiceProvider.readData(GribServiceProvider.java:177)
> > > > >          at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:967)
> > > > >          at ucar.nc2.Variable._read(Variable.java:786)
> > > > >          at ucar.nc2.Variable.read(Variable.java:500)
> > > > >          at ucar.nc2.Variable.read(Variable.java:451)
> > > > >          at
> > > > >
> > > >
> > com.barrodale.openDAP2.TileProducerBasic.nextTile(TileProducerBasic.java:313)
> > > > >
> > > > > However, my application doesn't see the exception because it
> > appears to be
> > > > > trapped somewhere inside the Grib code and then ignored; I only
> > know that
> > > > > the exception has occurred because that same code is printing out
> > the above
> > > > > stack trace to the console.
> > > > >
> > > > > Presumably this trapping is a bug. Anybody got a work-around?
> > > > >
> > > > > I have a second application I use to dump the meta data of the grib 
> > > > > (as
> > > > > seen through the netcdf interface). It appears below:
> > > > >
> > > > > dimensions:
> > > > > time=6
> > > > > y=219
> > > > > x=173
> > > > > global attributes:
> > > > >
> > > > > .Conventions =  CF-1.0
> > > > > .Originating_center =  US National Weather Service (NWSTG) subcenter =
> > > > -9999
> > > > > .Product_Status =  Operational products
> > > > > .Product_Type =  Forecast products
> > > > > .FileFormat =  GRIB-2
> > > > > .DataType =  GRID
> > > > > .DatasetLocation =  c:\gribdata\ds.mint.bin
> > > > > .Processing =  direct read of GRIB into NetCDF-Java 2.2 API
> > > > > Minimum_temperature[ time y x]=
> > > > > ;
> > > > > Minimum_temperature.units =  K
> > > > > Minimum_temperature.long_name =  Minimum temperature @ surface
> > > > > Minimum_temperature.missing_value = []= 9999.0
> > > > > Minimum_temperature.coordinates =  lat lon
> > > > > Minimum_temperature.grid_mapping =  Lambert_Conformal
> > > > > Minimum_temperature.GRIB_param_discipline =  Meteorological_products
> > > > > Minimum_temperature.GRIB_param_category =  Temperature
> > > > > Minimum_temperature.GRIB_param_number = []= 5
> > > > > Minimum_temperature.GRIB_product_definition_type =  Average,
> > accumulation,
> > > > > extreme values or other statistically processed value at a
> > horizontal level
> > > > > Minimum_temperature.GRIB_level_type = []= 1
> > > > > time[ time]=
> > > > > ;
> > > > > time.long_name =  forecast time
> > > > > time.units =  hours since 2005-09-01T15:00:00Z
> > > > > time.GRIB2_referenceTime =  Start of forecast
> > > > > time._CoordinateAxisType =  Time
> > > > > Lambert_Conformal[]=
> > > > > ;
> > > > > Lambert_Conformal.grid_mapping_name =  lambert_conformal_conic
> > > > > Lambert_Conformal.standard_parallel = []= 25.0
> > > > > Lambert_Conformal.longitude_of_central_meridian = []= 265.0
> > > > > Lambert_Conformal.latitude_of_projection_origin = []= 25.0
> > > > > Lambert_Conformal.GRIB_earth_shape =  Earth spherical with radius
> > specified
> > > > > by producer
> > > > > Lambert_Conformal.GRIB_earth_shape_code = []= 1
> > > > > Lambert_Conformal.GRIB_spherical_earth_radius_meters = []= 6371200.0
> > > > > y[ y]=
> > > > > ;
> > > > > y.units =  km
> > > > > y.long_name =  y coordinate of projection
> > > > > y.standard_name =  projection_y_coordinate
> > > > > y.grid_spacing =  5.0794062 km
> > > > > y._CoordinateAxisType =  GeoY
> > > > > x[ x]=
> > > > > ;
> > > > > x.units =  km
> > > > > x.long_name =  x coordinate of projection
> > > > > x.standard_name =  projection_x_coordinate
> > > > > x.grid_spacing =  5.0794062 km
> > > > > x._CoordinateAxisType =  GeoX
> > > > > lat[ y x]=
> > > > > ;
> > > > > lat.units =  degrees_north
> > > > > lat.long_name =  latitude coordinate
> > > > > lat.standard_name =  latitude
> > > > > lat._CoordinateAxisType =  Lat
> > > > > lon[ y x]=
> > > > > ;
> > > > > lon.units =  degrees_east
> > > > > lon.long_name =  longitude coordinate
> > > > > lon.standard_name =  longitude
> > > > > lon._CoordinateAxisType =  Lon
> > > > > projectionCoordSys[]=
> > > > > ;
> > > > > projectionCoordSys._CoordinateAxes =  time y x
> > > > > projectionCoordSys._CoordinateTransforms =  Lambert_Conformal
> > > > >
> > > > > **********************************************
> > > > > Eric Davies, M.Sc.
> > > > > Barrodale Computing Services Ltd.
> > > > > Tel: (250) 472-4372 Fax: (250) 472-4373
> > > > > Web: http://www.barrodale.com
> > > > > Email: address@hidden
> > > > > **********************************************
> > > > > Mailing Address:
> > > > > P.O. Box 3075 STN CSC
> > > > > Victoria BC Canada V8W 3W2
> > > > >
> > > > > Shipping Address:
> > > > > Hut R, McKenzie Avenue
> > > > > University of Victoria
> > > > > Victoria BC Canada V8W 3W2
> > > > > **********************************************
> > > > >
> > > > >
> > > > >
> > > >
> > > > > Unidata User Support                                    UCAR Unidata
> > > > > (303)497-8643                                                  P.O. 
> > > > > Box
> > > > > address@hidden                                   Boulder, CO
> > > > >
> > > >
> > > > > Unidata WWW
> > > > >
> > > >
> > > > > NOTE: All email exchanges with Unidata User Support are recorded in 
> > > > > the
> > > > > Unidata inquiry tracking system and then made publicly available
> > > > > through the web.  If you do not want to have your interactions made
> > > > > available in this way, you must let us know in each email you send
> > to us.
> > > > >
> > > >
> > > >=======================================================================
> > ========
> > > >Robb Kambic                                Unidata Program Center
> > > >Software Engineer III                      Univ. Corp for Atmospheric
> > Research
> > > >address@hidden                   WWW:
> > http://www.unidata.ucar.edu/
> > > >=======================================================================
> > ========
> > > >
> > >
> > > **********************************************
> > > Eric Davies, M.Sc.
> > > Barrodale Computing Services Ltd.
> > > Tel: (250) 472-4372 Fax: (250) 472-4373
> > > Web: http://www.barrodale.com
> > > Email: address@hidden
> > > **********************************************
> > > Mailing Address:
> > > P.O. Box 3075 STN CSC
> > > Victoria BC Canada V8W 3W2
> > >
> > > Shipping Address:
> > > Hut R, McKenzie Avenue
> > > University of Victoria
> > > Victoria BC Canada V8W 3W2
> > > **********************************************
> > >
> > >
> > >
> >
> >===============================================================================
> >Robb Kambic                                Unidata Program Center
> >Software Engineer III                      Univ. Corp for Atmospheric 
> >Research
> >address@hidden                   WWW: http://www.unidata.ucar.edu/
> >===============================================================================
> >
>
> **********************************************
> Eric Davies, M.Sc.
> Barrodale Computing Services Ltd.
> Tel: (250) 472-4372 Fax: (250) 472-4373
> Web: http://www.barrodale.com
> Email: address@hidden
> **********************************************
> Mailing Address:
> P.O. Box 3075 STN CSC
> Victoria BC Canada V8W 3W2
>
> Shipping Address:
> Hut R, McKenzie Avenue
> University of Victoria
> Victoria BC Canada V8W 3W2
> **********************************************
>
>
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================