Re: [netcdf-java] Differing results reading an unlimited dimension between Java and C (?) binary distribution

Hi Jean-Michel.:

Ok, heres the situation:

Its actually illegal to use STRING here:

   writeableFile.addVariable("cellId", DataType.STRING, spatialDim);

you should either use:

   writeableFile.addVariable("cellId", DataType.CHAR, spatialDim);

or

   writeableFile.addStringVariable("cellId",  spatialDim, max_strlen);

However, the library did not catch this problem, so it put an incorrect 
"vsize" into the netcdf header.

The java-library uses this vsize, even though incorrect, and so 
everything works ok, but the C library ignores it, and so reads bad values.

Im not sure whether the cellId variable values are correct, however.

So technically you have invalid files, which you should rewrite if you 
need them to interoperate with C programs. I will release a new version 
of the library tomorrow  which should prevent the previous error.

Let me know if I can clarify anything else for you. Sorry for the problems.

John



address@hidden wrote:
> Hi John,
>
> I have put a small data set with the code that produce it, and exhibiting the 
> behavior mentioned, in ftp://ftp.csiro.au/netCDFJava/.
> The API version is
> ## Version 4.0.44.20090527.1935
> ## Build Date = 2009-05-27 19:35:38
>
> Of course let me know if you need further information/clarifications.
> Regards
>
> -----Original Message-----
> From: John Caron [mailto:address@hidden] 
> Sent: Tuesday, 2 June 2009 10:07 PM
> To: Perraud, Jean-Michel (CLW, Black Mountain)
> Cc: address@hidden
> Subject: Re: [netcdf-java] Differing results reading an unlimited dimension 
> between Java and C (?) binary distribution
>
> Hi Jean-Michel:
>
> Thats pretty strange. Can you put the file on an ftp or http server so we can 
> look at it?
>
> BTW, which version on netcdf-java are you using?
>
>
> address@hidden wrote:
>   
> > Hi,
> >
> > I have a lat/lon/time data set for which the lat dimension is unlimited. On 
> > disk the file is ~14 GiB. It was created using the Java API. More precisely, a 
> > C# application calling netCDF-Java through IKVM; worked beautifully. Colleagues 
> > and I tried to read this file from other tools (R via Rnetcdf, ncdump, etc.). 
> > The values of the lat variable are not read correctly (either all zeroes, or a 
> > mix with other numbers presumably from other variables).
> >
> > Has anyone had similar issues? This is really not a blocking issue and can live 
> > with it, but I am intrigued. Below are sample code and CDL. I cannot 
> > investigate more in-depth due to approaching milestones, but happy to provide 
> > clarifications if needed.
> > I am reluctant to cross-post for now, but this may well be an issue outside of 
> > the Java API.
> >
> > Cheers
> >
> > ### From IronPython or Jython, using the Java API, read works as expected:
> > clr.AddReferenceToFileAndPath(r"C:\lib\netcdf\netcdfAll-4.0.dll")
> > clr.AddReferenceToFileAndPath(r"C:\bin\ikvm\bin\IKVM.OpenJDK.ClassLibrary.dll")
> > from ucar.nc2 import NetcdfFile
> > netcdfFile = NetcdfFile.open(r"\\somename\tmp\scenarioA.nc")
> > latVar = netcdfFile.findVariable("lat")
> > latArray = latVar.read()
> > latArray
> > # prints:
> > # <ucar.ma2.ArrayFloat+D1 object at 0x000000000000002B [-39.097 -39.047 -38.997 
> > -3
> > # 8.947 -38.897 -38.847 -38.797 -38.747 -38.697 -38.647 -38.597 -38.547 -38.497 
> > -3
> > netcdfFile.close() 
> >
> >
> > ### using ncdump from a linux distro on a ncks subset. Same behavior without 
> > the full 14GB data set.
> >
> > netcdf smallScenarioA {
> > dimensions:
> >         lat = UNLIMITED ; // (4 currently)
> >         lon = 9 ;
> >         time = 40907 ;
> > variables:
> >         char cellId(lat, lon) ;
> >                 cellId:long_name = "Cell ID" ;
> >         float lat(lat) ;
> >                 lat:units = "degrees_north" ;
> >                 lat:axis = "Y" ;
> >                 lat:standard_name = "latitude" ;
> >         float lon(lon) ;
> >                 lon:units = "degrees_east" ;
> >                 lon:axis = "X" ;
> >                 lon:standard_name = "longitude" ;
> >         float runoff(lat, lon, time) ;
> >                 runoff:long_name = "runoff" ;
> >                 runoff:units = "mm" ;
> >                 runoff:missing_value = -9999.f ;
> >                 runoff:_FillValue = -9999.f ;
> >         int time(time) ;
> >                 time:units = "days since 1895-01-01" ;
> >                 time:axis = "T" ;
> >                 time:standard_name = "time" ;
> >                 time:long_name = "time" ;
> >
> > // global attributes:
> >                 :history = "Sat May 30 19:58:55 2009: ncks -d lat,30,33 -d 
> > lon,160,168 scenarioA.nc smallScenarioA.nc" ;
> > data:
> >
> >  cellId =
> >   "",
> >   "",
> >   "",
> >   "" ;
> >
> >  lat = 0, 0, 0, 0 ;
> >
> >  lon = 146.6, 146.65, 146.7, 146.75, 146.8, 146.85, 146.9, 146.95, 147 ;
> > _______________________________________________
> > netcdf-java mailing list
> > address@hidden
> > For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ 
> >