[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: I solved the Nexrad mystery! (was: Can rad handle new stuff...)



Dan and all,

Well, It's pretty ugly still, I wrote some comments to what's
going on, but here's what I used. This script *does* strip
off the uncompressed header to begin with, but does *not* strip off the
header that shows up at the beginning of the uncompressed data after the
first call to inflate.  I did that by hand, but another call to 
substr prior to the first print to STDOUT would do it.

Pete

#------- Cut Here --------------
#!/usr/bin/perl
use Compress::Zlib ;
$input = '' ;
binmode STDIN;
binmode STDOUT;

# Read in the whole file, 1 Mb should cover it. It'll end w/o error
# when it hits the end of stdin

read (STDIN, $input, 1000000);

# strip off the uncompressed header. Don't need the swap  in variable
# but with my limited perl knowledge I didn't want to mess with the
# original prog much

$tempin = substr($input,41);
$input=$tempin;

# While the input string still has stuff in it - this works
# because the inflate function modifies input to have whatever
# is left in it after the first complete chunk is uncompressed
# (the 4000 byte section Dan mentioned)

while (length($input) > 0 )
{
# Initialize a new structure - this needs to be done for each call
# to inflate. die gracefully if it fails from lack of memory, whatever

$x = inflateInit()
   or die "Cannot create a inflation stream\n" ;

# call inflate. It returns
#  $output - the decompressed stream, which will be max 4000 bytes
#  $status - a return code. 0 is success,
#                           1 is end of data (i.e. you didn't send
#                             a complete stream, and it expected
#                             more data
#  $input  - contains the remainder of $input after the stream that
#            got decoded into $output
#

($output, $status) = $x->inflate(\$input) ;

# write $output to STDOUT. each write gets appended to the previous, so
# you eventually build up the whole file. The first write still needs
# to be modified to strip off the header that is in the file after
# decompression

print $output;

# Gracefully exit if the decode failed. This means something bad
# happened. This is how I discovered that I had both the compressed
# and encrypted data in one file. The compressed stuff worked but the
# encrypted didn't.

if ($status < 0 or $status > 2) {
die "bogus second header";
}
}
#------- Cut Here --------------

In a previous message to me, you wrote: 

 >I have also been able to uncompress the ARX NIDS data but I needed to do some 
 >tricks to get it to work.  This is what I did... its pretty ugly!!
 >
 >It turns out that NWS has split the NIDS file into 4000 byte blocks and then 
 >compressed the data using the zlib compress utility.  Then the compressed 
 >blocks are sent out sequentially as part of the whole product.  This means 
 >you have separate the blocks to decode.  
 >
 >The problem is that the uncompress utility doesn't work that effectively.  
 >The problem is you don't know the size of the compressed blocks.  The block 
 >sizes are buried in the SBN header which is masked off by the LDM.  So you 
 >have to guess what the block sizes are.  The uncompress utility will return 
 >the size of the uncompressed data but won't reveal the size of the compressed 
 >data. 
 >
 >What I've done is to forward through the data 1000 bytes at a time and then 
 >try to uncompress the data.  If the function returns an error, I increment 
 >through the data a byte at a time until I get a valid return on the 
 >uncompression.  This is just plain ugly....
 >
 >What I'm looking for is a simpler solution!!  If there is another utility 
 >that gives me the compressed block sizes, this would make my job much easier!
 >
 >BTW, I read where the uncompress utility will uncompress all the data if the 
 >file is memory mapped???  Otherwise, it only works a block at a time.
 >
 >So Pete, can you forward the Perl script so I can check what functions are 
 >used??
 >
 >Dan.... from cloudy Chicago!!


--
+>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>+<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<+
^ Pete Pokrandt                    V 1447  AOSS Bldg  1225 W Dayton St^
^ Systems Programmer               V Madison,         WI     53706    ^
^                                  V      address@hidden       ^
^ Dept of Atmos & Oceanic Sciences V (608) 262-3086 (Phone/voicemail) ^
^ University of Wisconsin-Madison  V       262-0166 (Fax)             ^
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<+>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>+


pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy