Re: 19990709: duplicate products in the noaaport idd

Bill,

We inspected the NOAAport data stream and found out that there was 21k
duplicate products but the duplicates had different sequence numbers.  The
current LDM includes the sequence numbers in the checksum of  the product
so these products are considered unique and the LDM processes them, not
disgards them.  This was on the adgenda to eliminate the sequence
number from the checksum but since Glenns departure the schedule is
uncertain.

Robb...

ps That 21K figure was for a 24 hour period, almost 1000 products/hour 
with an estimate size of 1 megabyte.

On Fri, 9 Jul 1999, William Noon wrote:

> Robb -- The following are some links to the data I gathered last month.
> I basically looked at just the products (skipped sequence number and I don't
> have arrival time) by checksumming the product and sorting by product
> header:chksum pair.  I compared the data delivered from the alden satellite
> dd+ service to the idd feed.  I have the idd feed for April, May and the 
> first few days of June but only May for the dd+ feed.
> 
> http://nrcc2.cit.cornell.edu/~noon/ddp_dup.9905  # May dd+ feed
> http://nrcc2.cit.cornell.edu/~noon/idd_dup.9904  # April idd feed
> http://nrcc2.cit.cornell.edu/~noon/idd_dup.9905  # May idd feed
> http://nrcc2.cit.cornell.edu/~noon/idd_dup.9906  # June idd feed
> 
> Here is a file with more details,  It list the number of products, 
> number of duplicate products, and the headers of the products with
> more than 4 duplicates followed by the crc and dup count.
> This is only for June 8, 1999.
> 
> http://nrcc2.cit.cornell.edu/~noon/chk_ldm.990608
> 
> This was just exploratory and I didn't have time to delve into it more.
> 
> I can send you the python script that got the data if you want.
> 
> Thanks -- Bill Noon
> 
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================