L3 Multiday (8-Day & Monthly) Algorithm
There are a number of general L3 Multiday algorithm characteristics:
a) Only the L3 Daily files are used as input. (Greatly improves algorithm efficiency.)
b) L3 Daily and Multiday HDF files use an identical grid, SDS dimensions, and histogram bin definitions. (This is an algorithm requirement.)
c) There is no “valid range” check, only “Fill_Value” grid cells are universally excluded.
d) One of three weighting schemes are used to compute Multiday statistics from Daily statistics.
Computational weighting schemes for multiday statistics
The weighting technique used for each SDS is documented in the L3 File Specification (File Spec), which can be found on the MODIS Atmosphere web site (modis-
atmos.gsfc.nasa.gov/MOD08_M3/spec.html). This information is also attached to each SDS within the HDF file by a local attribute called “Weighting” – and if this is not set to “None,” there is an additional local attribute called “Weighted_Parameter_Data_Set,” which specifies which SDS in the Daily L3 (input) file is used to weight the daily grid cell statistics in the computation of the multiday statistics.
The various multiday weighting schemes used in L3 are:
a) Unweighted (a simple time-averaged mean, meaningful for computing temperature averages).
b) Pixel-count weighted (a count-averaged mean, used to ensure computed means match means computed from histograms).
c) Pixel-count weighted with pixel-count screen (special logic implemented for Aerosol-derived parameters to remove low confidence (low pixel count) daily grids usually occurring at the poleward terminator of the retrieval domain).
Table 4 summarizes what multiday weighting scheme is used for each broad set (grouping) of L3 parameters.
Table 4. L3 Multiday weighting scheme used (generally) for categories of parameters.
Types of multiday statistics computed
A total of 12 different general types of statistics are computed in the multiday (Eight Day or Monthly) products. They are grouped into broad categories and always computed in pre-determined sets.
Statistics in the Eight Day and Monthly files are always based on the set of L3 Daily pixels read from the L3 Daily input product files that cover the time period being summarized (8 days or 1 month).
Simple statistics based on the daily mean
Mean_Mean and Std_Deviation_Mean statistics can either be unweighted or weighted depending on a local attribute setting (set per instructions from the L2 algorithm team) in the HDF structure file. An unweighted statistic is computed by taking an average of all L3 daily values for a given 1° × 1° L3 grid cell for the time period in question. Weighted statistics are computed by using information stored in an SDS local attribute called
“Weighted_Parameter_Data_Set,” typically a Daily Pixel Count or Fraction SDS. To determine the technique used to weight the daily grids, refer to the local attribute “Weighting” found in the file specification or a local attribute attached to each SDS in the HDF file.
• Mean_Mean. The Scientific Data Set (SDS) name suffix “_Mean_Mean” stands for
“Mean of the Daily Mean.” This statistic is computed by averaging non-fill Daily Means from the L3 Daily files within the (eight day or monthly) time period being summarized. The averaging computation uses the weighting scheme specified in the local attribute “Weighting.” This can be set to “Unweighted,” “Pixel-Weighted,” or “Pixel_Weighted_Screen” (See 4.1 for details).
• Mean_Std. The Scientific Data Set (SDS) name suffix “_Mean_Std” stands for “Standard Deviation of the Daily Mean.” This statistic is computed by reading all the non-fill Daily Means from the L3 Daily files within the (eight day or monthly) time period being summarized and then computing their standard deviation.
• Mean_Min. The Scientific Data Set (SDS) name suffix “_Mean_Min” stands for “Minimum of the Daily Mean.” This statistic is computed by reading all the non-fill Daily Means from the L3 Daily files within the (eight day or monthly) time period being summarized and reporting the minimum value. There is never any weighting performed for this statistic.
• Mean_Max. The Scientific Data Set (SDS) name suffix “_Mean_Max” stands for “Maximum of the Daily Mean.” This statistic is computed by reading all the non-fill Daily Means from the L3 Daily files within the (eight day or monthly) time period being summarized and reporting the maximum value. There is never any weighting performed for this statistic.
Simple statistics based on the daily standard deviation
Std_Deviation_Mean. The Scientific Data Set (SDS) name suffix
“Std_Deviation_Mean” is a proxy for “Mean of the Daily Standard Deviation.” This statistic is computed by reading all the non-fill Daily Standard Deviations from the L3 Daily files within the (eight day or monthly) time period being summarized, then averaging the standard deviations using the same weighting as used for the Mean_Mean.
QA-weighted statistics based on the daily mean
All Mean and Standard Deviation statistics computed can either be unweighted or weighted depending on a local attribute setting in the HDF structure file. An unweighted statistic is computed by taking an average of all L3 daily values for a given 1° L3 grid cell for the time period in question. A weighted statistic is computed by using a “Weighted_Parameter_Data_Set,” typically a Pixel Count or Fraction SDS, to weight each of the Daily grid cell values. To determine the technique used to weight the daily grids, refer to the local attribute “Weighting” found in the file specification or a local attribute attached to each SDS in the HDF file.
• QA_Mean_Mean. The Scientific Data Set (SDS) name suffix “QA_Mean_Mean” stands for “Mean of the Daily QA-weighted Mean.” This statistic is computed by averaging all the non-fill Daily QA-weighted Mean grid cells from the L3 Daily files within the (eight day or monthly) time period being summarized using the same weighting used for the Mean_Mean.
• QA_Mean_Std. The Scientific Data Set (SDS) name suffix “QA_Mean_Std” stands
for “Standard Deviation of the Daily QA-weighted Mean.” This statistic is computed by reading all the non-fill Daily QA-weighted Means from the L3 Daily files within the (eight day or monthly) time period being summarized, then computing their standard deviation.
• QA_Mean_Min. The Scientific Data Set (SDS) name suffix “QA_Mean_Min” stands for “Minimum of the Daily QA-weighted Mean.” This statistic is computed by reading all the non-fill Daily QA-weighted Means from the L3 Daily files within the (eight day or monthly) time period being summarized and reporting the minimum value. There is no additional weighting performed for this statistic.
• QA_Mean_Max. The Scientific Data Set (SDS) name suffix “QA_Mean_Max” stands for “Maximum of the Daily QA-weighted Mean.” This statistic is computed by reading all the non-fill Daily QA-weighted Means from the L3 Daily files within the (eight day or monthly) time period being summarized and reporting the maximum value. There is no additional weighting performed for this statistic.
• QA_Std_Deviation_Mean. The Scientific Data Set (SDS) name suffix “QA_Std_Deviation_Mean” is a proxy for “Mean of the Daily QA-weighted Standard Deviation.” This statistic is computed by reading all the non-fill Daily QA-weighted Standard Deviations from the L3 Daily files within the (eight day or monthly) time period being summarized, then averaging the standard deviations using the same weighting as used for the Mean_Mean.
Fraction statistics. These statistics are only used for computing cloud fraction.
• FMean. Mean of the Daily Cloud Fraction.
• FStd. Standard Deviation of the Daily Cloud Fraction.
4.2.5. Pixel count statistics. These statistics are only computed for some parameters. It is similar to a histogram computation except that instead of multiple bins there is only a single bin that covers the full range of all non-fill L2 data that are read in for each L3 1° × 1° grid cell.
• Pixel Count. The count of all non-fill L2 pixel data that are read in and used to compute statistics at L3. This is computed by simply summing the Daily Pixel Count SDS.
Logarithm statistics. These statistics are only computed for cloud optical thickness parameters. The mean statistic is typically weighted using the “Pixel_Weighted” scheme, which weights each daily mean by a pixel count SDS (this allows the multiday means to match those computed from histograms).
• Log_Mean_Mean. Mean of the Daily Log Mean.
• Log_Mean_Std. Standard Deviation of the Daily Log Mean.
• Log_Mean_Min. Minimum of the Daily Log Mean.
• Log_Mean_Max. Maximum of the Daily Log Mean.
• Log_Std_Deviation_Mean. Mean of the Daily Log Standard Deviation.
• QA_Log_Mean_Mean. Mean of the Daily QA Log Mean.
• QA_Log_Mean_Std. Standard Deviation of the Daily QA Log Mean.
• QA_Log_Mean_Min. Minimum of the Daily QA Log Mean.
• QA_Log_Mean_Max. Maximum of the Daily QA Log Mean.
• QA_Log_Std_Deviation_Mean. Mean of the Daily QA Log Standard Deviation.
Uncertainty statistics.
These statistics are only reported for a few selected Cloud Optical Property parameters. Monthly uncertainty calculations are derived from daily uncertainties; calculations assume that individual daily uncertainties are uncorrelated with respect to each other. The equation to compute multiday uncertainties is as follows. Uncertainty = , (2) where,
i = DailyUncertainties
wi = DailyPixelCountWeights
It should be noted that L2 uncertainties are reported in percentage (valid range 0 to 200%) – these are called relative uncertainties. In L3, uncertainties are reported as absolute uncertainties in the same units as the parameter whose uncertainty is being measured. To convert these back into relative uncertainties (%) one must divide the L3 uncertainty by the L3 mean value of the parameter in question. The conversion of relative to absolute uncertainties in the Daily (D3) file was done to make the computation of the multiday (E3 and M3) uncertainties easier to add to the multiday production software.
• Mean_Uncertainty. A multiday absolute uncertainty estimate derived from the daily absolute uncertainty estimate.
• QA_Mean_Uncertainty. A multiday QA-weighted absolute uncertainty estimate derived from the daily QA-weighted absolute uncertainty estimate.
Logarithm of uncertainty statistics. An estimate of the log uncertainty that is derived from pixel-level uncertainties.
• Log_Mean_Uncertainty. A multiday absolute log uncertainty estimate derived from the daily absolute log uncertainty estimate.
• QA_Log_Mean_Uncertainty. A multiday QA-weighted absolute log uncertainty estimate derived from the daily QA-weighted absolute log uncertainty estimate.
Histogram. A distribution of L2 pixels.
• Histogram. A histogram that contains pixel counts showing the distribution of nonfill L2 Pixels that went into the computation of L3 statistics for each L3 grid cell. In the eight day and monthly products, this is computed by simply summing the daily counts in each histogram bin. Histogram bin boundaries are set by a local attribute attached to the histogram SDS. Note that these L2 count values are sampled totals for
Water Vapor, Cirrus Detection, and Cloud Optical Property parameters (See Table 1).
It should also be noted that the lowest (1st) histogram bin includes L2 data points that
fall on either the lowest (1st) bin boundary or the 2nd bin boundary (exactly). All subsequent bins only contain points that fall on the higher bin boundary. Any L2 data point that falls outside the specified range of bin boundaries is not counted.
Histogram of confidence. A distribution of L2 pixel-level retrieval confidence.
• Confidence_Histogram. A histogram that contains pixel counts showing the number of Questionable (QA = 1), Good (QA = 2), Very Good (QA = 3), and Total Level-2 Input Pixels (# non-fill L2 Pixels) that went into the computation of L3 statistics for each L3 grid cell. Note that these values are sampled totals for Water Vapor, Cirrus Detection, and Cloud Optical Property parameters (See Table 1). In the eight day and monthly products, this is computed by simply summing the daily counts in each bin.
Joint histogram. A distribution of L2 pixels comparing one parameter against another.
• Joint_Histogram. A 2-dimensional histogram that contains pixel counts showing the distribution of non-fill L2 Pixels comparing one parameter against another. It should be noted that only a few Cloud parameters have Joint Histograms defined. In the eight day and monthly products, this is computed by simply summing the daily counts in each joint histogram bin. Histogram bin boundaries are set by a local attribute attached to the histogram SDS. Note that these L2 count values are sampled totals for Cloud Optical Property parameters (See Table 1). It should also be noted that the lowest (1st) histogram bin includes L2 data points that fall on either the lowest (1st) bin boundary or the 2nd bin boundary (exactly). All subsequent bins only contain points that fall on the higher bin boundary. Any L2 data point that falls outside the specified range of bin boundaries are not counted.
Joint regression. A regression fit of L2 pixels comparing one parameter against another. Note that these are only computed for a few Aerosol parameters.
• Regression_Slope. An unweighted mean of the Daily regression slope that describes the linear fit distribution of L2 pixels when comparing one parameter against another.
• Regression_Intercept. An unweighted mean of the Daily regression intercept that describes the linear fit distribution of non-fill L2 pixels when comparing one parameter against another.
• Regression_R_squared. An unweighted mean of the Daily regression r2 that describes the linear fit distribution of non-fill L2 pixels when comparing one parameter against another.
• Regression_Mean_Square_Error. An unweighted mean of the Daily regression mean squared error (MSE) that describes the linear fit distribution of non-fill L2 pixels when comparing one parameter against another.