Response to Request for Correction (RFC)

History / Timeline of this RFC

The RFC was received by the NOAA Information Quality Act (IQA) Officer on January 17,
2024 pursuant to NOAA’s Information Quality Guidelines that implement Section 515, Office of
Management and Budget (OMB) and Department of Commerce (DOC) Information Quality
Guidelines. The NOAA IQA Officer provided you an update on May 16, 2024 that additional
time was needed to process your request.

About NOAA Information Quality Guidelines

NOAA Information Quality Guidelines describe the administrative mechanisms for NOAA’s
pre-dissemination review of information products and describe mechanisms to enable affected
persons to request corrections from NOAA regarding disseminated information that they believe
does not comply with NOAA, DOC, or OMB guidelines.

Summary of the RFC

In your RFC, you petitioned NOAA to correct the NOAA National Centers for Environmental
Information (NCEI) U.S. Billion-Dollar Weather and Climate Disasters data set. NOAA refers to
this below as the “Billion Dollar Disaster data set” or the “data set”, and you refer to this as the
“billion dollar disaster data set”.

The RFC requests seven corrections to the Billion Dollar Disaster data set:

1. Publish all data, including all historical versions of the Billion Dollar Disaster data set;
2. Document and publish baseline loss estimates and their provenance;
3. Clearly describe all methodologies employed to adjust baseline data;
4. Document every change made to the data set, give each successive version of the data set
a unique name, and publish all versions of the data;
5. Maintain all historical versions of the data set in a publicly accessible archive;
6. Subject the methods and results to annual peer review by experts independent of NOAA
and with relevant expertise. Make the peer review reports public;
7. Align with IPCC methods and standards for any claims of detection and attribution.

The materials you submitted include a pre-print of your article Scientific Integrity and the U.S.
“Billion Dollar Disasters” dated January 5, 2024. The pre-print identifies additional specificity
regarding the above seven corrections and was reviewed in concert with the RFC.

Page 1 of 10 of Response to Request for Correction (RFC)

Summary of NOAA Response

Information relevant to this response can be found in the following public documents. There are
two peer-reviewed articles related to the Billion Dollar Disaster data set:

● Smith, A., and J. Matthews, 2015: Quantifying Uncertainty and Variable Sensitivity
within the U.S. Billion-dollar Weather and Climate Disaster Cost Estimates. Natural
Hazards., DOI: 10.1007/s11069-015-1678-x
● Smith, A., and R. Katz, 2013: U.S. Billion-dollar Weather and Climate Disasters: Data
Sources, Trends, Accuracy and Biases. Natural Hazards., DOI:
10.1007/s11069-013-0566-5

NOAA has carefully considered all requests for correction in your letter.

NOAA examined the Billion-Dollar Weather and Climate Disasters data set, and determined that
the data set meets the threshold for influential scientific information (ISI) under NOAA
Information Quality Guidelines. NOAA will be reviewing and updating its data management
practices for the data set as a result. Thank you for your patience while this additional review
was undertaken.

Considering your RFC, NOAA will undertake actions to improve the documentation and
transparency of the data set for greater compliance with NOAA’s Information Quality
Guidelines. These actions include:
● Disclosing and facilitating access to several publicly available input sources.
● Disclosing and facilitating access to methodologically relevant factors, such as insurance
penetration rates and underlying economic data.
● Formalizing procedures currently used to check robustness of NOAA’s analyses, and
stating these on the Billion Dollar Disaster website.
● Instituting a peer-review cycle consistent with the data set’s ISI determination.

NOAA also determined that no correction is warranted for several items raised in the letter.
These include:
● NOAA can and does disclose the use of private sector insurance data. The data
themselves cannot be revealed for privacy and intellectual property considerations.
Following the general principle that we maximize the use of open data in our analyses,
NOAA will evaluate the criticality of these non-disclosable data sets for continued
inclusion in the data set and its derivative products.
● Requests for correction with which current practice already aligns. These practices
include publication of previous reports, archival and access of quarterly reports, instituted

Page 2 of 10 of Response to Request for Correction (RFC)

 in 2020, and publication of unadjusted loss data. We will indicate where these
information resources are available below.
● Requests for changes in methodology, such as the addition of sub-billion-dollar events, or
adoption of new methods, such as adopting standards for attribution. We acknowledge the
value of these suggestions, but note that they are outside the scope of the RFC process.

Finally, NOAA notes that the RFC did not identify specific data points that need correcting. In
conducting its review of the RFC, NOAA has not identified any data inaccuracies in the Billion
Dollar Disaster data set. NOAA’s actions in response to the RFC are to ensure the Billion Dollar
Disaster data set complies with the highest standards for transparency and information quality.

Detailed NOAA Response to Request for Correction

Regarding the sources used:

NOAA details the sources underlying the Billion Dollar Disaster data set near the beginning of
its presentations; in the peer-reviewed, published research paper Smith, A., and R. Katz, 20131.
Sources already publicly disclosed as providing data following a disaster include the Insurance
Services Office (ISO) Property Claim Services (PCS), Federal Emergency Management Agency
(FEMA) National Flood Insurance Program (NFIP) and Presidential Disaster Declaration (PDD)
assistance, the National Interagency Fire Center (NIFC), the Army Corps, and the United States
Department of Agriculture (USDA) National Agricultural Statistics Service (NASS) & Risk
Management Agency (RMA), and state agencies.

NOAA has determined that the additional sources of information should be disclosed:
● Munich Re
● NOAA’s Storm Events database
● United States Department of Defense Reports2

1
Smith, A., and R. Katz, 2013#: U.S. Billion-dollar Weather and Climate Disasters: Data Sources, Trends, Accuracy
and Biases. Natural Hazards., DOI: 10.1007/s11069-013-0566-5; and on the homepage of the website:
https://www.ncei.noaa.gov/access/billions/
2
NOAA relies on reports made publicly available after each disaster. Reports relied on thus far are available the
following links:
● https://www.marinecorpstimes.com/news/your-marine-corps/2018/12/12/36-billion-price-tag-to-rebuild-lej
eune-buildings-damaged-by-hurricane-florence/
● https://www.afimsc.af.mil/News/Article-Display/Article/3560414/five-years-after-hurricane-michael-afims
c-continues-shaping-tyndall-as-installa/
● https://www.offutt.af.mil/Home/Flood-Rebuild/
● https://www.pacaf.af.mil/News/Article-Display/Article/3543834/afimsc-leads-andersen-rebuild-in-typhoon
-mawar-aftermath/

Page 3 of 10 of Response to Request for Correction (RFC)

 ● United States Federal Highway Administration Reports, available at:
https://highways.dot.gov/newsroom/fhwa-announces-729-million-repair-roads-and-bridg
es-damaged-natural-disasters-and-extreme

In addition, NOAA has determined the specific state agency resources relied on should be
disclosed:
● CalFire: https://www.fire.ca.gov
● Florida Office of Insurance Regulation: https://www.floir.com/
● Univ. of Missouri/MU Extension:
https://extension.missouri.edu/news/calculating-winter-feed-costs-for-beef-cows

Regarding underlying economic data, the following list specifies the available underlying data:
● USDA RMA Data: https://rma.usda.gov/SummaryOfBusiness/CauseOfLoss
● USDA NASS Data: https://quickstats.nass.usda.gov/
● NOAA Storm Data: https://www.ncdc.noaa.gov/stormevents/
● NIFC: https://www.nifc.gov/fire-information
● FEMA NFIP Data: https://www.fema.gov/about/openfema/data-sets
● FEMA PA Data: https://www.fema.gov/openfema-data
set-public-assistance-applicants-v1
● FEMA IA Data: https://www.fema.gov/about/openfema/data-sets#individual

The Billion Dollar Disaster website will be updated to include all of these sources. In the interest
of improving transparency, NOAA will identify contributed data from these and other sources
that were used in past Billion Dollar Disaster reports, facilitate access to archived versions at
their host institutions, and will investigate whether and how to archive contemporaneous data
sets from these sources.

NOAA relies on some private sector insurance data which cannot be revealed for privacy and
intellectual property considerations. These sources contain property insurance claims data
provided to NOAA under an agreement to refrain from disclosing the information. NOAA
understands that where public access to the information cannot be provided due to compelling
privacy and confidentiality interests, there must be an especially rigorous robustness check to
analytic results. That robustness check is built into the methodology as detailed in prior peer
reviewed reports. In addition to using peer reviewed methods, the Billion Dollar Disaster data set
includes the following additional robustness checks:
● The annual technical reports are internally peer reviewed by NOAA staff;
● The Billion Dollar Disaster data set is compared to other independent disaster reports that
estimate the same costs using similar data sources and methods, such as Gallagher Re, the
Insurance Information Institute, AON, and Munich Re.

Page 4 of 10 of Response to Request for Correction (RFC)

These robustness checks indicate that NOAA’s report, overall, is conservative in its estimates
versus contemporary industry analyses. For instance, there were 28 disasters in 2023, and for 12
of these events, NCEI’s damage estimates were smaller than those in other sources, such as
Gallagher Re, AON, and Munich Re. For the other 16 events, the NCEI estimates generally fell
within the range of the other data sources. The Gallagher Re, AON and Munich Re 2023 disaster
cost reports are available:
● https://www.ajg.com/gallagherre/-/media/files/gallagher/gallagherre/news-and-insights/2
024/january/natural-catastrophe-and-climate-report-2023.pdf
● https://assets.aon.com/-/media/files/aon/reports/2024/climate-and-catastrophe-insights-re
port.pdf
● https://www.munichre.com/en/company/media-relations/media-information-and-corporat
e-news/media-information/2024/natural-disaster-figures-2023.html

NOAA has determined that these robustness check procedures should be formalized and stated
on the Billion Dollar Disaster website.

Regarding preserving historical versions of the data and documenting changes:

Updates to the data set are now made in a transparent manner. From 2003-2015, updates to the
data set were made annually and only the most recent version of the data set could be
downloaded. Versions from this period are not available. From 2016-2023, NOAA supplemented
the annual update with an annual report. These reports are available on the website under
“References.” In 2020, NCEI added an archive of the data set every quarter. In doing so, NOAA
now keeps historical versions and makes them available. There are currently 18 versions of
quarterly data updates available for download in the NCEI archive under the "Lineage" tab:
https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.nodc:0209268.
All quarterly updates will continue to be archived and available at the above link.

Finally, starting in 2023, NOAA began updating the data monthly on the website, and this data
can be downloaded under the "Events" section of the website above. Through these updates,
NOAA introduces events into the time series as they "inflate" their way above $1B in costs in
today's dollars. Every year, this leads to the introduction of several new events added from earlier
in the time series. This is noted in the NOAA FAQ section.

It should be noted that changes from quarter to quarter in NOAA’s analysis result from updates
and additions to the input data, not from changes in methodology.

Regarding baseline loss estimates and their provenance and publishing relied upon
methodologies:

Page 5 of 10 of Response to Request for Correction (RFC)

NOAA has determined no correction is warranted to publish baseline loss estimates. The total,
direct loss baseline data for all the U.S. Billion Dollar Disaster data set events are available for
download in several formats. To download the unadjusted, baseline cost estimates for any year or
event click the ‘Unadjusted’ button at the top right of the events table at the following link:
https://www.ncei.noaa.gov/access/billions/events/. Then, click a data format button (i.e., CSV,
JSON, XML, PDF) at the top left of the events table to download the data. These baseline data
incorporate a variety of private and public sector data.

NOAA has also determined that no correction is warranted to publish the adjusted baseline loss
estimates. The CPI-adjusted baseline loss data can be downloaded at the same link as the
unadjusted baseline loss data and is available in the same variety of formats.

To account for uninsured losses, a common data standard in this type of analysis is to use
multipliers of insured losses (i.e., the reciprocal of the insurance penetration rate multiplied by
the known insured loss). The penetration rate data differ by peril, geography, and asset class and
are often from private sector information. As discussed above, certain source information cannot
be disclosed due to privacy and agreements in place with the underlying sources. However,
NOAA has determined that additional information regarding the data sources can and should be
disclosed for the insurance penetration rates. The Billion Dollar Disaster website will be updated
to include the following list of data resources for insurance penetration rates:

● For automotive:
○ Insurance Information Institute:
https://www.iii.org/article/background-on-compulsory-auto-uninsured-motorists
https://www.iii.org/table-archive/21141
○ Insurance Research Council:
https://www.insurance-research.org/sites/default/files/IRC%20Uninsured%20Mot
orists%20Summary%20Page.pdf
● For residences:
○ Census American Housing Surveys:
http://www.census.gov/programs-surveys/ahs.html
○ Insurance Information Institute:
https://resilience.iii.org/disasters/floods/#5.5/30.729/-96.609
https://resilience.iii.org/disasters/ratings/#5/30.91/-90.81
● For agriculture and crops:
○ USDA:
https://www.ers.usda.gov/topics/farm-practices-management/risk-management/cr
op-insurance-at-a-glance/
https://www.rma.usda.gov/SummaryOfBusiness
● Business data based on private sector industry surveys:

Page 6 of 10 of Response to Request for Correction (RFC)

 ○ National Association of Insurance Commissioners (NAIC 2007) Marshall &
Swift/Boeckh (2011) MSB 4Q 2011 RTC Product Bulletin. Marshall &
Swift/Boeckh (2013) MSB ITV index for residential properties
○ https://www.irmi.com/articles/expert-commentary/insuring-the-home-based-busin
ess-(part-1)
○ https://www.independentagent.com/media/pages/2004/na20040225120203.aspx
○ MunichRe Natcat historic event data
○ Travelers (2014) Business risk index.

NOAA has determined no correction is warranted to further describe the methodologies for
computing the Billion Dollar Disaster data set. These are publicly available at the following two
peer-reviewed publications:

Smith, A., and J. Matthews, 2015: Quantifying Uncertainty and Variable Sensitivity
within the U.S. Billion-dollar Weather and Climate Disaster Cost Estimates. Natural
Hazards., DOI: 10.1007/s11069-015-1678-x

Smith, A., and R. Katz, 2013: U.S. Billion-dollar Weather and Climate Disasters: Data
Sources, Trends, Accuracy and Biases. Natural Hazards., DOI:
10.1007/s11069-013-0566-5

These publications disclose the methodology for calculating total direct losses. The papers
describe the methods, equations, parameters, and assumptions used. Along with a narrative
description, the 2013 publication provides a table detailing the method for developing
billion-dollar disaster event loss calculations by disaster type and data sources to convert from
insured to total losses. A copy of that table3 is reproduced below:

3
See page 9 of Smith, A., and R. Katz, 2013: U.S. Billion-dollar Weather and Climate Disasters: Data Sources,
Trends, Accuracy and Biases. Natural Hazards., DOI: 10.1007/s11069-013-0566-5

Page 7 of 10 of Response to Request for Correction (RFC)

Regarding bias correction adjustments, the publications specify the methodology used to
determine if an adjustment was warranted and how that adjustment was made to the baseline
data. The correction referenced in the 2015 paper refers to adjustments made to the inputs rather
than the methodology.

Finally, regarding indirect costs, the Billion Dollar Disaster data set only relies on direct costs in
accordance with the peer reviewed publications. The publications state that “the loss estimates
reflect direct effects of weather and climate events (i.e., not including indirect effects) and
constitute total losses (i.e., both insured and uninsured).”4 The publications detail the direct
losses considered:

physical damage to residential, commercial and government/municipal buildings,

material assets within a building, time element losses (i.e., businesses interruption),
vehicles, boats, offshore energy platforms, public infrastructure (i.e., roads, bridges,
buildings) and agricultural assets (i.e., crops, livestock, timber).5

The publications also state what is not considered (i.e., losses to natural capital/assets, healthcare
related losses, or values associated with loss of life).6

NOAA determined that no correction regarding Hurricane Idalia is required. This event was
calculated consistent with NOAA’s standard operational practice utilizing the above formulas
and updating the impact based on the latest updates from the public and private sector. NCEI's
initial estimate of $2.5 billion in October 2023 increased to $3.6 billion by December 2023
largely due to more insurance claims data (ISO/PCS) becoming available in December 2023
4
Page 3 of Smith, A., and J. Matthews, 2015: Quantifying Uncertainty and Variable Sensitivity within the U.S.
Billion-dollar Weather and Climate Disaster Cost Estimates. Natural Hazards., DOI: 10.1007/s11069-015-1678-x
5
Id.
6
Id.

Page 8 of 10 of Response to Request for Correction (RFC)

valued at approximately $900 million. For larger, impactful events like a hurricane, the insurance
industry will provide several updates to their survey of losses, as more claims data become
available. This amount excludes additional Idalia losses to USDA agriculture, flood losses, and
infrastructure, among other assets. Similar to the NCEI analysis, insurance (AON) and
reinsurance (Gallagher Re) annual reports concluded the total, direct U.S. losses for Idalia at $3.5
billion.

Regarding ongoing peer review:

NOAA’s Information Quality Guidelines do not require or recommend annual peer review for all
information products. However, peer review is required for information that has been determined
to meet the threshold for influential scientific information (ISI). For continually updated ISI data
sets, peer review should occur at least once every 5 years.

NOAA has determined that the Billion-Dollar Weather and Climate Disasters data set is
influential scientific information under NOAA Information Quality Guidelines. Consequently,
the methodology and data set should undergo peer review at least once every five years. The last
peer review undertaken for the methodology was for the 2015 peer reviewed publication
discussed above. NOAA will undertake a peer review process consistent with the IQA for the
Billion Dollar Disasters data set and is now developing the methodology for that review process.

Regarding Alignment on Detection and Attribution:

NOAA appreciates the suggestion to align with the Intergovernmental Panel on Climate Change
methods and standards for detection and attribution. Whether further alignment is warranted is a
question of policy. This request falls outside the scope of NOAA’s Information Quality
Guidelines. OMB M-19-15 states that the request for correction process is not an opportunity for
the agency or requestor “to debate the policy itself” but is only an opportunity to request
corrections to the technical facts or scientific determinations that are the basis for the information
underlying such policies. Further OMB guidance states that NOAA should not opine in response
to an RFC on statements regarding policy. See OMB, Improving Implementation of the
Information Quality Act: Frequently Asked Questions (Dec. 2023).

Your Right to Appeal

If you are dissatisfied with the response, you may submit a Request for Reconsideration (RFR)
as described in NOAA’s Information Quality Guidelines. NOAA requests that any such RFR be
submitted within 90 days of the date of NOAA’s response. If you choose to submit an RFR,
please send a written request to the NOAA Information Quality Guidelines Processing Staff by

Page 9 of 10 of Response to Request for Correction (RFC)

mail (NOAA IQA Officer, 14th & Constitution Avenue NW, Bldg. HCHB, Washington DC
20230-0001); or electronic mail (adrienne.thomas@noaa.gov). If you submit an RFR, please
reference the case number assigned to this original Request for Correction (2024-01-RFC).
Additional information about how to submit an RFR is located on the NOAA Information
Qualities Guidelines website at
https://www.noaa.gov/organization/information-technology/policy-oversight/information-quality
/information-quality-guidelines.

Page 10 of 10 of Response to Request for Correction (RFC)