Datacenter handbook-pages-14
Datacenter handbook-pages-14
Christopher O. Muller
Muller Consulting, Lawrenceville, Georgia, United States of America
Data Center Handbook: Plan, Design, Build, and Operations of a Smart Data Center, Second Edition. Edited by Hwaiyu Geng.
© 2021 John Wiley & Sons, Inc. Published 2021 by John Wiley & Sons, Inc.
239
240 COrrosion And Contamination Control For Mission Critical Facilities
center, and application of gas‐phase and particulate filtration TABLE 15.1 ISA classification of reactive environments
to fresh (outside) air systems, recirculating air systems, and G1 G2 G3 GX
computer room air conditioners. Severity level Mild Moderate Harsh Severe
The best possible control of airborne pollutants would
allow for separate sections in the mechanical system for par- Copper reactivity
level (Å)a <300 <1,000 <2,000 ≥2,000
ticulate and gaseous contaminant control. However, physical
limitations placed on mechanical systems, such as restric- Silver reactivity
tions in size and pressure drop, and constant budgetary con- level (Å)a <200 <1,000 <2,000 ≥2,000
straints require new types of chemical filtration products. a
Measured in angstroms after 1 month’s exposure. Source: ISA.
This document will discuss application of gas‐phase air
and particulate filtration for the data center environment,
with primary emphasis on the former. General aspects of air ceptible to corrosion failures and are expected to be
filtration technology will be presented with descriptions of much more susceptible than traditional hot air solder
chemical filter media, filters, and air cleaning systems and leveling (HASL) coatings. The use of these two coat-
where these may be employed within the data center envi- ings may make the PCB the weak link regarding the
ronment to provide for enhanced air cleaning. sensitivities of the electronic devices to corrosion.
2. None of the coatings can be considered immune from
failure in an ISA Class G3 environment.
15.2 DATA CENTER ENVIRONMENTAL 3. The gold and silver coatings could not be expected to
ASSESSMENT survive a mid to high Class G2 environment based on
these test results.
A simple quantitative method to determine the airborne
corrosivity in a data center environment is by “reactive According to a leading world authority on RoHS, ERA
monitoring” as first described in ISA Standard 71.04‐1985 Technology, “Recent research has shown that PCBs made
Environmental Conditions for Process Measurement and using lead‐free materials can be more susceptible to corro-
Control Systems: Airborne Contaminants. Copper coupons sion than their tin/lead counterparts”. Experts are working
are exposed to the environment for a period of time and diligently to address these concerns, but they cannot be
quantitatively analyzed using electrolytic (cathodic, coulo- addressed overnight.
metric) reduction to determine corrosion film thickness and The Reliability and Failure Analysis group at ERA
chemistry. Silver coupons should be included with copper Technology has diagnosed failures in electronic devices due
coupons to gain a complete accounting of the types and to interaction with low levels of gaseous sulfides—failures
nature of the corrosive chemical species in the environment. that caused both a financial impact to the manufacturers and
For example, sulfur dioxide alone will corrode only silver to safety issues with their customers. Recent work showed that
form Ag2S (silver sulfide), whereas sulfur dioxide and corrosion could occur even with measured hydrogen sulfide
hydrogen sulfide in combination will corrode both copper levels as low as 0.2 μg/m3 (0.14 ppb). Another reference
and silver forming their respective sulfides. describes the formation of a 200‐Å thick layer of silver
sulfide in 100 hours at a concentration of just 100 μg/m3
[72 ppb].
15.2.1 ISA1 Standard 71.04‐2013
ANSI/ISA‐71.04‐2013 classifies several levels of environ-
15.2.2 Corrosive Gases
mental severity for electrical and electronic systems: G1,
G2, G3, and GX, providing a measure of the corrosion There are three types of gases that can be considered as
potential of an environment. G1 is benign and GX is open‐ prime candidates in the corrosion of data center electronics:
ended and the most severe (Table 15.1). acidic gases such as hydrogen sulfide, sulfur and nitrogen
In a study performed by Rockwell Automation looking at oxides, chlorine, and hydrogen fluoride; caustic gases, such
lead‐free finishes, four alternate PCB finishes were sub- as ammonia; and oxidizing gases, such as ozone. Of these,
jected to an accelerated mixed flowing gas corrosion test. the acidic gases are of particular concern. For instance, it
Important findings can be summarized as follows: takes only 10 ppb (28.98 μg/m3) of chlorine to inflict the
same amount of damage as 25,000 ppb (17.40 mg/m3) of
1. The electroless nickel immersion gold (ENIG) and ammonia.
immersion silver (ImmAg) surface finishes failed Each site may have different combinations and concen-
early in the testing. These coatings are the most sus- tration levels of corrosive gaseous contaminants. Performance
degradation can occur rapidly or over many years, depend-
1
International Society of Automation (www.isa.org). ing on the specific conditions at a site. Descriptions of
15.3 GUIDELINES AND LIMITS FOR GASEOUS CONTAMINANTS 241
p ollutants common to urban and suburban locations in which 15.2.2.5 Photochemical Species
most data centers are located and a discussion of their contri-
The atmosphere contains a wide variety of unstable, reac-
butions to IT equipment performance degradation follow.
tive species that are formed by the reaction of sunlight
with moisture and other atmospheric constituents. Some
15.2.2.1 Sulfur Oxides have lifetimes measured in fractions of a second as they
participate in rapid chain reactions. In addition to ozone
Oxidized forms of sulfur (SO2, SO3) are generated as
(O3), a list of examples would include the hydroxyl radical
combustion products of fossil fuels and from motor vehicle
as well as radicals of hydrocarbons, oxygenated hydrocar-
emissions. Low parts per billion levels of sulfur oxides can
bons, nitrogen oxides, sulfur oxides, and water. Ozone can
cause reactive metals to be less reactive and thus retard cor-
function as a catalyst in sulfide and chloride corrosion
rosion. At higher levels, however, they will attack certain
of metals.
types of metals. The reaction with metals normally occurs
when these gases dissolve in water to form sulfurous and
sulfuric acid acids (H2SO3 and H2SO4). 15.2.2.6 Strong Oxidants
This includes ozone plus certain chlorinated gases (chlorine,
15.2.2.2 Nitrogen Oxides (NOX) chlorine dioxide). Ozone is an unstable form of oxygen that
Some common sources of reactive gas compounds (NO, is formed from diatomic oxygen by electrical discharge or
NO2, N2O4) are formed as combustion products of fossil by solar radiation in the atmosphere. These gases are
fuels and have a critical role in the formation of ozone in the powerful oxidizing agents. Photochemical oxidation—the
atmosphere. They are also believed to have a catalytic effect combined effect of oxidants and ultraviolet light (sunlight)—
on the corrosion of base metals caused by chlorides and is particularly potent.
sulfides. In the presence of moisture, some of these gases Sulfur dioxide (SO2), hydrogen sulfide (H2S), and sand
form nitric acid (HNO3) that, in turn, attacks most common active chlorine compounds (Cl2, HCl, ClO2) have all been
metals. shown to cause significant corrosion in electrical and elec-
tronic equipment at concentrations of just a few parts per
billion in air. Even at levels that are not noticed by or harm-
15.2.2.3 Active Sulfur Compounds ful to humans, these gases can be deadly to electronic
Active sulfur compound refers to hydrogen sulfide (H2S), equipment.
elemental sulfur (S), and organic sulfur compounds such as
mercaptans (R‐SH). When present at low ppb levels, they
rapidly attack copper, silver, aluminum, and iron alloys. The 15.3 GUIDELINES AND LIMITS FOR GASEOUS
presence of moisture and small amounts of inorganic CONTAMINANTS
chlorine compounds and/or nitrogen oxides greatly
accelerate sulfide corrosion. Note, however, that attack still Established gaseous composition environmental limits,
occurs in low relative humidity environments. Active sulfurs listed in Table 15.2, have been published in standards such
rank with inorganic chlorides as the predominant cause of as ISA 71.04, IEC 60721‐3‐3, Telcordia GR‐63‐CORE, and
electronic equipment corrosion. IT equipment manufacturers’ own internal standards. These
limits serve as requirements and guides for specifying data
center environmental cleanliness, but they are not useful for
15.2.2.4 Inorganic Chlorine Compounds
surveying the corrosivity or predicting the failure rates of
This group includes chlorine (Cl2), chlorine dioxide (ClO2), hardware in the data center environment for two reasons.
hydrogen chloride (HCl), etc., and reactivity will depend First, gaseous composition determination is not an easy
upon the specific gas composition. In the presence of task. Second, predicting the rate of corrosion from gaseous
moisture, these gases generate chloride ions that, in turn, contamination composition is not a straightforward
attack most copper, tin, silver, and iron alloys. These exercise.
reactions are significant even when the gases are present at An additional complication to determine corrosivity is
low ppb levels. At higher concentrations, many materials are the synergy between gases. For example, it has been demon-
oxidized by exposure to chlorinated gases. Particular care strated that hydrogen sulfide alone is relatively not corrosive
must be given to equipment that is exposed to atmospheres to silver when compared to the combination of hydrogen
which contain chlorinated contaminants. Sources of chloride sulfide and nitrous oxide, which is very corrosive to silver.
ions, such as seawater, cooling tower vapors, and cleaning Correspondingly, neither sulfur dioxide nor nitrous oxide
compounds, etc., should be considered when classifying alone is corrosive to copper, but together they attack copper
data center environments. at a very fast rate.
242 COrrosion And Contamination Control For Mission Critical Facilities
140 μg/m3
Nitrogen oxides (NOX) — 700 ppb 50 ppb
(5.05782 × 10–12 lb/in3)
Although Table 15.2 can be used to provide some indica- failure rates, but effective control of environmental pollutants
tion of the possible harmful effects of several common con- requires the use of an air cleaning strategy optimized for
taminants, the data center environment needs a single set of both particulate and chemical removal.
limits, which will require considerable study and research.
As the industry works toward a single set of limits, caveats
15.4.1 Particulate Filtration
or exceptions to generally accepted limits will exist. These
exceptions will improve as the interactions of concentration, The control of particulates can be a considered a “mature”
composition, and the thermal environment combine and air cleaning application based on the number of technologies
become better understood along with their effects on the in everyday use and the relative ease of applying these
datacom equipment. technologies for a specific application. ASHRAE Technical
Committee 9.9 has published recommended particulate
filtration requirements for data centers.
15.4 AIR CLEANING TECHNOLOGIES
15.4.2 Gas‐Phase Air Filtration
Increasingly, enhanced air cleaning is being used in data
centers to provide and maintain acceptable air quality with Just as there are many options available for the control of
many options available for the control of particulate particulate pollutants, there are nearly as many options for
pollutants, and nearly as many options for the control of the control of gaseous pollutants. The problem is that for
gaseous pollutants. Employing the proper level and type(s) most data center designers this type of air cleaning is not as
of air filtration can effectively reduce airborne contaminants well understood and is not as easily applied. Also, most
to well below specified levels and minimize equipment ventilation systems and computer room air conditioners/
15.5 CONTAMINATION CONTROL FOR DATA CENTERS 243
The rate of room air recirculation will be determined by the 15.5.1.4 Proper Sealing of Protected Space
type of equipment used and the construction parameters of
Without a tightly sealed room, it will be very difficult to
the data center. Typical recommendations call for 6–12 air
control the four points mentioned above. It is essential that
changes per hour (approximately 10–20% of the gross room
the critical space(s) be protected by proper sealing. Actions
volume per minute).
taken to accomplish this include the use of airlock entries/
exits, sealing around doors and windows, door jambs
15.5.1.3 Temperature and Humidity Control should fit tightly or door sweeps should be used, closing
and sealing all holes, cracks, wall and ceiling joints and
The corrosive potential of any environment increases dra-
cable, pipe, and utility penetrations with a fireproof vapor‐
matically with increasing relative humidity. Rapid changes
retarding material. Care should be taken to assure that any
in relative humidity can result in localized areas of conden-
space above a drop ceiling or below a raised floor is sealed
sation and, ultimately, in corrosive failure.
properly.
ASHRAE Technical Committee 9.9 published Thermal
Guidelines for Data Processing Environments which
extended the temperature–humidity envelope to provide 15.5.2 Advanced Design Requirements
greater flexibility in data center facility operations, particu-
15.5.2.1 Particulate Control
larly with the goal of reducing energy consumption.
For high reliability, TC 9.9 recommends that data centers Filtration is an effective means of addressing airborne par-
be operated in the ranges shown in Table 15.3. These guide- ticulate in the data center environment. It is important that all
lines have been agreed to by all major IT manufacturers and air handlers serving the data center have the appropriate par-
are for legacy IT equipment. A downside of expanding the ticulate filters to ensure appropriate conditions are main-
temperature–humidity envelope is the reliability risk from tained within the room, in this case to meet the cleanliness
higher levels of gaseous and particulate contamination enter- level of ISO Class 8. The necessary efficiency is dependent
ing the data center. Lack of humidity control is creating a on the design and application of the air handlers.
controversy. Unfortunately, decisions are being based more In‐room process cooling with recirculation is the recom-
on financial concerns than engineering considerations. mended method of controlling the data center environment.
Recommended (Applies to all A classes; individual data centers can choose to expand this range based upon the analysis described in this
document)
Allowable
Note: Please visit original document for superscript symbols for footnotes. Source: ASHRAE 2016 Thermal Guidelines
15.5 CONTAMINATION CONTROL FOR DATA CENTERS 245
FIGURE 15.5 Side access system installed at a bank data center. FIGURE 15.6 Positive pressurization unit. Source: Courtesy of
Source: Courtesy of Purafil, Inc. Purafil, Inc.
15.5 CONTAMINATION CONTROL FOR DATA CENTERS 247
SAS,
Key: Corrosion Classification Coupon – CCC Outside air DBS
Environmental Reactivity Monitor – ERM
Cold air
Extruded Carbon Composite Filter – ECC
Adsorbent-Loaded Nonwoven Fiber Filter – ALNF Hot air ECC, ALNF
Side Access System – SAS
Recirculating Air Unit – RAU CCC Makeup
Positive Pressurization Unit – PPU
Deep-Bed Scrubber – DBS
air unit
ALNF, ECC
ERM, CCC CCC
CRAC
unit
RAU,
PPU
FIGURE 15.10 Data center schematic with possible locations of enhanced air cleaning and corrosion monitoring systems. Source: Courtesy
of Purafil, Inc.
15.5.2.3 Air-Side Economizers in data centers, and the increasing interest in the use of air‐
side economizers for “free cooling,” data centers located in
An economizer is a mechanical device used to reduce energy
regions with poor ambient air quality will struggle to main-
consumption. Economizers recycle energy produced within
tain an environment conducive for the protection of sensitive
a system or leverage environmental temperature differences
electronic equipment. ECC filters and/or ALNF filters can
to achieve efficiency improvements. The primary concern in
be easily applied in these systems to address this serious
this approach to data center cooling is that outside air con-
contamination issue.
taminants—both particulate and gas‐phase—will have a
negative impact on electronics.
Research performed by Lawrence Berkley National 15.6 TESTING FOR FILTRATION
Laboratory stated “The pollutant of primary concern, EFFECTIVENESS AND FILTER LIFE
when introducing particulate matter to the data center
environment, is fine particulate matter that could cause Once enhanced air cleaning has been specified and installed,
conductor bridging,” The study also concluded that one must be able to determine the effectiveness of the par-
“. . .filtration systems in most data centers do just fine in ticulate and gas‐phase air filters. One must also be able to
keeping contaminants out.” However, this was referenc- replace the filters on a timely basis so as not to compromise
ing particulate filtration and not the use of gas‐phase air the data center environment.
filtration to address the potential damage to electronic
equipment from the introduction of unwanted gaseous
15.6.1 Particulate Contamination Monitoring
contaminants.
Air‐side economizers typically include filters with a min- Filtration effectiveness can be measured using real‐time
imum ASHRAE‐rated particulate removal efficiency of 40% particle counters in the data center. Excess particle counts
(MERV 9) to reduce the amount of particulate matter or con- or concentrations can indicate filter failure, filter bypass,
taminants that are brought into the data center space. and/or internal sources of particle generation, e.g., CRAC
However, in areas with high ambient particulate levels, belt dust, tin whiskers. Particle monitoring in a data center
ASHRAE 60–90% (MERV 11–13) filter may be required. is generally needed daily; it is usually done only when there
Some references have been found describing the use of is a notable problem that could be caused by particulate
gas‐phase air filter in economizers, especially since the contamination.
advent of RoHS and other “lead‐free” regulation. However, Particulate filters have specified initial and final pres-
with the increasing pressure to reduce energy consumption sure drops at rated air flows and differential pressure
15.7 DESIGN/APPLICATION OF DATA CENTER AIR CLEANING 249
gauges can be used to observe filter life and set alarm lim-
its. Timely replacement of prefilters, primary and final fil-
ters not only protects the electronic equipment but also
maintains optimum performance of the air handling
equipment.
other types of precision air conditioning designed to protect offices, regardless of size, utilize central systems. With these
the datacom equipment, not people. ALNF filters that are systems there are opportunities for the use of ECC filters,
available in various sizes or 2 in or 4 in ECC filters may be ALNF filters, or bulk media modules (Fig. 15.15) for pri-
added to these systems to provide continuous cleaning of the mary control of chemical contaminants. Often the size and
air (Fig. 15.13). Stand‐alone recirculation systems can be location of these air handling units will dictate which type of
used to provide local filtration (Fig. 15.14) if contamination chemical filter can be applied.
cannot be adequately controlled otherwise. Where the data center is not maintained under a posi-
tive pressure, installation of ALNF filters or ECC filters in
the CRAC units may be required to further reduce con-
15.7.1.2 Outside Air: No Pressurization
tamination (corrosion) levels to within manufacturers’
Outside air is introduced to the data center space for one of guidelines or specifications. Consideration should be
the following four reasons: to meet and maintain indoor air given to providing additional outdoor (ventilation) air to
quality requirements, to pressurize the space to keep con- prevent infiltration of contaminants either through the cen-
taminants out, as makeup air for smoke purge, or to conserve tral HVAC system or using positive pressurization units
energy when outside air conditions are conducive to free (PPUs). To provide clean pressurization air in locations
cooling. with high levels of outdoor contaminants, deep‐bed bulk
Some larger datacom facilities use central station air han- media air filtration systems (DBSs) may be required
dling units. Specifically, many telecommunications central (Fig. 15.16).
FIGURE 15.15 SAS installed on outdoor air intake. Source: Courtesy of Purafil, Inc.
15.7 DESIGN/APPLICATION OF DATA CENTER AIR CLEANING 251
severe (ISA Class G3, GX) prior to any corrective actions. the IT/datacom equipment manufacturers’ site planning
Providing a benign environment for electronic equipment guides or environmental specifications. The length of
(ISA Class G1) will not reverse any corrosive damage that time that these levels of cleanliness can be provided is a
may have already occurred but will serve to extend the life of function of the total contaminant load, air cleaning sys-
those devices that would have failed without these actions tem design, and filters/equipment employed. Factors that
being taken. may cause the data center environment to exceed these
classifications include critical space(s) not being properly
sealed, high pedestrian traffic, high levels of internally
15.8 SUMMARY AND CONCLUSION generated contaminants, the air conditioning system and/
or negative pressure ductwork being located outside the
Electronic equipment used in data centers are protected protected space, and air filtration system being under-
against the potential threats posed by fire, power, shock, sized. In properly sealing a room, often the spaces beneath
humidity, temperature, and (to a degree) particulate contami- raised floors and/or the space above dropped ceilings are
nation. Unfortunately, the potential damage to this equip- neglected. These areas are critical, especially when they
ment caused by the corrosive effects of gaseous contaminants are used as supply and return plenums. Long forgotten
has still not been fully recognized or addressed. cable penetrations, floor drains, cracks in the walls and
Recognizing the severity of the problem, the world’s ceiling, etc., can cause infiltration of contaminated air.
leading manufacturers of computer systems jointly pub- The solution is to seal all penetrations, supply an adequate
lished a white paper titled “2011 Gaseous and Particulate amount and distribution of supply and recirculation air,
Contamination Guideline for Data Centers”. That, along provide clean outdoor air to achieve positive pressure,
with the ASHRAE Handbook on “Particulate and Gaseous and balance the system.
Contamination in Datacom Environments” summarizes the With the increasing pressure to reduce energy consump-
acceptable levels of data center contamination as shown in tion in data centers, and the increasing use of air‐side econo-
Table 15.4. mizers, data centers located in regions with poor ambient air
Enhanced air filtration systems for data centers should quality will struggle to maintain efficient operations without
be designed to the ASHRAE guidelines as described or to the application of enhanced air cleaning. This means increas-
ing particulate filter efficiencies to at least ASHRAE 85%
TABLE 15.4 Particulate and gaseous contamination and the addition of gas‐phase air filtration designed to con-
guidelines for data centers trol specific contaminants of concern.
Data centers must be kept clean to ISO 14644‐1 Class 8. This
The issue and potential problem of corrosion in data cent-
level of cleanliness can generally be achieved by an appropriate ers has been presented (ASHRAE 2009a;). The problem needs
filtration scheme as outlined here: to be addressed by monitoring of the environment and removal
of contaminants where needed. Ultimately, the successful
1. The room air may be continuously filtered with MERV 8 implementation of a contamination control program requires:
filters as recommended by ANSI/ASHRAE Standard
127‐2012, Method of Testing for Rating Computer and Data
Processing Room Unitary Air Conditioners (ASHRAE 1. Knowledge and understanding that corrosion of elec-
2012). tronic equipment is a serious problem.
2. Air entering a data center may be filtered with MERV 11 or 2. Commitment to a monitoring program to detect the
MERV 13 filters as recommended by ASHRAE handbook potential for IT equipment failure before this
titled “Particulate and Gaseous Contamination in Datacom equipment is damaged and costly shutdowns occur.
Environments”. 3. Commitment to an integrated contamination control
Sources of dust inside data centers should be reduced. Every system.
effort should be made to filter out dust that has deliquescent 4. Commitment to take corrective action whenever
relative humidity greater than the maximum allowable relative necessary.
humidity in the data center.
The gaseous contamination should be within the ANSI/
ISA‐71.04‐2013 severity level G1‐Mild that meets: 15.9 APPENDIX 1: ADDITIONAL DATA CENTER
SERVICES
1. A copper reactivity rate of less than 300 Å/month, and
2. A silver reactivity rate of less than 200 Å/month.
Many companies can, upon the request of the data center
For data centers with higher gaseous contamination levels, owner/operator, provide any or all the following services.
gas‐phase filtration of the inlet air and the air in the data center These services should be considered if air quality/equipment
is highly recommended.
reliability problems are discovered and/or are ongoing in the
Source: Christpher O. Muller. data center environment.
15.10 APPENDIX 2: DATA CENTER HISTORY 253
15.10.1 Requirements for Modern Data Centers 15.10.2 Data Center Classification
IT operations are a crucial aspect of most organizational The TIA‐942: Data Center Standards Overview4 describes
operations. One of the main concerns is business continuity; the requirements for the data center infrastructure. The
companies rely on their information systems to run their simplest is a Rated 1 data center, which is basically a server
operations. If a system becomes unavailable, company oper- room,5 following basic guidelines for the installation of
ations may be impaired or stopped completely. It is neces- computer systems. The most stringent level is a Rated 4 data
sary to provide a reliable infrastructure for IT operations, in center, which is designed to host mission critical computer
order to minimize any chance of disruption. Information systems, with fully redundant subsystems and compartmen-
security is also a concern, and for this reason a data center talized security zones controlled by biometric access con-
has to offer a secure environment which minimizes the trols methods. Another consideration is the placement of the
chances of a security breach. A data center must therefore data center in a subterranean context, for data security as
keep high standards for assuring the integrity and functional- well as environmental considerations such as cooling
ity of its hosted computer environment. This is accomplished requirements. The rated levels describe the availability of
through redundancy of both fiber optic cables and power, data from the hardware at a location—the higher the rating
which includes emergency backup power generation. the greater the accessibility. The levels are:
Telcordia GR‐3160, NEBS Requirements for
Telecommunications Data Center Equipment and Spaces,3
provides guidelines for data center spaces within telecom- Rated
munications networks, and environmental requirements for level Requirements
the equipment intended for installation in those spaces. 1 • Single nonredundant distribution path serving the IT
These criteria were developed jointly by Telcordia and equipment
industry representatives. They may be applied to data center • Nonredundant capacity components
spaces housing data processing or IT. The equipment may be • Basic site infrastructure guaranteeing 99.671%
used to: availability
2 • Fulfills all Rated 1 requirements
• Operate and manage a carrier’s telecommunication • Redundant site infrastructure capacity components
network guaranteeing 99.741% availability
• Provide data center‐based applications directly to the 3 • Fulfills all Rated 1 and Tier 2 requirements
carrier’s customers • Multiple independent distribution paths serving the
• Provide hosted applications for a third party to provide IT equipment
services to their customers • All IT equipment must be dual‐powered and fully
compatible with the topology of a site’s architecture
• Provide a combination of these and similar data center
• Concurrently maintainable site infrastructure
applications. guaranteeing 99.982% availability
Effective data center operation requires a balanced invest- 4 • Fulfills all Rated 1, 2, and 3 requirements
• All cooling equipment is independently dual‐
ment in both the facility and the housed equipment. The first
powered, including chillers and Heating, Ventilating
step is to establish a baseline facility environment suitable and Air Conditioning (HVAC) systems
for equipment installation. Standardization and modularity • Fault‐tolerant site infrastructure with electrical
can yield savings and efficiencies in the design and construc- power storage and distribution facilities
tion of telecommunications data centers. guaranteeing 99.995% availability
Standardization means integrated building and equipment
engineering. Modularity has the benefits of scalability and
easier growth, even when planning forecasts are less than
15.10.3 Physical Layout
optimal. For these reasons, telecommunications data centers
should be planned in repetitive building blocks of equip- A data center can occupy one room of a building, one or
ment, and associated power and support (conditioning) more floors, or an entire building. Most of the equipment is
equipment when practical. The use of dedicated centralized often in the form of servers mounted in 19‐in rack cabinets,
systems requires more accurate forecasts of future needs to which are usually placed in single rows forming corridors
prevent expensive over construction, or perhaps worse— between them. This allows people access to the front and
under construction that fails to meet future needs.
4
http://www.adc.com/Attachment/1270711929361/102264AE.pdf.
5
A server room is a room that houses mainly computer servers. In IT
3
http://telecom‐info.telcordia.com/site‐cgi/ido/docs.cgi?ID=SEARCH&D circles, the term is generally used for smaller arrangements of servers;
OCUMENT=GR‐3160&. larger groups of servers are housed in data centers.
15.10 APPENDIX 2: DATA CENTER HISTORY 255
6
The size of a piece of rack‐mounted equipment is frequently described as
Are both
a number in “U.” For example, one rack unit is often referred to as “1U”, Cu and Ag
two rack units as “2U” and so on. Yes corrosion rates
7
http://tc99.ashraetcs.org/documents/ASHRAE%20Whitepaper%20‐%20 at ISA-71.04
Class G1?
2011%20Thermal%20Guidelines%20for%20Data%20Processing%20
Environments.pdf.
8
https://www.techstreet.com/ashrae/standards/thermal‐guidelines‐for‐data‐
processing‐environments‐4th‐ed?product_id=1909403. FIGURE 15.17 Data center corrosion control process flow.
256 COrrosion And Contamination Control For Mission Critical Facilities
The intent of outlining the process herein is to dem- recommending under raised floor cabling for security
onstrate a methodology and provide general guid- reasons and to consider the addition of cooling sys-
ance. This paper contains generic server equipment tems above the racks in case this enhancement is nec-
metrics and does not necessarily represent the char- essary. Smaller/less expensive data centers without
acteristics of any particular piece of IT equipment. raised flooring may use antistatic tiles for a flooring
For specific equipment information, contact the IT surface. Computer cabinets are often organized into a
manufacturer. hot aisle arrangement to maximize airflow
The other major change in the environmental specifica- efficiency.
tion is in the data center classes. Previously there were
two classes applying to IT equipment used in data 15.10.4 Applications
center applications: Classes 1 and 2. The new environ-
mental guidelines have more data center classes to The main purpose of a data center is running the applications
accommodate different applications and priorities of IT that handle the core business and operational data of the
equipment operation. This is critical because a single organization. Such systems may be proprietary and devel-
data center class forces a single optimization whereas oped internally by the organization or bought from enter-
each data center needs to be optimized based on the prise software vendors. Such common applications are ERP
operator’s own criteria (e.g., fulltime economizer use and CRM systems.
versus maximum reliability). A data center may be concerned with just operations
• Today’s data centers try to use economizer cooling, architecture or it may provide other services as well.
where outside air is used to keep the data center Often these applications will be composed of multiple
cool. Washington State now has a few data centers hosts, each running a single component. Common compo-
that cool all the servers using outside air 11 months nents of such applications are databases, file servers, appli-
out of the year. They do not use chillers/air condi- cation servers, middleware, and various others.
tioners, which creates potential energy savings in Data centers are also used for offsite backups. Companies
the millions. may subscribe to backup services provided by a data center.
This is often used in conjunction with backup tapes. Backups
• Backup power consists of one or more uninterruptible
can be taken of servers locally on to tapes; however, tapes
power supplies and/or diesel generators.
stored on site pose a security threat and are also susceptible
• To prevent single points of failure, all elements of the to fire and flooding. Larger companies may also send their
electrical systems, including backup system, are typi- backups off site for added security. This can be done by
cally fully duplicated, and critical servers are connected backing up to a data center. Encrypted backups can be sent
to both the “A‐side” and “B‐side” power feeds. This over the Internet to another data center where they can be
arrangement is often made to achieve N + 1 Redundancy stored securely.
in the systems. Static switches are sometimes used to For disaster recovery, several large hardware vendors
ensure instantaneous switchover from one supply to the have developed mobile solutions that can be installed and
other in the event of a power failure. made operational in very short time. Vendors such as Cisco
• Data centers typically have raised flooring made up of Systems, Sun Microsystems, IBM, and HP have developed
2 × 2 ft (60 × 60 cm) removable square tiles. The trend is systems that could be used for this purpose.
toward 31–39 in2 (80–100 cm2) void to cater for better
and uniform air distribution. These provide a plenum
for air to circulate below the floor, as part of the air
15.11 APPENDIX 3: REACTIVITY MONITORING
conditioning system, as well as providing space for
DATA EXAMPLES: SAMPLE CORROSION
power cabling.
MONITORING REPORT
Telcordia GR‐2930, NEBS: Raised Floor Generic
Requirements for Network and Data Centers,9 pre- World Data Center, Inc.
sents generic engineering requirements for raised Corrosion Monitoring Report
floors that fall within the strict NEBS guidelines. Report for data collected Date 1 – Date 2
Data cabling is typically routed through overhead
cable trays in modern data centers. But some are still
15.11.1 Executive Summary
9
https://telecom‐info.telcordia.com/site‐cgi/ido/docs.cgi?ID=SEARCH& Seven copper/silver CCCs were placed at World Data Center,
DOCUMENT=GR‐2930&. Inc. (Anytown, USA) to provide an assessment of the air
15.11 APPENDIX 3: REACTIVITY MONITORING DATA EXAMPLES: SAMPLE CORROSION MONITORING REPORT 257
outside and inside the data center. All CCCs were analyzed In the context of electronic equipment, corrosion is
via electrolytic reduction to identify and quantify corrosive defined as the deterioration of a base metal resulting from a
contaminants to which the coupons had been exposed. The reaction with its environment. More specifically, corrosive
electrolytic reduction analysis shows the presence of high gases and water vapor coming into contact with a base metal
contaminant concentrations. result in the buildup of various chemical reaction products.
Analysis results indicate that the air outside the data As the chemical reactions continue, these corrosion prod-
center would be classified as GX – SEVERE according to ucts can form insulating layers on circuits which can lead to
ASHRAE TC 9.9 ANSI / ISA Standard S71.04 which sig- thermal failure or short‐circuits. Pitting and metal loss can
nificantly exceeds the recommended severity level of also occur.
G1 – MILD. The air inside the data center would be clas-
sified, as G3 – HARSH which, on average, also exceeds
15.11.2.1 Corrosive Gases
the recommended severity level for these environments.
The presence of oxidized forms of sulfur and active sulfur Three types of gases are the prime suspects in the corrosion
compounds was detected, and it is estimated that the con- of electronics: acidic gases, such as hydrogen sulfide, sulfur
centrations in air would be 10–100 parts per billion (ppb) and nitrogen oxides, chlorine, and hydrogen fluoride; caus-
and >10 ppb, respectively, for these two sulfur species. tic gases, such as ammonia; and oxidizing gases, such as
It is also suspected that there are significant levels of ozone. Of the gases that can cause corrosion, the acidic gases
nitrogen oxides and ozone present in the ambient are typically the most harmful.
(outdoor) environment. Each site may have different combinations and concen-
Oxidized forms of sulfur include sulfur dioxide (SO2) tration levels of corrosive gaseous contaminants and perfor-
and sulfur trioxide (SO3). Active sulfur compounds include mance degradation can occur rapidly or over many years,
elemental sulfur, hydrogen sulfide (H2S), organic sulfur depending on the concentration levels and combinations
compounds (e.g., mercaptans), as well as sulfuric and sul- present at a site. The following paragraphs describe how
furous acids. These contaminants will cause corrosion‐ various pollutants contribute to equipment performance
related problems should adequate control measures not be degradation.
put in place. Active sulfur compounds (H2S). This group includes
The active sulfur contamination (as Cu2S) detected on all hydrogen sulfide (H2S), elemental sulfur (S), and organic
CCCs placed both outside and inside the data center indi- sulfur compounds such as the mercaptans (RSH). When pre-
cates a SEVERE risk potential if steps are not taken to reduce sent at low parts per billion levels, they rapidly attack cop-
and maintain lower levels of these contaminants in the data per, silver, aluminum, and iron alloys. The presence of
center. moisture and small amounts of inorganic chlorine com-
It is recommended that reactivity monitoring be pounds and/or nitrogen oxides greatly accelerate sulfide cor-
continued—either with CCCs or a real‐time environmen- rosion. Note, however, that attack still occurs in low relative
tal reliability monitor (ERM)—to provide a continuous humidity environments. Active sulfurs rank as one of the
environmental assessment of the data center air quality predominant causes of atmospheric corrosion in datacom
and to assure that chemical contamination is being main- equipment.
tained at acceptable levels. Sulfur oxides. Oxidized forms of sulfur (SO2, SO3) are
generated as combustion products of sulfur‐bearing fossil
fuels. Low parts per billion levels of sulfur oxides can pas-
15.11.2 Background
sivate reactive metals and thus retard corrosion. At higher
Corrosion of metals is a chemical reaction caused primarily levels, however, they will attack certain types of metals. The
by attack of gaseous contaminants and is accelerated by heat reaction with metals normally occurs when these gases dis-
and moisture. Rapid shifts in either temperature or humidity solve in water to form sulfurous and sulfuric acid.
cause small portions of circuits to fall below the dew point Nitrogen oxides (NOX). NOX compounds (NO, NO2,
temperature, thereby facilitating condensation of contami- N2O4) are formed as combustion products of fossil fuels and
nants. Relative humidity above 50% accelerates corrosion have a critical role in the formation of ozone in the atmos-
by forming conductive solutions on a small scale on elec- phere. They are also believed to have a catalytic effect on
tronic components. Microscopic pools of condensation then corrosion of base metals by chlorides and sulfides. In the
absorb contaminant gases to become electrolytes where presence of moisture, some of these gases form nitric acid
crystal growth and electroplating occur. Above 80% RH, that, in turn, attacks most common metals.
electronic corrosive damage will occur regardless of the Inorganic chlorine compounds. This group includes
levels of contamination. chlorine (Cl2), chlorine dioxide (ClO2), hydrogen chloride
258 COrrosion And Contamination Control For Mission Critical Facilities
(HCl), etc., and reactivity will depend upon the specific gas these gases can be deadly to electronic equipment. Most of
composition. In the presence of moisture, these gases gener- the odor threshold levels are much higher than the levels at
ate chloride ions that react readily with copper, tin, silver, which corrosive damage will occur.
and iron alloys. These reactions are significant even when
the gases are present at low parts per billion levels. At higher
15.11.2.2 Environmental Classifications
concentrations, many materials are oxidized by exposure to
chlorinated gases. Particular care must be given to equip- Table 15.5 lists a standard classification scheme that
ment that is exposed to atmospheres which contain chlorin- directly correlates corrosion rates to environmental clas-
ated contaminants. Sources of chloride ions, such as sifications. These classifications are being refined based
bleaching operations, sea water, cooling tower vapors, and on the results of testing and the specific needs of this mar-
cleaning compounds, etc., should be considered when clas- ket. Typical uses of reactivity monitoring in data centers
sifying data center environments. have been for the characterization of outdoor air used for
Hydrogen fluoride (HF). This compound is a member of ventilation and pressurization, the identification of “hot
the halogen family and reacts like inorganic chloride spots” within a facility, and the effectiveness of various
compounds. preventive measures. Reactivity monitoring is being used
Ammonia and derivatives. Reduced forms of nitrogen for the purpose of developing the cause‐and‐effect rela-
(ammonia (NH3), amines, ammonium ions (NH4+)) occur tionship between gaseous pollutants and the damage it
mainly in fertilizer plants, agricultural applications, and may cause within data centers to sensitive electronic
chemical plants. Copper and copper alloys are particularly equipment.
susceptible to corrosion in ammonia environments. Generally speaking, the silver and copper corrosion
Photochemical species. The atmosphere contains a rates should be class G1 or better unless otherwise agreed
wide variety of unstable, reactive species that are formed upon. The individual corrosion films quantified using
by the reaction of sunlight with moisture and other atmos- reactivity monitoring may be used to further characterize
pheric constituents. Some have lifetimes measured in frac- the environment and to determine the proper control strat-
tions of a second as they participate in rapid chain reactions. egies. Based upon these recommended control levels and
In addition to ozone (O3), a list of examples would include test results from laboratory and field‐exposed silver cou-
the hydroxyl radical as well as radicals of hydrocarbons, pons, acceptance criteria relevant to these applications
oxygenated hydrocarbons, nitrogen oxides, sulfur oxides, have been determined.
and water. Because of the transient nature of most of these These criteria consider total corrosion as well as the
species, their primary effect is on outdoor installations and relative contribution of each individual corrosion film.
enclosures.
Strong oxidants. This includes ozone plus certain chlo-
rinated gases (chlorine, chlorine dioxide). These gases are TABLE 15.5 Classification of reactive environments
powerful oxidizing agents. Ozone can function as a catalyst G1 G2 G3 GX
in sulfide and chloride corrosion of metals. Photochemical Severity level Mild Moderate Harsh Severe
oxidation—the combined effect of oxidants and ultraviolet
Copper reactivity level <300 <1,000 <2,000 ≥2,000
light (sunlight)—is particularly potent.
(in angstroms)a
Hydrogen sulfide (H2S), sulfur dioxide (SO2), and active
chlorine compounds (Cl2, HCl, ClO2) have all been shown to Silver reactivity level <200 <1,000 <2,000 ≥2,000
(in angstroms)a
cause significant corrosion in electrical and electronic equip-
ment at concentrations of just a few parts per billion in air. a
Normalized to a 30‐day exposure. See ANSI/ISA‐71.04‐2013 Annex C,
Even at levels that are not noticed by or harmful to humans, item numbers C.2, C.3.
The control specifications for the individual corrosion When both films are detected, as highlighted above, it
films are listed in Table 15.6. These specifications are more often than not indicates the presence of active sulfur
more general in their application than those listed above compounds such as elemental sulfur, hydrogen sulfide (H2S),
and are most often used for the characterization of an envi- organic sulfur compounds (e.g., mercaptans), and sulfuric
ronment prior to the implementation of pollutant control and sulfurous acids as well. When both films are present and
measures. the amount of Cu2S is greater than 50% of the total corro-
If the total corrosion AND each individual corrosion film sion, this is further evidence of the presence of active sulfur
meets the recommended criteria, the local environment in compounds in the subject environment.
which that particular coupon has been exposed meets the Active sulfur compounds include hydrogen sulfide, ele-
requirements of a Class G1 classification. ANY of the crite- mental sulfur, and organic sulfur compounds such as the
ria which are not met indicates that the local environment mercaptans. When present at low parts per billion levels,
may not be sufficiently well‐controlled to minimize the cor- they rapidly attack copper, silver, aluminum, and iron alloys.
rosion of sensitive electronic equipment due to the presence The presence of moisture and small amounts of inorganic
of gaseous pollutants. Steps should be taken to determine chlorine compounds greatly accelerates sulfide corrosion.
what problems exist and what corrective actions may be Note that corrosive attack still occurs in low relative humid-
appropriate. ity environments.
Chloride corrosion (AgCl) which was not detected on
any of the silver coupons would have indicated the pres-
15.11.3 Results and Discussion
ence of (an) inorganic chlorine compound(s), e.g., chlo-
As shown in Table 15.7 the total copper and silver corrosion rine (Cl2), chlorine dioxide (ClO2), and/or hydrogen
rates for all the CCCs placed in the data center at World Data chloride (HCl). High levels of chloride (halogen) contami-
Center, Inc. (Anytown, USA) exceed the generally recom- nation can also serve to effectively mask evidence of sul-
mended severity level of Class G1. fur contamination on the corresponding copper coupons
When interpreting the analysis results for the individual and can cause a large “unknown” copper corrosion film to
corrosion films, the detection of a silver sulfide (Ag2S) film appear.
without a corresponding copper sulfide (Cu2S) film indicates Chlorine contamination, whether as chlorine or hydrogen
the presence of oxidized forms of sulfur such as sulfur diox- chloride, is a most dangerous contaminant for metals. At
ide (SO2) and sulfur trioxide (SO3). All silver coupons elevated levels, many materials are oxidized by exposure to
showed evidence of sulfur oxide contamination. chlorinated gases.
Oxidized forms of sulfur are generated as combustion Based on these results, it is estimated that the active sul-
products of sulfur‐bearing fossil fuels. Low parts per billion fur concentration is >10 ppb and sulfur oxides are in the
levels of sulfur oxides can passivate reactive metals and thus range of 10–100 ppb. Further, the ratio of Cu2S to Ag2S
retard corrosion. At higher levels they attack certain types of indicates a high likelihood that significant levels of nitro-
metals, elastomers, and plastics. The reaction with metals gen oxides are also present in the local environment.
normally occurs when these gases dissolve in water to form Fluctuating humidity may also play a role in the high cop-
sulfurous and sulfuric acid. per corrosion rates.
TABLE 15.7 CCC monitoring results for World Data Center, Inc. (Anytown, USA)
Copper corrosion Silver corrosion
CCC panel # Location Cu2S Cu2O Cu‐unk Total AgCl Ag2S Ag‐unk Total Class ISA class
60127 Outdoor air 4,255 0 0 4,255 0 2,212 0 2,212 GX GX
60126 Entrance to data center 934 112 130 1,176 0 1,590 0 1,590 G3 G3
100
0
an
eb
ay
27 l
ug
ep
ay
l
ul
ar
pr
19 t
ov
ec
an
11 b
ar
pr
Ju
Ju
Ju
Ju
-O
-M
-J
-M
-J
A
A
-D
-J
-F
-S
-J
-F
M
-N
-A
2-
1-
4-
3-
30
29
9-
8-
15
14
22
12
7-
24
11
6-
17
12
Date
Copper corrosion Silver corrosion
FIGURE 15.18 Data center reactivity monitoring results. Source: Courtesy of Purafil, Inc.
Geneva: International Organization for Standardization; Shah JM, Oluwaseun A, Agarwal P, Akhigbe I, Agonafer D, Singh
2015. P, Kannan N, Kaler M. Qualitative study of cumulative
U.S. Department of Defense (USDOD). 1995. Method 102.9.1: corrosion damage of IT equipment in a data center utilizing
DOP‐smoke penetration and air resistance of filters. Military air‐side economizer. Proceedings of ASME 2016
standard, filter units, protective clothing, gas‐mask compo- International Mechanical Engineering Congress and
nents and related products: performance test methods. Exposition; November 11–17, 2016; Phoenix, AZ.
MIL‐STD‐282. England WG, McShane WJ, Muller CO. Developments in
Singh PJ. Gaseous and particulate contamination limits for data measurement and control of corrosive gases to avoid electrical
centers. The Air Conditioning and Refrigeration Journal, equipment failure. Proceedings of PITA Annual Technical
2010. Delhi: Indian Society of Heating, Refrigerating and Conference; September 14–16, 1999; Manchester, England.
Air Conditioning Engineers (April‐June 2010 edition). Muller C. What’s Creeping Around in Your Data Center?
Shehabi A, Tschudi W, Gadgil A. 2007. Data center economizer ASHRAE Transactions. Atlanta: American Society of
contamination and humidity study. LBNL/Pacific Gas & Heating, Refrigerating, and Air‐Conditioning Engineers,
Electric. Available at https://buildings.lbl.gov/sites/default/ Inc.; 2010.
files/2424e.pdf. Accessed September 2020. Gaseous and particulate contamination guidelines for data centers.
ASHRAE. Standard 52.2: Method of Testing General Ventilation Whitepaper prepared by ASHRAE Technical Committee
Air‐Cleaning Devices for Removal Efficiency by Particle (TC) 9.9 Mission Critical Facilities, Technology Spaces, and
Size. Atlanta: American Society of Heating, Refrigerating, Electronic Equipment. https://www.ashrae.org/File%20
and Air‐Conditioning Engineers, Inc; 2017. https://standards. Library/Technical%20Resources/Publication%20Errata%20
globalspec.com/std/10148034/ASHRAE%20STD%2052.2. and%20Updates/2011-Gaseous-and-Particulate-Guidelines.
Accessed 9/22/2020. pdf. Accessed 9/22/2020.
Muller C. There’s no such thing as a free ride: the real costs of ISA Standard ANSI/ISA‐71.04‐2013. Environmental Conditions
free cooling. Proceedings of the Green Data Center for Process Measurement and Control Systems: Airborne
Conference and Exhibition; February 24–26, 2015; San Contaminants. Research Triangle Park, NC: International
Diego, CA. Society for Automation; 2013.