Open Compute Project Open Vault Storage Specification
Open Compute Project Open Vault Storage Specification
Open Compute Project Open Vault Storage Specification
Hardware V0.7
OR-draco-bueana-0.7
Authors:
Mike Yan, Storage Hardware Design Engineer, Facebook
Jon Ehlen, Mechanical Engineer, Facebook
Scope
This document describes the technical specifications used in the design of the storage
unit for the Open Compute Project, known as the Open Vault.
Contents
1
Scope ........................................................................................................................................................ 2
2
Contents .................................................................................................................................................. 2
3
Overview ................................................................................................................................................. 5
3.1
License........................................................................................................................................... 5
4
Open Vault Storage Unit Overview ................................................................................................. 5
4.1
Open Vault Front View ............................................................................................................. 7
4.2
Open Vault Rear View ............................................................................................................... 7
4.3
System Component Layout .................................................................................................... 8
4.4
System Block Diagram .............................................................................................................. 8
4.5
System I2C Topology .............................................................................................................. 10
5
Open Vault SAS Expander Board ................................................................................................... 11
5.1
Block Diagram and Configurations ................................................................................... 11
5.2
SEB Placement and Form Factor ........................................................................................ 11
5.3
SAS Expander IC ....................................................................................................................... 12
5.4
Voltage Monitor ...................................................................................................................... 12
5.5
SAS Signal Re-driver ............................................................................................................... 12
5.6
Connectors ................................................................................................................................ 12
5.7
LEDs ............................................................................................................................................. 15
5.8
PCB Stack-up ............................................................................................................................. 16
6
Open Vault Drive Plane Board ....................................................................................................... 17
6.1
Block Diagram .......................................................................................................................... 17
6.2
DPB Placement and Form Factor ....................................................................................... 17
6.3
12.5V to 5V Buck Converter .................................................................................................. 18
6.4
Disk Drive Power Control ..................................................................................................... 18
6.5
Voltage Monitor ...................................................................................................................... 18
6.6
Connectors and Cables ......................................................................................................... 18
6.7
LEDs ............................................................................................................................................. 20
6.8
PCB Stack-up ............................................................................................................................. 20
7
Open Vault Power Transition Board ............................................................................................ 21
7.1
Functional Block Diagram .................................................................................................... 22
7.2
PTB Placement and Form Factor ........................................................................................ 22
http://opencompute.org
Overview
When data center design and hardware design move in concert, they can improve
efficiency and reduce power consumption. To this end, the Open Compute Project is a
set of technologies that reduces energy consumption and cost, increases reliability and
choice in the marketplace, and simplifies operations and maintenance. One key
objective is opennessthe project is starting with the opening of the specifications and
mechanical designs for the major components of a data center, and the efficiency
results achieved at facilities using Open Compute technologies.
A main component of this project is the Open Vault storage server. The Open Vault is a
simple and cost-effective storage solution with a modular I/O topology that's built for
the Open Rack. The Open Vault offers high disk densities, holding 30 drives in a 2U
chassis, and can operate with almost any host server. Its innovative, expandable design
puts serviceability first, with easy drive replacement no matter the mounting height.
3.1
License
As of April 7, 2011, the following persons or entities have made this Specification
available under the Open Web Foundation Final Specification Agreement (OWFa 1.0),
which is available at
http://www.openwebfoundation.org/legal/the-owf-1-0-agreements/owfa-1-0
Facebook, Inc.
You can review the signed copies of the Open Web Foundation Agreement Version 1.0
for this Specification at http://opencompute.org/licensing/, which may also include
additional parties to those listed above.
Your use of this Specification may be subject to other third party rights. THIS
SPECIFICATION IS PROVIDED "AS IS." The contributors expressly disclaim any warranties
(express, implied, or otherwise), including implied warranties of merchantability, noninfringement, fitness for a particular purpose, or title, related to the Specification. The
entire risk as to implementing or otherwise using the Specification is assumed by the
Specification implementer and user. IN NO EVENT WILL ANY PARTY BE LIABLE TO ANY
OTHER PARTY FOR LOST PROFITS OR ANY FORM OF INDIRECT, SPECIAL, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES OF ANY CHARACTER FROM ANY CAUSES OF ACTION OF ANY
KIND WITH RESPECT TO THIS SPECIFICATION OR ITS GOVERNING AGREEMENT, WHETHER
BASED ON BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), OR OTHERWISE, AND
WHETHER OR NOT THE OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGE.
http://opencompute.org
keeping the whole system running. Each fan module is hot pluggable and field
replaceable from the rear of the chassis. There are bus bar clips to the Open Rack bus
bars so that the 12.5V main power from the Open Rack can feed into the Open Vault
enclosure.
For the purposes of this specification, "front" refers to the cold aisle side of the chassis,
which is where all service (except fan module removal and replacement) occurs; "rear"
refers to the hot aisle side of the chassis, which is where the fan module service occurs;
"SEB" refers to the SAS expander board, "DPB" refers to the drive plane board, "PTB"
refers to the power transition board and "FCB" refers to the fan control board.
Figure 1 and Figure 2 show an overview of the Open Vault storage unit.
Figure 2 Open Vault Detail Showing the Hinged Disk Drive Tray
4.1
4.2
http://opencompute.org
4.3
FAN 5
FAN 4
FAN 3
FAN 2
FAN 1
Fan
Control
Board
Power
Transition
Board
HDD 0
HDD 1
HDD 2
HDD 3
HDD 4
Drive
Plane
Board
HDD 5
HDD 6
HDD 7
HDD 8
HDD 9
HDD 10
HDD 11
HDD 12
HDD 13
HDD 14
SAS
Expander
Board
SEB-A
SEB-B
4.4
http://opencompute.org
4.5
Signal
Re-driver
LED
Driver
Voltage
Monitor
EEPROM
SEB-1B
SEB-2A
EEPROM
Voltage
Monitor
LED
Driver
Signal
Re-driver
Signal
Re-driver
LED
Driver
Voltage
Monitor
EEPROM
SEB-2B
I2C_A
SAS
Expander
I2C_C
I2C
Buffer
I2C_B
I2C
Buffer
I2C_D
Temp
Sensor
1
Temp
Sensor
1
Temp
Sensor
2
Temp
Sensor
2
Temp
Sensor
3
Temp
Sensor
3
I2C_D
SAS
Expander
I2C_A
I2C
Buffer
DPB-1
Temp
Sensor
1
Temp
Sensor
2
EEPROM
I2C
Buffer
I2C_B
I2C
Buffer
I2C_C
I2C
Buffer
Voltage
Monitor
Temp
Sensor
3
Temp
Sensor
4
PTB-1
FCB
I2C
Buffer
Fan
Controller
EEPROM
I2C_A
SAS
Expander
I2C_C
I2C
Buffer
I2C_B
I2C
Buffer
I2C_D
Temp
Sensor
1
Temp
Sensor
1
Temp
Sensor
2
Temp
Sensor
2
Temp
Sensor
3
Temp
Sensor
3
I2C_D
SAS
Expander
I2C_A
I2C
Buffer
DPB-2
Temp
Sensor
1
Temp
Sensor
2
EEPROM
I2C
Buffer
I2C_B
I2C
Buffer
I2C_C
I2C
Buffer
Power
Monitor
Voltage
Monitor
Temp
Sensor
3
Temp
Sensor
4
LED
Driver
PTB-2
I2C
Buffer
10
5.1
SAS
2.0
x4
SAS
2.0
x4
Temp
Sensors
1,
2,
3
SAS 2.0 x4
Voltage
Monitor
I2C_D
Signal
Re-driver
I2C_A
EEPROM
Expander
LSISAS2x28
Ext.
Mini-SAS
Connector
Int.
Mini-SAS
Connector
Int.
Mini-SAS
Connector
Debug
Header
LED Driver
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
I2C_C,
PWM,
GPIOs
5.2
http://opencompute.org
11
SAS
Expander
Enclosure
Debug
Status
LED Header
5.3
SAS Expander IC
The Open Vault SAS expander board contains a single 6G SAS expander IC, part number
LSISAS2x28. The board design is compatible with LSISAS2x24 and LSISAS2x20 under
different configurations.
5.4
Voltage Monitor
A voltage monitor is required for the Open Vault SAS expander board in order to ensure
proper operation of all power rails at all times. The voltages are reported as part of the
enclosure status as described in section 10.1. The power rails to be monitored are shown
in Figure 10.
Power Rail
Voltage
VDDIO33
3.3V
VDDIO
1.8V
VDD
1.0V
1.2V
5.5
5.6
Connectors
Sections 5.6.1 through 5.6.4 describe the connectors that reside on the Open Vault SAS
expander board.
12
5.6.1
5.6.2
Assignment
Pin
Assignment
A1
GND
B1
GND
A2
Rx0P
B2
Tx0P
A3
Rx0N
B3
Tx0N
A4
GND
B4
GND
A5
Rx1P
B5
Tx1P
A6
Rx1N
B6
Tx1N
A7
GND
B7
GND
A8
Rx2P
B8
Tx2P
A9
Rx2N
B9
Tx2N
A10
GND
B10
GND
A11
Rx3P
B11
Tx3P
A12
Rx3N
B12
Tx3N
A13
GND
B13
GND
http://opencompute.org
13
5.6.3
Assignment
Pin
Assignment
A1
GND
B1
GND
A2
Rx0P
B2
Tx0P
A3
Rx0N
B3
Tx0N
A4
GND
B4
GND
A5
Rx1P
B5
Tx1P
A6
A7
Rx1N
GND
B6
B7
Tx1N
GND
A8
NC
B8
NC
A9
NC
B9
NC
A10
NC
B10
NC
A11
NC
B11
NC
A12
GND
B12
GND
A13
Rx2P
B13
Tx2P
A14
Rx2N
B14
Tx2N
A15
GND
B15
GND
A16
Rx3P
B16
Tx3P
A17
Rx3N
B17
Tx3N
A18
GND
B18
GND
5.6.4
Debug Header
The SAS expander board includes a debug header on the front side. It supports hot
plugging for an existing debug card. The card is used in Open Compute servers and
contains the following functionality:
Two 7-segment LED displays: Show firmware POST information and system error
codes.
One RS-232 serial connector: Provides console redirection.
One reset switch: Triggers a system reset when pressed.
The connector for the debug header is a 14-pin, shrouded, right-angled, 2mm pitch
connector. Figure 14 shows an illustration. The debug card has a key to match with the
notch to avoid pin shift when plugging it in.
14
Function
Serial Transmit
10
Serial Receive
11
System Reset
12
13
GND
14
VCC (5VDC)
5.7
LEDs
The SAS expander board has several LEDs on its front edge to display various types of
information:
Blue LED
Red LED
ON
OFF
OFF
ON
Tray Identify
Flashing
OFF
OFF
Flashing
http://opencompute.org
15
Blue LED
Red LED
ON
OFF
OFF
ON
No SAS links
OFF
OFF
5.8
PCB Stack-up
The PCB stack-up and impedance control for the SAS expander board are defined in
Figure 18 and Figure 19 below.
Layer
Plane Description
Solder Mask
L1
TOP
Signal
0.5 + 1.0
GND1
Ground
IN1
Signal
2.0
VCC1
Power
4.0
1.0
VCC2
Power
13.6
2.0
IN4
Signal
4.0
2.0
GND2
Ground
13.6
1.0
BOT
Signal
4.4
1.3
4.0
2.0
3.7
2.6
PrePreg
L8
4.1
2.6
Core
L7
4.4
2.6
PrePreg
L6
3.7
1.3
Core
L5
3.5
2.6
PrePreg
L4
3.8
2.7
Core
L3
Dielectric (er)
0.5
1.9
PrePreg
L2
Thickness (mil)
2.7
0.5 + 1.0
3.5
1.9
Solder Mask
0.5
3.8
Total
62.4
Tolerance: +/-6mil
Trace Width
(mil)
4.0
4.0
9.0
4.5
4.0
8.0
Impedance
Type
Layer
Impedance
Target (ohm)
Tolerance
(+/- %)
Single
1, 8
50
15.0
Differential
1, 8
100
15.0
Single
3, 6
50
10.0
Differential
3, 6
100
10.0
16
6.1
Block Diagram
Figure 20 illustrates the functional block diagram of the drive plane board (DPB).
High
Power
Card
Edge
Connector
To
Power
Transition
Board
EE-
PROM
Voltage
Monitor
Temp.
Sensor
1
Temp.
Sensor
2
Temp.
Sensor
3
Temp.
Sensor
4
I2C_B
HDD
GPIO
Drive
Status
LED
(Blue/Red)
Drive
Power
Control
Present
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
HDD
6.2
HDD1
HDD2
HDD3
HDD4
HDD5
HDD6
HDD7
HDD8
HDD9
HDD10
HDD11
HDD12
HDD13
HDD14
http://opencompute.org
17
6.3
6.4
6.5
Voltage Monitor
A voltage monitor ensures proper operation of all power rails at all times. The voltages
are reported as part of the enclosure status as described in section 10.1. The voltage rails
that are monitored are shown in Figure 22.
Power Rail
Voltage
5V to HDD group A
5V
5V to HDD group B
5V
5V to HDD group C
5V
12V
6.6
6.6.1
6.6.2
Assignment
Pin
Assignment
P1
12.5V
P19
GND
P2
12.5V
P20
GND
P3
12.5V
P21
GND
P4
12.5V
P22
GND
18
P5
12.5V
P23
GND
P6
12.5V
P24
GND
P7
12.5V
P25
GND
P8
12.5V
P26
GND
P9
GND
P27
GND
P10
GND
P28
GND
P11
GND
P29
12.5V
P12
GND
P30
12.5V
P13
GND
P31
12.5V
P14
GND
P32
12.5V
P15
GND
P33
12.5V
P16
GND
P34
12.5V
P17
GND
P35
12.5V
P18
GND
P36
12.5V
Figure 23 Drive Plane Board Power and Signal Connector Pin-out, Power Portion
Pin
Assignment
Pin
Assignment
S1
GND
S13
GND
S2
3.3V
S14
S3
3.3V
S15
S4
S16
Tray ID
S5
S17
S6
S18
S7
S19
GND
S8
GND
S20
S9
I2C_C SDA
S21
S10
GND
S22
GND
S11
I2C_C SCL
S23
GND
S12
GND
S24
5V (Reserved)
Figure 24 Drive Plane Board Power and Signal Connector Pin-out, Signal Portion
6.6.3
Assignment
Pin
Assignment
P1
3.3V_Precharge
S1
GND
P2
3.3V
S2
Tx1P
P3
3.3V
S3
Tx1N
P4
GND
S4
GND
P5
GND
S5
Rx1N
http://opencompute.org
19
P6
GND
S6
Rx1P
P7
5V_Precharge
S7
GND
P8
5V
S8
GND
P9
5V
S9
Tx2P
P10
GND
S10
Tx2N
P11
Ready_LED
S11
GND
P12
GND
S12
Rx2N
P13
12V_Precharge
S13
Rx2P
P14
12V
S14
GND
P15
12V
6.7
LEDs
On the drive plane board, each hard disk drive has one bi-color LED to indicate its status,
both driven by SAS expander chip:
When the HDD is online and healthy, the blue LED turns on;
When any fault for the HDD occurs, the red LED turns on;
When the HDD is not inserted, both LEDs turn off;
When HDD is powered off, toggle the LED between blue and red.
Each drive's LEDs are located near the corresponding drive's cage and clearly visible
from the top when HDD tray is pulled out. If needed, an optical path for the drive status
LEDs can be provided for easy access by users.
Figure 26 summarizes the conditions the LEDs are to represent:
Disk Drive Status
Blue LED
Red LED
No Drive Inserted
OFF
OFF
ON
OFF
Drive Failure
OFF
ON
Toggle
Toggle
6.8
PCB Stack-up
The PCB stack-up and impedance control for drive plane board are defined in Figure 27
and Figure 28 below.
Layer
Plane Description
Copper Weight
(oz)
Solder Mask
L1
TOP
L2
GND1
Signal
0.5 + 1.0
PrePreg
Ground
20
IN1
Signal
Dielectric (er)
0.5
3.8
1.9
2.7
2.0
Core
L3
Thickness
(mil)
4.0
1.0
3.5
2.6
3.7
1.3
PrePreg
L4
VCC1
Power
13.6
2.0
Core
L5
VCC2
L6
IN4
Power
4.0
2.0
13.6
1.0
GND2
Ground
4.0
2.0
BOT
Signal
3.7
2.6
PrePreg
L8
4.4
1.3
Core
L7
4.1
2.6
PrePreg
Signal
4.4
2.6
2.7
0.5 + 1.0
3.5
1.9
Solder Mask
0.5
3.8
Total
85.2
Tolerance: +/8mil
Trace Width
(mil)
Air Gap
Spacing (mil)
4.0
4.0
9.0
4.5
4.0
8.0
Impedance
Type
Layer
Impedance
Target (ohm)
Tolerance
(+/- %)
Single
1, 8
50
15.0
Differential
1, 8
100
15.0
Single
3, 6
50
10.0
Differential
3, 6
100
10.0
http://opencompute.org
21
7.1
Tray
Power
ON/OFF
Power:
12.5V
/
GND
Tray
Pull-out
Detection
Tray_IN
7.2
7.3
22
7.4
7.4.1
Golden Finger to High Power Card Edge Connector on Drive Plane Board
Golden fingers are designed on the PTB to mate with the high power card edge
connector on the DPB. They total 36 power contacts and 24 low speed signals. Figure 32
shows the PCB footprint. For detailed pin definition, refer to 6.6.2.
7.4.2
http://opencompute.org
23
Pin
Description
P1
12.5V
P2
12.5V
P3
GND
P4
GND
Figure 34 Power and Signal Connector Pin Definition on PTB, Power Portion
Pin
Assignment
Pin
Assignment
A1
I2C_C SDA
C1
A2
GND
C2
A3
I2C_C SCL
C3
A4
A5
GND
PWM from expander B
C4
C5
B1
GND
D1
B2
D2
Tray ID
B3
GND
D3
B4
D4
B5
GND
D5
Figure 35 Power and Signal Connector Pin Definition on PTB, Signal Portion
7.4.3
24
7.5
PCB Stack-up
The PCB stack-up and impedance control for power transition board are defined in the
following figures.
Layer
Plane Description
Copper
Weight (oz)
Solder Mask
S1
TOP
Signal
P2
PLANE
0.5 + 1.0
PrePreg
Power / GND
PLANE
Power / GND
BOT
Signal
0.5
3.4
1.7
3.7
2.6
47.0
2.0
PrePreg
S4
Dielectric (er)
3.5
2.0
Core
P3
Thickness
(mil)
4.2
2.6
3.5
0.5 + 1.0
3.7
1.7
Solder Mask
0.5
3.4
Total
63.6
Trace Width
(mil)
Air Gap
Spacing (mil)
5.5
Impedance
Type
Layer
Impedance
Target (ohm)
Tolerance
(+/- %)
Single
1, 4
50
15.0
http://opencompute.org
25
8.1
Block Diagram
Figure 39 illustrates the functional block diagram of the fan control board (FCB).
Fan
Header
6
Fan
Header
5
Fan
Header
4
Fan
Tachs
Fan
Header
3
Fan
Header
2
Fan
Header
1
PWM
Comparator
Max
Speed
BJT
x
2
(Temp
Sensor)
PWMs
Hardware
Monitor
EEPROM
I2C_C
Power
Monitor
Fan
Module
Status
LED
LED Driver
8.2
Fan
Fan
Header
5 Header
4
Bus-bar
Clips
Fan
Fan
Header
3 Header
2
Fan
Header
1
FusioLug
to
Bus-bar
Clips
8.3
26
Power Rail
Voltage
V12_input
12V
V12_upper_tray
12V
V12_lower_tray
12V
V3.3_FCB
3.3V
8.4
8.5
Connectors
8.5.1
A1
I2C_C SDA
I2C_C SDA
A2
GND
GND
A3
I2C_C SCL
I2C_C SCL
A4
GND
GND
A5
B1
GND
GND
B2
B3
GND
GND
B4
SEB 1B Heartbeat
SEB 2B Heartbeat
B5
GND
GND
C1
C2
C3
SEB 1A Heartbeat
SEB 2A Heartbeat
C4
SEB 2A Heartbeat
SEB 1A Heartbeat
C5
SEB 2B Heartbeat
SEB 1B Heartbeat
D1
D2
D3
D4
D5
http://opencompute.org
27
Pin
To Upper Tray
To Lower Tray
P1
12.5V
12.5V
P2
12.5V
12.5V
P3
GND
GND
P4
GND
GND
8.5.2
Pin
Description
12.5V
GND
28
8.5.3
Description
Tachometer 1
PWM
Tachometer 2
12VDC
GND
8.6
LEDs
The fan control board has six bi-color LEDs on its edge to display the fan module status,
one for each fan module and fault.
Fan Module Status
Blue LED
Red LED
Normal operation
ON
OFF
OFF
ON
8.7
PCB Stack-up
The PCB stack-up and impedance control for the fan control board are defined in the
following tables.
Layer
Plane Description
Copper
Weight (oz)
Solder Mask
S1
TOP
Signal
0.5 + 1.0
PrePreg
P2
PLANE
Power / GND
PLANE
Power / GND
2.0
BOT
Signal
0.5
3.4
1.7
3.7
2.6
47.0
2.0
PrePreg
S4
Dielectric (er)
3.5
Core
P3
Thickness
(mil)
4.2
2.6
3.5
0.5 + 1.0
3.7
1.7
Solder Mask
0.5
3.4
Total
63.6
Trace Width
(mil)
Air Gap
Spacing (mil)
5.5
Impedance
Type
Layer
Impedance
Target (ohm)
Tolerance
(+/- %)
Single
1, 4
50
15.0
http://opencompute.org
29
9.1
Power Budget
Overall power consumption of the Open Vault storage system is approximately 600W:
Figure 50 shows the Open Vault system power budget calculation in full-load operation,
based on each module.
Item
Quantity
in 1
System
Power
Consumption/
Module (W)
Derating %
Power
Consumption
after Derating
(W)
HDD
30
14
420
90%
378
Drive Plane
Board
18
90%
16.2
SAS Expander
Board
15.6
31.2
90%
28.08
Fan Control
Board
90%
5.4
Fan Module
18
108
55%
59.4
Total
583.2
487.08
Figure 51 shows the Open Vault system power budget calculation in idle mode, based on
each module.
Item
Quantity
in 1
System
Power
Consumption/
Module (W)
Derating %
Power
Consumption
after Derating
(W)
HDD
30
7.5
225
100%
225
Drive Plane
Board
18
60%
10.8
SAS Expander
Board
15.6
31.2
70%
21.84
Fan Control
Board
60%
3.6
Fan Module
18
108
30%
32.4
Total
30
583.2
487.08
9.2
9.3
9.3.1
HDD 0
HDD 1
HDD 2
HDD 3
HDD 4
5V
Power
Plane
A
HDD 5
HDD 6
HDD 7
HDD 8
HDD 9
5V
Power
Plane
B
5V
Power
Plane
C
HDD
10
HDD 11
HDD 12
HDD 13
HDD 14
The drive plane board design ensures that the PCB traces and power planes are
wide/thick enough to support the required continuous power as well as the inrush
current to start the drive from idle. The 5VDC regulator supports the additional inrush
current required by each drive as well.
9.3.2
5V Output Protection
The 5V output power regulator protects against shorts and overload conditions. The
protection mode is achieved using a constant current system.
http://opencompute.org
31
9.3.3
9.4
9.5
Power-on Sequencing
The SAS expander chips power on in sequence without any violations, and ensure power
cycling with adequate reliability.
The use of a power button is not required to power on. The system always resumes
operation upon restoration of power during a power failure event.
10
10.1
32
SMP functions
PHY error counters
Topology discovery and routing table management
Staggered disk drive spin-up
10.2
Page Name
00h
01h
Configuration
02h
04h
05h
Enclosure Control
Enclosure Status
String Out
String In
Threshold Out
Threshold In
07h
Element Descriptor
0Ah
0Eh
SMP Function
Report general
00h
Report manufacturing
information
01h
Discover
10h
11h
12h
PHY control
91h
http://opencompute.org
33
10.3
One scheme is to control fan power management by each SAS expander chip
itself with thermal sensors; a PWM comparator on the FCB selects the maximum
PWM value from the four SAS expanders and drives the fans.
The other scheme is each expander chip only reports all temperature values to
the host server; the host server calculates suitable PWM numbers and controls
fan speed via SES commands sent to the SAS expander(s).
Figure 55 shows the temperature sensors' location within the Open Vault system.
HDD 0
HDD 1
HDD 2
HDD 3
HDD 4
Drive-Plane
Board
HDD 5
HDD 6
HDD 7
HDD 8
HDD 9
HDD 10
HDD 11
HDD 12
HDD 13
HDD 14
SAS
Expander
Board
Figure 55 Open Vault System Thermal Sensor Locations
The ODM provides a detailed implementation of the fan speed control requirements.
Figure 56 shows the high-level strategy of Open Vault fan speed control executed by
firmware in each SAS expander.
34
Ambient
Temp
Sensor
A1
Ambient
Temp
Sensor
A2
Expander
Temp
Sensor
A
Ambient
Temp
Sensor
B1
Ambient
Temp
Sensor
B2
Expander
Temp
Sensor
B
Drive
Board
Temp
Sensor
1
Drive
Board
Temp
Sensor
2
Drive
Board
Temp
Sensor
3
Drive
Board
Temp
Sensor
4
Top tray
Duty
Duty
Duty
Duty
Duty
Duty
Duty
Duty
Duty
Duty
Ambient
Temp
Sensor
A1
Ambient
Temp
Sensor
A2
Expander
Temp
Sensor
A
Ambient
Temp
Sensor
B1
Ambient
Temp
Sensor
B2
Expander
Temp
Sensor
B
Drive
Board
Temp
Sensor
1
Drive
Board
Temp
Sensor
2
Drive
Board
Temp
Sensor
3
Drive
Board
Temp
Sensor
4
Bottom tray
10.4
Thermal Protection
There are different levels of consideration of system/hardware thermal protection for
the Open Vault system. They include:
10.5
http://opencompute.org
35
10.6
The group definition of hard disk drives follows the SAS expander chip vendor's
strategy.
The ODM defines the quantity of hard disk drives in each group.
The ODM defines the delay interval between each group.
10.7
10.8
10.9
No
Error
Critical
Crash
-
Expander
Critical
Crash
-
I2C
Bus
Reserved
Fan
Fault
Critical
Reserved
Temperature
Sensor
Critical
Reserved
Voltage
&
Current
Sensor
Critical
Reserved
HDD
SMART
Temp
Critical
Expander
Internal
Temp
Critical
Reserved
HDD
Fault
Reserved
Mini-SAS
Link
Error
Tray
Pulled-out
Reserved
36
11
Mechanical
11.1
External Chassis
The overall dimensions of the chassis are: 536mm wide x 94.5mm tall x 878mm deep.
The individual HDD trays may be carried fully loaded. The chassis may be carried with 2
fully loaded HDD trays. The main chassis has a cover for airflow management and to
improve stiffness. The cover is removed by releasing 2 thumbscrews at the rear, when it
is not installed in the rack.
11.1.1 Serviceability
Most service is from the front of the chassis, which is defined as the "cold aisle" or intake
end of the chassis. Fan service is performed from the rear or "hot aisle" end of the
chassis. All FRU removal, replacement and service is tool-less.
http://opencompute.org
37
38
11.2
HDD Tray
The HDD tray comprises a primary tray, drive plane and SAS expander boards, which are
mounted to the underside of the tray. The expander boards are removable from the
front of the tray. Each tray holds 15x HDDs horizontally, engaging right angle
connectors on the drive plane board.
The HDD tray slides in and out of the chassis from the cold aisle. It rolls on multiple
roller bearings for smooth movement. It has hard stops with deceleration features at
the rear and front of its travel, to minimize operational shock. When it reaches the end
of its travel outside the chassis, it may be tilted downward by 30 degrees for easier
service. The HDD is easily removable without tools, for replacement. See 11.2.1.
The entire HDD tray may be removed for service or replacement by releasing the HDD
tray release levers at the rear of the tray. The HDD tray is retained in the chassis by left
and right side cam latches. The tray's drop rate is controlled by its pair of friction hinges
in such a manner as to avoid HDD damaging deceleration. The power and signal cables
to each tray may be released, and the supporting cable arm is released by unscrewing
the cable arm thumbscrew (Figure 63). The HDD tray may then be removed for service.
http://opencompute.org
39
40
http://opencompute.org
41
42
2. Lift the latch upwards, slide the HDD forward, then lift the HDD out of the bay.
3. To install, slide the new HDD into the bay (press horizontally into the connector),
and press the latch downwards until it latched in place.
http://opencompute.org
43
11.3
44
11.4
http://opencompute.org
45
11.5
Fan Module
There are six fan modules mounted to the hot aisle end of the chassis.
Each fan module carries one dual-rotor fan, 60mm x 56mm size or equivalent, mounted
to individual fan carriers via elastomeric isolators in order to minimize vibration
transmitted to the main chassis. The fans are retained by panel-mounted thumbscrews
in each fan carrier. The carriers are installed onto individual trays, horizontally onto
guide pins.
Installing the fan module involves simply pushing the fan carrier directly into the rear of
the chassis, then engaging the thumbscrew. Removing the fan is done by reversing
these steps.
46
12
Environmental
12.1
Environmental Requirements
The Open Vault storage system meets the following environmental requirements:
In addition, the full Open Vault storage system has an operating altitude with no deratings of 1000m (3300 feet).
12.2
Non-Operating
Vibration
Shock
13
Prescribed Materials
13.1
Sustainable Materials
Wherever possible and practical, and where cost or performance are not compromised,
environmentally sustainable materials and finishes are used. This includes the use of
non-hexavalent metal finishes, recycled and recyclable base materials and materials
made from renewable resources, with associated material certifications.
13.2
Disallowed Components
The following components shall not be used in the design of the motherboard:
http://opencompute.org
47
13.3
Only SMT inductors may be used. The use of through-hole inductors is disallowed.
13.4
Component De-Rating
For inductors, capacitors, and FETs, de-rating analysis should be based on at least 20%
de-rating.
14
14.1
GND
SAS_PRE15
B1
A2
SAS0_RX+
FCB_HW_REV
B2
A3
SAS0_RX-
GND
B3
A4
GND
SAS0_TX+
B4
A5
GND
SAS0_TX-
B5
A6
SAS1_RX+
GND
B6
A7
SAS1_RX-
GND
B7
A8
GND
SAS1_TX+
B8
A9
GND
SAS1_TX-
B9
A10
SAS2_RX+
GND
B10
A11
SAS2_RX-
DPB_HW_REV
B11
Key
Key
Key
Key
A12
Tray_ID
SAS2_TX+
B12
A13
GND
SAS2_TX-
B13
A14
SAS3_RX+
GND
B14
A15
SAS3_RX-
GND
B15
A16
GND
SAS3_TX+
B16
48
A17
GND
SAS3_TX-
B17
A18
SAS4_RX+
GND
B18
A19
SAS4_RX-
GND
B19
A20
GND
SAS4_TX+
B20
A21
GND
SAS4_TX-
B21
A22
SAS5_RX+
GND
B22
A23
SAS5_RX-
GND
B23
A24
GND
SAS5_TX+
B24
A25
GND
SAS5_TX-
B25
A26
SAS6_RX+
GND
B26
A27
SAS6_RX-
GND
B27
A28
GND
SAS6_TX+
B28
A29
GND
SAS6_TX-
B29
A30
SAS7_RX+
GND
B30
A31
SAS7_RX-
GND
B31
A32
GND
SAS7_TX+
B32
A33
GND
SAS7_TX-
B33
A34
SAS8_RX+
GND
B34
A35
SAS8_RX-
GND
B35
A36
GND
SAS8_TX+
B36
A37
GND
SAS8_TX-
B37
A38
SAS9_RX+
GND
B38
A39
SAS9_RX-
GND
B39
A40
GND
SAS9_TX+
B40
A41
Shutdown_Release
SAS9_TX-
B41
A42
SAS1_FLT
GND
B42
A43
SAS0_FLT
SAS9_FLT
B43
A44
SAS2_FLT
SAS10_FLT
B44
A45
SAS3_FLT
SAS11_FLT
B45
A46
SAS4_FLT
SAS12_FLT
B46
A47
SAS5_FLT
SAS13_FLT
B47
A48
SAS6_FLT
GND
B48
A49
SAS7_FLT
SAS10_TX+
B49
A50
SAS8_FLT
SAS10_TX-
B50
A51
GND
GND
B51
A52
SAS10_RX+
SAS14_FLT
B52
A53
SAS10_RX-
SAS15_FLT
B53
A54
GND
SEB_HTBT_OUT
B54
A55
SCL_C
SEB_HTBT_IN
B55
A56
SDA_C
SCL_B
B56
A57
SCL_D
SDA_B
B57
A58
SDA_D
SEB_ID
B58
http://opencompute.org
49
A59
GND
PWM_OUT
B59
A60
SAS11_RX+
PEER_SEBA_HB
B60
A61
SAS11_RX-
GND
B61
A62
GND
SAS11_TX+
B62
A63
GND
SAS11_TX-
B63
A64
SAS12_RX+
GND
B64
A65
SAS12_RX-
GND
B65
A66
GND
SAS12_TX+
B66
A67
A68
GND
SAS13_RX+
SAS12_TXGND
B67
B68
A69
SAS13_RX-
GND
B69
A70
GND
SAS13_TX+
B70
A71
GND
SAS13_TX-
B71
A72
SAS14_RX+
GND
B72
A73
SAS14_RX-
GND
B73
A74
GND
SAS14_TX+
B74
A75
GND
SAS14_TX-
B75
A76
SAS15_RX+
GND
B76
A77
SAS15_RX-
GND
B77
A78
A79
GND
PEER_SEBB_HB
SAS15_TX+
SAS15_TX-
B78
B79
A80
12V
GND
B80
A81
12V
GND
B81
A82
12V
GND
B82
Figure 68 Pin Definition from SEB to DPB I (164 Pin Connector to Drive Plane Board)
A1
SAS_PRE0
SAS_PWR0
B1
A2
SAS_PRE1
SAS_PWR1
B2
A3
SAS_PRE2
SAS_PWR2
B3
A4
SAS_PRE3
SAS_PWR3
B4
A5
SAS_PRE4
SAS_PWR4
B5
A6
SAS_PRE5
SAS_PWR5
B6
A7
SAS_PRE6
SAS_PWR6
B7
A8
SAS_PRE7
SAS_PWR7
B8
A9
SAS_PRE8
SAS_PWR8
B9
A10
SAS_PRE9
SAS_PWR9
B10
A11
SAS_PRE10
SAS_PWR10
B11
Key
Key
Key
Key
A12
SAS_PRE11
SAS_PWR11
B12
A13
SAS_PRE12
SAS_PWR12
B13
A14
SAS_PRE13
SAS_PWR13
B14
50
A15
SAS_PRE14
SAS_PWR14
B15
A16
SELF_TRAY_IN
SAS_PWR15
B16
A17
PEER_TRAY_IN
PCIe_MATED
B17
A18
GND
PEER_SEB_IN
B18
Figure 69 Pin Definition from SEB to DPB II (36 Pin Connector to Drive Plane Board)
14.2
A1
GND
HDD_PRE15
B1
A1
GND
HDD_PRE15
B1
A2
HDD11_RX+
FCB_HW_REV
B2
A2
HDD14_RX+
FCB_HW_REV
B2
A3
HDD11_RX-
GND
B3
A3
HDD14_RX-
GND
B3
A4
GND
HDD11_TX+
B4
A4
GND
HDD14_TX+
B4
A5
GND
HDD11_TX-
B5
A5
GND
HDD14_TX-
B5
A6
HDD12_RX+
GND
B6
A6
HDD4_RX+
GND
B6
A7
HDD12_RX-
GND
B7
A7
HDD4_RX-
GND
B7
A8
GND
HDD12_TX+
B8
A8
GND
HDD4_TX+
B8
A9
GND
HDD12_TX-
B9
A9
GND
HDD4_TX-
B9
A10
HDD13_RX+
GND
B10
A10
HDD9_RX+
GND
B10
A11
HDD13_RX-
DPB_HW_REV
B11
A11
HDD9_RX-
DPB_HW_REV
B11
Key
Key
Key
Key
Key
Key
Key
Key
A12
Tray_ID
HDD13_TX+
B12
A12
Tray_ID
HDD9_TX+
B12
A13
GND
HDD13_TX-
B13
A13
GND
HDD9_TX-
B13
A14
HDD14_RX+
GND
B14
A14
HDD3_RX+
GND
B14
A15
HDD14_RX-
GND
B15
A15
HDD3_RX-
GND
B15
A16
GND
HDD14_TX+
B16
A16
GND
HDD3_TX+
B16
A17
GND
HDD14_TX-
B17
A17
GND
HDD3_TX-
B17
A18
HDD9_RX+
GND
B18
A18
HDD2_RX+
GND
B18
A19
HDD9_RX-
GND
B19
A19
HDD2_RX-
GND
B19
A20
GND
HDD9_TX+
B20
A20
GND
HDD2_TX+
B20
A21
GND
HDD9_TX-
B21
A21
GND
HDD2_TX-
B21
A22
HDD8_RX+
GND
B22
A22
HDD8_RX+
GND
B22
A23
HDD8_RX-
GND
B23
A23
HDD8_RX-
GND
B23
A24
GND
HDD8_TX+
B24
A24
GND
HDD8_TX+
B24
A25
GND
HDD8_TX-
B25
A25
GND
HDD8_TX-
B25
A26
HDD7_RX+
GND
B26
A26
HDD7_RX+
GND
B26
A27
HDD7_RX-
GND
B27
A27
HDD7_RX-
GND
B27
A28
GND
HDD7_TX+
B28
A28
GND
HDD7_TX+
B28
A29
GND
HDD7_TX-
B29
A29
GND
HDD7_TX-
B29
A30
HDD4_RX+
GND
B30
A30
HDD1_RX+
GND
B30
http://opencompute.org
51
A31
HDD4_RX-
GND
B31
A31
HDD1_RX-
GND
B31
A32
GND
HDD4_TX+
B32
A32
GND
HDD1_TX+
B32
A33
GND
HDD4_TX-
B33
A33
GND
HDD1_TX-
B33
A34
HDD3_RX+
GND
B34
A34
HDD0_RX+
GND
B34
A35
HDD3_RX-
GND
B35
A35
HDD0_RX-
GND
B35
A36
GND
HDD3_TX+
B36
A36
GND
HDD0_TX+
B36
A37
GND
HDD3_TX-
B37
A37
GND
HDD0_TX-
B37
A38
HDD6_RX+
GND
B38
A38
HDD13_RX+
GND
B38
A39
HDD6_RX-
GND
B39
A39
HDD13_RX-
GND
B39
A40
GND
HDD6_TX+
B40
A40
GND
HDD13_TX+
B40
A41
SD_Release
HDD6_TX-
B41
A41
SD_Release
HDD13_TX-
B41
A42
HDD12_FLT
GND
B42
A42
HDD4_FLT
GND
B42
A43
HDD11_FLT
HDD6_FLT
B43
A43
HDD14_FLT
HDD13_FLT
B43
A44
HDD13_FLT
HDD2_FLT
B44
A44
HDD9_FLT
HDD6_FLT
B44
A45
HDD14_FLT
HDD1_FLT
B45
A45
HDD3_FLT
HDD5_FLT
B45
A46
HDD9_FLT
HDD10_FLT
B46
A46
HDD2_FLT
HDD10_FLT
B46
A47
HDD8_FLT
HDD0_FLT
B47
A47
HDD8_FLT
HDD11_FLT
B47
A48
HDD7_FLT
GND
B48
A48
HDD7_FLT
GND
B48
A49
HDD4_FLT
HDD2_TX+
B49
A49
HDD1_FLT
HDD6_TX+
B49
A50
HDD3_FLT
HDD2_TX-
B50
A50
HDD0_FLT
HDD6_TX-
B50
A51
GND
GND
B51
A51
GND
GND
B51
A52
HDD2_RX+
HDD5_FLT
B52
A52
HDD6_RX+
HDD12_FLT
B52
A53
HDD2_RX-
HDD15_FLT
B53
A53
HDD6_RX-
HDD15_FLT
B53
A54
GND
SEB_HTBT_OUT
B54
A54
GND
SEB_HTBT_OUT
B54
A55
SCL_C
SEB_HTBT_IN
B55
A55
SCL_C
SEB_HTBT_IN
B55
A56
SDA_C
SCL_B
B56
A56
SDA_C
SCL_B
B56
A57
SCL_D
SDA_B
B57
A57
SCL_D
SDA_B
B57
A58
SDA_D
SEB_ID
B58
A58
SDA_D
SEB_ID
B58
A59
GND
PWM_OUT
B59
A59
GND
PWM_OUT
B59
A60
HDD1_RX+
PEER_SEBA_HB
B60
A60
HDD5_RX+
PEER_SEBA_HB
B60
A61
HDD1_RX-
GND
B61
A61
HDD5_RX-
GND
B61
A62
GND
HDD1_TX+
B62
A62
GND
HDD5_TX+
B62
A63
GND
HDD1_TX-
B63
A63
GND
HDD5_TX-
B63
A64
HDD10_RX+
GND
B64
A64
HDD10_RX+
GND
B64
A65
HDD10_RX-
GND
B65
A65
HDD10_RX-
GND
B65
A66
GND
HDD10_TX+
B66
A66
GND
HDD10_TX+
B66
A67
GND
HDD10_TX-
B67
A67
GND
HDD10_TX-
B67
A68
HDD00_RX+
GND
B68
A68
HDD11_RX+
GND
B68
A69
HDD00_RX-
GND
B69
A69
HDD11_RX-
GND
B69
A70
GND
HDD0_TX+
B70
A70
GND
HDD11_TX+
B70
A71
GND
HDD0_TX-
B71
A71
GND
HDD11_TX-
B71
52
A72
HDD5_RX+
GND
B72
A72
HDD12_RX+
GND
B72
A73
HDD5_RX-
GND
B73
A73
HDD12_RX-
GND
B73
A74
GND
HDD5_TX+
B74
A74
GND
HDD12_TX+
B74
A75
GND
HDD5_TX-
B75
A75
GND
HDD12_TX-
B75
A76
HDD15_RX+
GND
B76
A76
HDD15_RX+
GND
B76
A77
HDD15_RX-
GND
B77
A77
HDD15_RX-
GND
B77
A78
GND
HDD15_TX+
B78
A78
GND
HDD15_TX+
B78
A79
PEER_SEBB_HB
HDD15_TX-
B79
A79
PEER_SEBB_HB
HDD15_TX-
B79
A80
12V
GND
B80
A80
12V
GND
B80
A81
12V
GND
B81
A81
12V
GND
B81
A82
12V
GND
B82
A82
12V
GND
B82
A1
HDD_PRE11
HDD_PWR11
B1
A1
HDD_PRE14
HDD_PWR14
B1
A2
HDD_PRE12
HDD_PWR12
B2
A2
HDD_PRE4
HDD_PWR4
B2
A3
HDD_PRE13
HDD_PWR13
B3
A3
HDD_PRE9
HDD_PWR9
B3
A4
HDD_PRE14
HDD_PWR14
B4
A4
HDD_PRE3
HDD_PWR3
B4
A5
HDD_PRE9
HDD_PWR9
B5
A5
HDD_PRE2
HDD_PWR2
B5
A6
HDD_PRE8
HDD_PWR8
B6
A6
HDD_PRE8
HDD_PWR8
B6
A7
HDD_PRE7
HDD_PWR7
B7
A7
HDD_PRE7
HDD_PWR7
B7
A8
HDD_PRE4
HDD_PWR4
B8
A8
HDD_PRE1
HDD_PWR1
B8
A9
HDD_PRE3
HDD_PWR3
B9
A9
HDD_PRE0
HDD_PWR0
B9
A10
HDD_PRE6
HDD_PWR6
B10
A10
HDD_PRE13
HDD_PWR13
B10
A11
HDD_PRE6
HDD_PWR6
B11
A11
HDD_PRE2
HDD_PWR2
B11
Key
Key
Key
Key
Key
Key
Key
Key
A12
HDD_PRE1
HDD_PWR1
B12
A12
HDD_PRE5
HDD_PWR5
B12
A13
HDD_PRE10
HDD_PWR10
B13
A13
HDD_PRE10
HDD_PWR10
B13
A14
HDD_PRE0
HDD_PWR0
B14
A14
HDD_PRE11
HDD_PWR11
B14
A15
HDD_PRE5
HDD_PWR5
B15
A15
HDD_PRE12
HDD_PWR12
B15
A16
SELF_tray_IN
HDD_PWR15
B16
A16
SELF_tray_IN
HDD_PWR15
B16
A17
PEER_tray_IN
PCIe_MATED
B17
A17
PEER_tray_IN
PCIe_MATED
B17
A18
SEB_A_INS (G)
SEB_B_INS
B18
A18
SEB_B_INS (G)
SEB_A_INS
B18
http://opencompute.org
53
15
Description
Error
Code
Description
00
No error
50
01
Expander A fault
51
02
Expander B fault
52
03
53
04
54
05
55
06
56
07
Reserved
57
08
Reserved
58
09
Reserved
59
10
Reserved
60
11
61
12
62
13
63
14
64
15
65
16
66
17
67
Reserved
18
68
Reserved
19
69
Reserved
20
70
HDD0 fault
21
71
HDD1 fault
22
72
HDD2 fault
23
Reserved
73
HDD3 fault
24
Reserved
74
HDD4 fault
25
Reserved
75
HDD5 fault
26
Reserved
76
HDD6 fault
27
Reserved
77
HDD7 fault
28
Reserved
78
HDD8 fault
29
Reserved
79
HDD9 fault
30
Reserved
80
HDD10 fault
31
81
HDD11 fault
32
82
HDD12 fault
33
83
HDD13 fault
34
84
HDD14 fault
54
35
85
Reserved
36
86
Reserved
37
87
Reserved
38
88
Reserved
39
89
Reserved
40
90
41
91
42
92
43
Reserved
93
44
Reserved
94
45
95
Reserved
46
Reserved
47
Reserved
48
Reserved
49
Reserved
Reserved
99
16
Revision History
Version
Changes
0.5
0.6
0.7
http://opencompute.org
55