3ware: Supports The 9000 Series
3ware: Supports The 9000 Series
3ware: Supports The 9000 Series
3ware®
SATA RAID Controller
PN 720-0117-01
March 2005
Trademarks
3ware, Escalade, and 3DM are all registered trademarks of AMCC.
The 3ware logo, 3BM, StorSwitch, TwinStor, and R5 Fusion are all
trademarks of AMCC. All other trademarks herein are property of
their respective owners.
Disclaimer
AMCC assumes no responsibility for errors or omissions in this
document, nor does AMCC make any commitment to update the
information contained herein.
www.3ware.com i
www.3ware.com iii
This guide describes how to install the 3ware 9000 series RAID
controller, configure arrays, and make the arrays available to your
operating system—Microsoft Windows, Red Hat Linux, SuSE
Linux, or FreeBSD.
For more complete information about managing and using arrays
connected to the 3ware RAID controller, see 3ware 9000 Series
Serial ATA RAID Controller User Guide.
You may also want to refer to the Knowledgebase available at the
3ware website at http://www.3ware.com/support/
www.3ware.com v
Product Features
The 3ware 9000 Series Serial ATA family includes 9500S-4LP,
9500S-8, 9500S-12, 9500S-8MI, and 9500S-12MI.
Features of the 3ware 9000 series controllers include:
Advanced RAID features for greater data protection and
management.
Support for battery backup provides added data protection in
the event of a power outage. (Battery Backup Unit sold
separately.)
Support for RAID units greater than 2 terabytes with 64-bit
LBA support.
www.3ware.com 1
System Requirements
3ware RAID controllers require the following:
A workstation-class or server-class motherboard which
meets the following criteria:
PCI slots comply with PCI 2.2 or above standards.
PCI slot that meets the Plug and Play and PC99
specifications.
Note: For all 3ware 9000 series models, install the card in a 64-
bit, 66MHz PCI or PCI-X slot for best performance.
Drives
Depending on the particular model, the 3ware RAID controller
may be connected to up to, four, eight, or twelve SATA drives
using the supplied interface cables.
Drives must meet serial ATA 150 (SATA I) or serial ATA 300
(SATA 2) Gb/s standards. Drives may be of any capacity or
physical form factor.
The length of both shielded and unshielded interface cables
may not exceed 1M (39”) for serial ATA controllers.
Operating System
3ware RAID controllers may be used with:
Windows 2000 (Professional, Server, Advanced Server)
(SP3 or newer), Windows XP Professional (SP1 or newer),
Windows Server 2003 (Standard, Enterprise), Windows XP
x64 Edition, Windows Server 2003 x64 Edition.
Red Hat Linux, 32-bit and 64-bit x86
SuSE Linux, 32-bit and 64-bit x64
Other versions of Linux using the open source Linux driver
FreeBSD
Other Requirements
Adequate air flow and cooling
Adequate power supply for drives
For a complete listing of features and system requirements, refer to
the 9000 Series Datasheet, available from the website at
www.3ware.com/products/serial_ata9000.asp.
www.3ware.com 3
Package Contents
If your package is missing any of the items listed below, contact
3ware before proceeding with installation (disk drives and disk
mounting brackets are not included).
3ware Serial ATA RAID Controller in an ESD-protective bag
Serial ATA interface cables (one per port). (Cables are not
included in 9500S-8MI and 9500S-12MI.)
3ware installation CD with the following: 3DM 2, drivers,
Command Line Interface (CLI) and 3ware 9000 Series Serial
ATA RAID Controller User Guide (.pdf format)
3ware Release Notes
3ware 9000 Series Serial ATA RAID Controller Installation
Guide (this guide)
www.3ware.com 5
www.3ware.com 7
Installation Considerations
Cable Length, Routing Space, and Air Flow
When you are installing multiple drives with a controller, make sure
there is adequate space in the chassis for the required cables. It is
important that the cables do not obstruct the air flow or prevent
proper ventilation of the system.
www.3ware.com 9
Tools Required
You will need the following tools during installation:
An ESD grounding strap or mat
Standard hand tools to open your system’s case and install the
3ware RAID controller into an available PCI expansion slot.
Safety Factors
Be sure to follow the guidelines presented on the next few pages to
insure your own safety, and that of your equipment.
Personal Safety
www.3ware.com 11
Ports 10 Ports:
and 11 6 and 7
Ports 8
and 9 4 and 5
2 and 3
0 and 1
Serial
number
(on plate)
LED indicators Overall LED drive status I2C Serial ports are double-
for individual indicator: the last two pins connector stacked connectors.
drives on J7 of J7. The anode is the Odd-numbered ports 1
and J8 lower of the two pins and through 7 are located
the cathode is the upper. below even-numbered
ports 0 through 6.
Ports:
6 and 7
4 and 5
2 and 3
0 and 1
Serial
number
(on plate)
www.3ware.com 13
LED
Controller Pin Pair Comment
Header
9500S-4LP J3 : : : : : Orientation Horizontal
0 1 2 3 All Port number/All
(all activity indicator)
k-cathode-minus is on the top
a-anode-plus is on the bottom
9500S-8 J7 : : : : : Orientation Horizontal
(Pchip v1.4)
0 1 2 3 NU Port number/NU (Not Used)
J8 : : : : : Orientation Horizontal
4 5 6 7 NU Port number/NU (Not Used)
J9 : : : : : Orientation Horizontal
NU NU NU NU All Not used/All
(all activity indicator)
k-cathode-minus is on the top
a-anode-plus is on the bottom
9500S-8 J7 : : : : : Orientation Horizontal
(Pchip v1.5)
0 1 2 3 All Port number/All
(all activity indicator)
J8 : : : : : Orientation Horizontal
4 5 6 7 NU Port number / NU
(Not Used)
k-cathode-minus is on the top
a-anode-plus is on the bottom
LED
Controller Pin Pair Comment
Header
9500S-12 J7 : : : : : Orientation Horizontal
0 1 2 3 NU Port number/NU (Not Used)
J8 : : : : : Orientation Horizontal
4 5 6 7 NU Port number/NU (Not
Used)
J9 : : : : : Orientation Horizontal
8 9 10 11 All Port number/All (all activity
indicator)
k-cathode-minus is on the top
a-anode-plus is on the bottom
Ports:
6 and 7
4 and 5
2 and 3
0 and 1
Serial
number
(on plate)
www.3ware.com 15
Ports:
2 and 3
0 and 1
Serial
number
(on plate)
1 Take out the serial cables provided with the 3ware SATA RAID
controller.
3ware serial controllers are supplied with serial interface
cables, one for each port on the controller.
One edge of each interface cable connector is keyed so that it
can only be inserted in one direction. This helps to ensure
proper orientation and installation
2 Align the cable connector with the connector on the controller,
matching the slotted key and carefully mate the connectors.
2 Make sure you are properly grounded. (For details about safety
precautions, see page 10.)
4 Find the PCI slot you want to use for the serial 3ware RAID
controller.
For a discussion of which slot to use, see “Selecting the Slot in
Which to Install the Controller” on page 8
6 Position the card in the slot so that the contacts will mate with
the grooves in the slot, and all pins make proper contact with
the PCI slot pins when pushed into place.
The 3ware RAID controller is keyed to ensure proper
installation into a full-sized PCI slot.
Make sure that the contacts will mate with the grooves in the
slot.
www.3ware.com 17
Ports
8 to 11
Ports
4 to 7
Ports
0 to 3
www.3ware.com 19
www.3ware.com 21
3 Power down the system, disconnect the drives from the 7/8000
controller and remove the controller from the system.
Finishing Up
After you have installed the controller in the computer and attached
appropriate cables to the controller and drives, complete the
following steps to complete the hardware installation.
www.3ware.com 23
1 Upgrade your driver to the 9000 series driver. (For details, refer
to the Linux or FreeBSD Driver Installation section.)
3 Power down the system, disconnect the drives from the 8000
controller and remove the controller from the system.
4 Install the 9000 controller and attach the drives to the 9000
board, as described earlier in this section.
4 Power down the system, disconnect the drives from the 7/8000
controller and remove that controller from the system.
2 Power down the system, disconnect the drives from the 8000
controller and remove the controller from the system.
3 Install the 9000 controller and attach the drives to the 9000
board, as described earlier in this section.
4 Reboot the system and enter the 3ware 9000 series BIOS
(3BM) by pressing Alt-3. Enable the Export JBOD policy on
the Policy screen. (The next section in this guide, “Configuring
Units”, describes how to work in 3BM. Additional information
about 3BM is available in 3ware 9000 Series Serial ATA RAID
Controller User Guide.)
3 Power down the system, disconnect the drives from the 8000
controller and remove that controller from the system.
5 Reboot the system and enter the 3ware 9000 series BIOS
(3BM) by pressing Alt-3. Enable the Export JBOD policy on
the Policy screen. (The next section in this guide, “Configuring
Units”, describes how to work in 3BM. Additional information
about 3BM is available in 3ware 9000 Series Serial ATA RAID
Controller User Guide.)
www.3ware.com 25
www.3ware.com 27
www.3ware.com 29
Possible Configurations
# Drives
www.3ware.com 31
Possible Configurations
# Drives
2 RAID 0 or RAID 1
3 RAID 0
RAID 1 with hot spare
RAID 5
6 or more RAID 50
Depending on the number of drives, a RAID 50 may
contain from 2 to 4 subunits. For example, with 12
drives, possible RAID 50 configurations include 2
subunits of 6, 3 subunits of 4, or 4 subunits of 3. With
10 drives, a RAID 50 will contain 2 subunits of 5
drives each.
Combination of RAID 0, 1, 5, 10, hot spare, or single disks
Through drive coercion, the capacity used for each drive is rounded
down so that drives from differing manufacturers are more likely to
be able to be used as spares for each other. The capacity used for
each drive is rounded down to the nearest GB for drives under 45
GB (45,000,000,000), and rounded down to the nearest 5 GBytes
for drives over 45 GB. For example, a 44.3 GB drive will be
rounded down to 44 GBytes, and a 123 GB drives will be rounded
down to 120 GBytes.
www.3ware.com 33
Starting 3BM
You access the 3BM configuration utility during the start-up
process for your computer.
To start 3BM
www.3ware.com 35
Exiting 3BM
When you are ready to exit the 3BM configuration utility, you have
the option to save the configuration changes you have made, or to
discard the changes.
1 Press Esc.
If you have made changes that you have not saved, you are
asked whether to save them, or not.
The main 3BM screen shows a list of available drives. After you
have configured one or more drives, it also lists the existing
configuration.
Available Drives lists the drives that are available to create
units and hot spares.
Exportable Units lists any existing units along with the drives
contained in the units.
Incomplete Drives and Others. If you connected drives to the
controller that were previously used on a 7000/8000-series
3ware RAID controller, they may appear in a section titled
Incomplete Drives and Others. Before you can use these drives,
they must be deleted. For more information, see the discussion
of Deleting Units in 3ware 9000 Series Serial ATA RAID
Controller User Guide.
www.3ware.com 37
www.3ware.com 39
2 After all drives for the unit are selected, use the Tab or Right
Arrow key to move to the Create Unit button and press Enter.
The Create Disk Array screen appears (see Figure 11 and
Figure 12 for examples).
www.3ware.com 41
1 Use the arrow keys or press Tab to move to the field Array’s
Write Cache State.
2 Select whether you want the write cache to be enabled or
disabled for this unit.
The default is for write cache to be enabled. For more
information about write cache, see page 31.
3 Press Tab to move to the field Stripe Size and select the desired
stripe size (16KB, 64KB, or 256KB).
www.3ware.com 43
3 If you configured more than one unit, and you plan to install the
operating system on one of them, make that unit be the first unit
(Unit 0) in the list of Exportable Units.
To move a unit up in the list, highlight it and press the Page Up
key.
www.3ware.com 45
2 Type s to specify that the selected drive will be the hot spare.
You’ll see the words “Hot Spare” appear next to the drive in the
Available Drives list.
To enable auto-carving
www.3ware.com 47
Initializing Units
Performance of RAID 5 units with 5 or more disks, and RAID 50
units with 10 or 12 disks configured into two subunits will improve
after the unit has been initialized. For these configurations,
foreground initialization (also known as “zeroing”) starts
immediately. If you want to begin using the units right away, you
can halt the zeroing process and initialize them in the background
later, after the operating system is booted. (The initialization
process can take several hours, depending on the size of your
drives.) The disadvantage of initializing the units later in the
background is that the performance RAID 5 and RAID 50 arrays
will be lower until initialization is complete.
Information about initialization for each of the different RAID
types is described below.
RAID 0 Units
RAID 0 units do not need to be initialized and are immediately
available for use with full performance when created.
www.3ware.com 49
www.3ware.com 51
www.3ware.com 53
Materials Required
3ware software CD-ROM
Microsoft Windows 2000, XP, or Server 2003 installation CD-
ROM. (Not required if Windows is already installed on another
drive.).
Floppy diskette, to create a driver diskette.
If you are installing Windows, you must create a 3ware driver
diskette. If Windows is already installed on another device, you
may install the 3ware driver from either the 3ware software
CD-ROM or from a diskette.
2 When you see the message: “Setup could not determine the
type of one or more mass storage devices or you have chosen to
manually specify an adapter…”
Type S to specify that you have an additional 3ware RAID
controller.
www.3ware.com 55
1 Click the Next button and respond to the questions the Wizard
displays using the default selections.
www.3ware.com 57
www.3ware.com 59
After the driver has been installed, continue with the instructions
under “Making Units Managed by a 3ware Controller Available to
Windows” on page 63.
1 Click the Next button and respond to the questions the Wizard
displays.
2 When the Wizard prompts you to select a device driver, click
Have Disk, and then indicate that it is on the CD or floppy.
www.3ware.com 61
After the driver has been installed, continue with the instructions
below under “Making Units Managed by a 3ware Controller
Available to Windows” on page 63.
1 Remove the driver diskette or CD, reboot the system, and log in
as the system administrator.
2 Use Disk Administrator to partition and format the new arrays
or disks:
a From the Start menu, choose Administrative Tools >
Computer Management.
b In the Computer Management window, under Storage,
select Disk Management.
www.3ware.com 63
Finishing Up
Your RAID unit is now ready for use.
You may also want to install 3ware’s browser-based Disk
Management tool, 3DM 2, or the 3ware Command Line Interface
(CLI). With 3DM 2 and CLI, you can maintain your RAID units
while running Windows. For more information, see the 3ware 9000
Series Serial ATA RAID Controller User Guide and the 3ware
Escalade 9000 Series Serial ATA RAID Controller CLI Guide.
www.3ware.com 65
refer only to the 9000 series driver. For information about using the
7/8000 driver and controller, refer to the user guide or installation
procedure for the 7/8000-series controller.
www.3ware.com 67
If you have a 2.6 kernels with sysfs, type the following command:
cat /sys/class/scsi_host/<hostid>/stats
2 When the 3ware Escalade Menu appears, click the Driver &
Firmware Disks button.
3 In the 3ware Driver & Firmware Disk Images menu, click the
button for the driver disk you want to create.
A confirmation window opens.
4 Insert a blank diskette into a drive and click the Yes button to
begin creating the driver floppy diskette.
www.3ware.com 69
A number of files will load and then a message will prompt you
to insert your driver disk.
www.3ware.com 71
www.3ware.com 73
9 Reboot.
The 3ware driver will be loaded from the ram disk
automatically at boot time.
To install the 3ware driver and load the driver manually instead
of using a RAM disk
www.3ware.com 75
Notes:
You must have a full Linux kernel source tree in /usr/src/linux
in order to do the compile.
In addition, you must boot to the UP kernel to use the make targets.
Otherwise errors will result that will cause the driver build to fail.
The instructions below are for compiling the driver under a 2.4
kernel. For information about compiling the driver under a 2.6
kernel, see the Knowledgebase on the website at www.3ware.com
5 Red Hat 7.2 and later support multiple build targets from
makefile. Boot to UP version of kernel to build all
modules.
www.3ware.com 77
Note: In the commands below, remember that the text you enter
is case sensitive. (Be sure to type Makefile.rh, not
makefile.rh.)
www.3ware.com 79
3 In the 3ware Driver and Firmware Disk Images menu, click the
button for the driver disk you want to create.
A confirmation window opens.
4 Insert a blank diskette into a drive and click the Yes button to
begin copying the driver to the floppy diskette.
1 Log in as root.
www.3ware.com 81
3 If you are using SuSE 9.1 or earlier, after the existing 3w-xxx
entry, add
3w-9xxx to /lib/modules/<kernel string>/modules.dep
file
4 Mount the CD-ROM, copy and install the appropriate driver for
your system.
Note: The AMD 64-bit driver is also used for 64-bit Intel Xeon
systems.
mount /dev/cdrom /cdrom
cp /cdrom/packages/drivers/linux/suse/***/3w-9xxx.* /
lib/modules/<kernel string>/kernel/drivers/scsi/
/sbin/insmod /lib/modules/<kernel string>/
kernel/drivers/scsi/3w-9xxx.ko
5 Run /sbin/depmod -a
6 Type /sbin/mkinitrd
7 If you are using the GRUB boot loader, you are finished.
If you are using the LILO boot loader, run LILO to update the
boot record on disk by typing the following:
/sbin/lilo
1 Download the 3ware 2.4 driver source from the 3ware website
(www.3ware.com) or copy the driver source from the CD into a
working directory.
2 Install the Kernel source under /usr/src.
www.3ware.com 83
5 Copy the config file for the kernel you want to build module for
from the appropriate /usr/src/linux/arch directory (listed below)
to /usr/src/linux/.config.
kernel directory
x86 UP /usr/src/linux/arch/i386/defconfig.default
6 Make oldconfig
7 Untar the driver source with this command:
tar zxfv 3w-9xxx.tgz
1 Download the 3ware 2.6 driver source from the 3ware website
(www.3ware.com).
2 Copy the driver source into a working directory. (The latest
3ware driver source is bundled with the latest firmware and is
in the /src sub-directory.)
3 Install the Kernel source under /usr/src
6 Copy the config file for the kernel you want to build module for
from the appropriate /usr/src/linux/arch directory (listed below)
to /usr/src/linux/.config.
kernel directory
x86 UP /usr/src/linux/arch/i386/defconfig.default
x86_64 UP /usr/src/linux/arch/x86_64/defconfig.default
7 Make oldconfig
8 Untar the driver source with this command:
tar zxfv 3w-9xxx.tgz
www.3ware.com 85
www.3ware.com 87
Materials Required
3ware driver
The driver and the corresponding source files can be found on
the 3ware CD-ROM, at
/packages/drivers/freebsd/<OS version>
and
/packages/drivers/freebsd/src/<OS version>
7 At the loader prompt, run 'lsdev' to make sure the floppy drive
has been discovered.
8 Load the driver from the floppy. For example, if the floppy
drive is disk0, type
load disk0:twa.ko
9 Run 'boot'.
The system should now be able to boot.
10 After the system boots up, do the following:
a Create a directory for the floppy.
mkdir /floppy
www.3ware.com 89
www.3ware.com 91
TWA_FIRMWARE_FLASH opt_twa.h
Note: Since twa.ko behaves like a SCSI driver, the units that it
controls are seen as da0, da1 etc.
www.3ware.com 93
www.3ware.com 95
Hardware installation
Q1: The rail on the 3ware RAID controller doesn't fit in
the case.
www.3ware.com 97
www.3ware.com 99
Installation Overview
The Battery Backup Unit (BBU) attaches to the controller at three
points, as illustrated in Figures 1 through 3:
a Slots on the edge of the controller (visible after removing the
PCI bracket) match to clips on the BBU
b Receptacle on the controller matches to connector on the BBU
c Post hole on the controller mates to post on the BBU
100 3ware 9000 Series Serial ATA RAID Controller Installation Guide
www.3ware.com 101
Installation Instructions
1 For full-height controllers, turn the controller memory-side
down on a flat surface.
2 Remove the screws that connect the PCI bracket to the
controller and set the bracket aside.
102 3ware 9000 Series Serial ATA RAID Controller Installation Guide
Cable
management
clips
5 Match the clips on the end of the BBU to the slots on the edge
of the controller to create a hinge and rotate the BBU to
position it over the card, as shown in Figures 6 and 7.
www.3ware.com 103
104 3ware 9000 Series Serial ATA RAID Controller Installation Guide
www.3ware.com 105
106 3ware 9000 Series Serial ATA RAID Controller Installation Guide
www.3ware.com 107
108 3ware 9000 Series Serial ATA RAID Controller Installation Guide
www.3ware.com 109
110 3ware 9000 Series Serial ATA RAID Controller Installation Guide
www.3ware.com 111
Appendix C. Warranty,
Technical Support, and
Service
Limited Warranty
RAID Controller Hardware. 3-Year Hardware Warranty: AMCC
warrants this product against defects in material and workmanship
for a period of thirty-six (36) months from the date of original
purchase. AMCC, at no charge and at its option, will repair or
replace any part of this product which proves defective by reason of
improper workmanship or materials. Repair parts or replacement
products will be provided by AMCC on an exchange basis and will
be either new or refurbished to be functionally equivalent to new.
Products or parts replaced under this provision shall become the
property of AMCC.
Battery Backup Unit (BBU) Hardware. 1-Year Hardware
Warranty: AMCC warrants this product against defects in material
and workmanship for a period of twelve (12) months from the date
of original purchase. AMCC, at no charge and at its option, will
repair or replace any part of this product which proves defective by
reason of improper workmanship or materials. Repair parts or
replacement products will be provided by AMCC on an exchange
basis and will be either new or refurbished to be functionally
equivalent to new. Products or parts replaced under this provision
shall become the property of AMCC.
Software Warranty: AMCC will replace a defective media
purchased with this product for a period of up to 30 days from the
date of purchase.
112 3ware 9000 Series Serial ATA RAID Controller Installation Guide
For detailed terms and conditions, please see the Limited Hardware
Warranty and Software License Agreement at our website:
http://www.3ware.com/support/rma_form.asp
AMCC warranty service is provided by returning the defective
product to AMCC.
Exclusions
This warranty does not cover any damage to this product which
results from accident, abuse, misuse, natural or personal disaster, or
any unauthorized disassembly, repair or modification. AMCC shall
not be liable for any incidental or consequential damages, including
but not limited to loss of profits, other loss, damage or expense
directly or indirectly arising from the customer's misuse of or
inability to use the product, either separately or in combination with
other equipment, regardless of whether AMCC has been advised of
the possibility of such damages. AMCC is not liable for and does
not cover under warranty, any costs associated with servicing and/
or the installation of AMCC products. This warranty sets for the
entire liability and obligations of AMCC with respect to breach of
warranty and the warranties set forth or limited herein are the sole
warranties and are in lieu of all other warranties, expressed or
implied, including warranties or fitness for particular purpose and
merchantability.
www.3ware.com 113
114 3ware 9000 Series Serial ATA RAID Controller Installation Guide
www.3ware.com 115
116 3ware 9000 Series Serial ATA RAID Controller Installation Guide
www.3ware.com 117
K T
kernel strings for RedHat 71 Technical Support 112
tools and equipment 10
L tools required for installation 10
LED Status Connector Troubleshooting 97
details about connecting 13 troubleshooting 97
whether to use 9
LED status connectors 9 U
Linux, installing driver under 65 units
configuring 27, 40
M initializing 49
mirrored array 28
module names for RedHat W
Linux 71 Warranty 112
motherboard boot sequence 51 Windows, installing driver
multilane cable, with individual under 53
SATA connectors 20 write cache, enabling or
multilane cable, with multilane disabling 43
connectors on each end (847) 20
118 3ware 9000 Series Serial ATA RAID Controller Installation Guide