WS2012 FS SMBdirect v9
WS2012 FS SMBdirect v9
WS2012 FS SMBdirect v9
November 2012
Kent R. Swalin
Quang Le
IBM Systems and Technology Group
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 2
Executive Overview
File Servers are well known for providing common storage repositories for shared documents,
images, and user file storage. The Server Message Block (SMB) protocol is the file sharing
protocol that is used by default on Windows file servers. Now with SMB Direct in Windows
Server 2012, a Scale-Out File Server using IBM System x Servers, IBM Storage, IBM System
Networking and Chelsio T4 adapters can provide high performance file shares that are
continuously available for applications such as Microsoft SQL Server 2012. SMB Direct which
is SMB over Remote Direct Memory Access (RDMA), was introduced in Windows Server 2012.
SMB Direct has the typical benefits of SMB file storage: it is easy to provision, manage, and
migrate.
File server administrators can provide scale-out file shares and continuously available file
services to server applications and respond to increased demands quickly by simply adding more
servers to the cluster or bringing more storage online. All of this can be done in a production
environment and it is transparent to the server application.
Introduction
This document describes a Scale-Out solution that uses a two-node file server cluster and two
servers running Microsoft SQL Server 2012 on Windows Server 2012. The application servers in
this solution can access file shares using any of the cluster nodes. This is achieved by leveraging
new Windows Server Failover Clustering features and new capabilities in the new version of
Windows file server protocol - SMB 3.0. SMB Direct supports the use of network adapters that
have remote direct memory access (RDMA) capability. SMB Direct functionality requires that the
SMB client and SMB server support SMB 3.0.
The new SMB Direct storage protocol in Windows Server 2012 provides for extremely fast
responses to network requests, and as a result, makes remote file storage respond similarly to
directly attached block storage. SMB Direct also offloads CPU cycles when transferring data over
the network, which leaves more cycles available to server applications.
Key benefits provided by Scale-Out File Server in Windows Server 2012 include:
Active-Active file shares: All cluster nodes can accept and serve SMB client requests.
By making the file share content accessible through all cluster nodes simultaneously,
SMB 3.0 clusters and clients cooperate to provide transparent failover to alternative
cluster nodes during planned maintenance and unplanned failures without service
interruption.
Increased bandwidth: The maximum share bandwidth is the total bandwidth of all file
server cluster nodes. Unlike previous versions of Windows Server, the total bandwidth is
no longer constrained to the bandwidth of a single cluster node, but rather the capability
of the storage system. Bandwidth can be increased by adding nodes to the Scale-Out
cluster.
CHKDSK with zero downtime: CHKDSK in Windows Server 2012 is significantly
enhanced to dramatically shorten the time a file system is offline for repair. Clustered
shared volumes (CSVs) in Windows Server 2012 take this one step further and eliminate
the offline phase. A CSV File System (CSVFS) can perform CHKDSK without impacting
applications with open handles on the file system.
Clustered Shared Volume cache: CSVs in Windows Server 2012 introduces support
for a read cache, which can significantly improve performance in certain scenarios.
Simpler management: With Scale-Out File Servers necessary CSVs and file shares
can be created and added. It is no longer necessary to create multiple clustered file
servers, each with separate cluster disks, and then develop placement policies to ensure
activity on each cluster node.
Copyright IBM Corporation, 2012
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 3
Microsoft SQL Server 2012 offers full support for storing databases on SMB 3.0 file shares.
Scale-Out file shares provide the ability to share the same folder from multiple nodes in a cluster.
Utilizing a Scale-Out File Server can reduce SQL Server solution complexity. No longer does an
implementation of SQL Server require large amounts of local storage or attaching a server to a
Storage Area Network, when there is a Scale-Out File Server available that allows a set of shares
to be accessed by Universal Naming Convention (UNC) paths.
Components
This Scale-Out File Server architecture is comprised of IBM System x servers, IBM storage, and
IBM System Networking running Microsofts Windows 2012 operating system. Each component
provides a key element to the overall solution.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 4
By virtualizing with Microsoft Hyper-V technology on IBM System x3550 M4 servers, businesses
can make the best use of server hardware investments by consolidating multiple server roles as
separate virtual machines running on a single physical machine. This results in fewer physical
servers to manage. Running the same hypervisor allows IT administrators to manage more
servers than without virtualization. Since applications are managed within virtual machines and
isolated from the actual hardware, this again simplifies the physical server, maximizing uptime.
Greater IT flexibility is also realized by the ability of virtualization to streamline the deployment
and upgrade processes in both physical and virtual machines.
In addition, clients can see a lower total cost of ownership (TCO) from reduced hardware
requirements by maximizing the resource utilization of each server purchased. This usually
results in a substantial reduction in the number of physical servers required to support operations,
thereby lowering capital acquisition, power consumption, and cooling costs. Businesses also
benefit from infrastructure optimization that simplifies and standardizes IT administration, thus
helping to control labor costs. Reduced downtime costs can be achieved by the ability to easily
move and balance workloads across different resources, improve operational agility, and provide
flexibility in managing maintenance schedules and responding to todays dynamic IT demands.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 5
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 6
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 7
B
SP
L/A
MB
A
MS
Stacking
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
10101
Reset
Mgmt
System x3650 M4
16
17
DS3524
FC
FC
IBM DS3500
8
16
17
DS3524
Cluster
Witness
LUN
Cluster Shared Volume LUNs for File
Server data and log files.
Variety of configuration options
available to maximize IOPs &
capacity. Up to 192 Disks Max.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 8
Figure 7 shows a view of the interaction between the SMB Direct client and the SMB Direct
server. In this scenario the SMB Direct clients are the two SQL Servers while the SMB Direct
server is the x3650 M4 cluster. The SMB Direct client and server roles are included in Windows
Server 2012.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 9
Figure 8 shows the two options when adding the File Server Role to the cluster. For this solution
the option for File Server for scale-out application data was chosen.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 10
Table 1 and table 2 detail the configuration of the cluster nodes and SQL Servers.
Total Physical
Memory
Operating System
Network
Fibre Channel
Total Physical
Memory
Operating System
Network
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 11
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 12
Performance Monitoring
Performance Monitor was used to capture the relevant disk performance data for an SMB Client
and an SMB Server and shown on Figures 10 and 11. These are the counters that should be
checked to monitor IO performance. These are examples only and should not be used as a
guide.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 13
Conclusion
Utilizing IBM hardware combined with Microsoft Windows Server 2012, it is possible to create a
high-performance, interoperable, and reliable application file server. With Enterprise-class
hardware, fault tolerance is achieved by configuring a robust collection of industry-leading
servers, storage systems and networking components from IBM. The SMB direct framework
promotes a standardized and highly manageable environment which helps satisfy even the most
challenging business critical SQL Server storage demands.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 14
X3550 M4
SQL Server
X3550 M4
SQL Server
SQLSVR1"
ct
ire
SM
BD
BD
SM
ire
ct
SQLSVR2
X3650 M4
File Server Cluster
isk Ac
cess
nel D
Chan
Fibre
k
el Dis
hann
C
e
r
Fib
s
Acces
x3650-FS
Cluster nodes: x3650m4-1 x3650m4-2
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 15
Storage Configuration
The following steps illustrate how quick and easy it is to configure DS3500 storage.
Create each of the arrays needed for the configuration. Assign an array name, and select the
number of disks and RAID type to be used (Figure 13).
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 16
Disks can now be created off each of the arrays (Figure 14)
A Host Group is a logical group that will contain the host servers that should all see the
same storage volumes; in this case add the hosts x3650m4-1 and x3650m4-2.
Create a Host Group to that will contain each of the host servers (Figure12 )
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 17
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 18
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 19
Open Server Manager
For both servers x3650m4-1 and x3650m4-2, select Add Roles and Features and check the
File and Storage Services box.
On the next screen, select Add Feature and check the Failover Clustering box.
Click Install
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 20
Click Next
Enter the names of the servers, one at a time followed by clicking Add.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 21
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 22
Click Next on the Create a Cluster Wizard
Add the names of the servers to be members of the cluster (x3650m4-1 and x3650m4-2). Click
Next
Enter the name of the cluster x3650m4Cluster, click Next
On the Confirmation screen, Click Next
Click Finish
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 23
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 24
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 25
From the PowerShell window, type Get-ClusterNetwork | Select * displays the contents in the
above screen.
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 26
Repeat the same steps for SQLSVR2, change Log2 for Log1
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 27
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 28
Use the default directories. Click Next
In the Installation rules page, click Next
In the Instance Configuration page, click Next
In the Disk Space Requirements page, click Next
In the Server Configuration page, enter pf.net\slqservice as the account name for the SQL
Server Database Engine and the SQL Server Agent, set bot services to start automatically. Click
Next
In the Database Engine Configuration page, click Add Current User
Click the Data Directories tab. Enter \\x3650-fs\data1 as the Data Root Directory
Change the two path for Log directories to use \\x3650-fs\log
Click Yes when prompted to confirm the correct permissions
Click Next on the Error reporting page
Click Next on the Installation Configuration Rules page
Click Install on the Ready to Install page
Installation of SQL will complete
Running Microsoft SQL Server 2012 on a Scale-Out File Server Cluster via SMB Direct Connection
Solution Utilizing IBM System x Servers
Page 29
http://ibm.com/systems/x
http://ibm.com/systems/bladecenter/resources/powerconfig.html
http://ibm.com/systems/x/hardware/configtools.html
IBM ServerProven Program
Technical Support
Other Technical Support Resources
http://ibm.com/systems/info/x86servers/serverproven/compat/us
Legal Information
http://ibm.com/server/support
http://ibm.com/systems/support
This publication may contain links to third party sites that are
not under the control of or maintained by IBM. Access to any
such third party site is at the user's own risk and IBM is not
responsible for the accuracy or reliability of any information,
data, opinions, advice or statements made on these sites. IBM
provides these links merely as a convenience and the
inclusion of such links does not imply an endorsement.
Information in this presentation concerning non-IBM products
was obtained from the suppliers of these products, published
announcement material or other publicly available sources.
IBM has not tested these products and cannot confirm the
accuracy of performance, compatibility or any other claims
related to non-IBM products. Questions on the capabilities of
non-IBM products should be addressed to the suppliers of
those products.
MB, GB and TB = 1,000,000, 1,000,000,000 and
1,000,000,000,000 bytes, respectively, when referring to
storage capacity. Accessible capacity is less; up to 3GB is
used in service partition. Actual storage capacity will vary
based upon many factors and may be less than stated.
Performance is in Internal Throughput Rate (ITR) ratio based
on measurements and projections using standard IBM
benchmarks in a controlled environment. The actual
throughput that any user will experience will depend on
considerations such as the amount of multiprogramming in the
users job stream, the I/O configuration, the storage
configuration and the workload processed. Therefore, no
assurance can be given that an individual user will achieve
throughput improvements equivalent to the performance ratios
stated here.
Maximum internal hard disk and memory capacities may
require the replacement of any standard hard drives and/or
memory and the population of all hard disk bays and memory
slots with the largest currently supported drives available.
When referring to variable speed CD-ROMs, CD-Rs, CD-RWs
and DVDs, actual playback speed will vary and is often less
than the maximum possible.
Intel Virtualization Technology may require a computer
system with an enabled Intel processor, BIOS, virtual
machine monitor (VMM). Functionality, performance or other
benefits will vary depending on hardware and software
configurations. Software applications may not be compatible
with all operating systems. For more information, visit
http://www.intel.com/go/virtualization