Data Centers and Cloud Computing

• Data Centers

• Virtualization

• Cloud Computing

Computer Science Lecture 24, page 1

Data Centers
• Large server and storage farms
– 1000s of servers
– Many TBs or PBs of data
• Used by
– Enterprises for server applications
– Internet companies
• Some of the biggest DCs are owned by Google, Facebook, etc
• Used for
– Data processing
– Web sites
– Business apps

Computer Science Lecture 24, page 2

 Inside a Data Center

• Giant hardware warehouse

• Racks of servers
• Storage arrays
• Network switches

• Cooling infrastructure
• Power converters
• Backup generators

Computer Science Lecture 24, page 3

MGHPCC Data Center

• Data center in Holyoke

Computer Science Lecture 24, page 4
 Modular Data Centers
• ...or use shipping containers

• Each container filled with

thousands of servers

• Can easily add new containers

– “Plug and play”
– Just add electricity

• Allows data center to be easily

expanded
• Pre-assembled, cheaper
Computer Science Lecture 24, page 5

Virtualization

• Virtualization: extend or replace an existing interface to

mimic the behavior of another system.
– Introduced in 1970s: run legacy software on newer mainframe
hardware
• Handle platform diversity by running apps in virtual
machines (VMs)
– Portability and flexibility
Computer Science Lecture 24, page 6
 Types of Interfaces

• Different types of interfaces

– Assembly instructions
– System calls
– APIs
• Depending on what is replaced/mimicked, we obtain
different forms of virtualization
• Emulation (Bochs), OS level, application level (Java,
Rosetta, Wine)
Computer Science Lecture 24, page 7

Types of OS-level Virtualization

• Type 1: hypervisor runs on “bare metal”

• Type 2: hypervisor runs on a host OS
– Guest OS runs inside hypervisor
• Both VM types act like real hardware

Computer Science Lecture 24, page 8

 Server Virtualization
• Allows a server to be “sliced” into Virtual Machines (VMs)
• VM has own OS/applications
• Rapidly adjust resource allocation

VM 1 VM 2
Windows Linux

Windows Linux Virtualization Layer

Computer Science Lecture 24, page 9

Example: Virtualized Database

Servers
• Conventional: one physical server, one database server
• Data center: multiple physical servers, multiple database
servers per (virtualized) physical server

Data Center
Workload

Workload 1

Workload 2

Workload 1

Workload 2

Workload 1

Workload 2

Tenant 1 Tenant 2 Tenant 1 Tenant 2 Tenant 1 Tenant 2

Database Database Database Database Database Database

Database Server
Tenant 3 Tenant 4 Tenant 3 Tenant 4 Tenant 3 Tenant 4
Database Database Database Database Database Database

Physical Server
Workload 3

Workload 4

Workload 3

Workload 4

Workload 3

Workload 4

Server 1 Server 2 Server 3

Computer Science Lecture 24, page 10

Virtualization in Data Centers
• Virtual Servers
– Consolidate servers
– Faster deployment
– Easier maintenance

• Virtual Desktops
– Host employee desktops in VMs
– Remote access with thin clients
– Desktop is available anywhere Work
– Easier to manage and maintain

Home
Computer Science Lecture 24, page 11

Data Center Challenges

• Resource management
– How to efficiently use server and storage resources?
– Many apps have variable, unpredictable workloads
– Want high performance and low cost
– Automated resource management
– Performance profiling and prediction

• Energy efficiency
– Servers consume huge amounts of energy
– Want to be “green”
– Want to save money
Computer Science Lecture 24, page 12
 Data Center Costs

• Running a data center is expensive

http://perspectives.mvdirona.com/2008/11/28/
CostOfPowerInLargeScaleDataCenters.aspx

Computer Science Lecture 24, page 13

Economy of Scale
• Larger data centers can be cheaper to buy and run than
smaller ones
– Lower prices for buying equipment in bulk
– Cheaper energy rates

• Automation allows small number of sys admins to manage

thousands of servers

• General trend is towards larger mega data centers

– 100,000s of servers
• Has helped grow the popularity of cloud computing
Computer Science Lecture 24, page 14
 What is the cloud?

Remotely available
Pay-as-you-go
High scalability
Shared infrastructure

Azure

Computer Science Lecture 24, page 15

The Cloud Stack

Software as a Service
Hosted applications
Managed by provider
Office apps, CRM
Platform as a Service
Platform to let you run
Azure
your own apps
Software platforms Provider handles scalability
Infrastructure as a Service
Raw infrastructure
Can do whatever you
Servers & storage want with it
Computer Science Lecture 24, page 16
 IaaS: Amazon EC2
• Rents servers and storage to customers
– Uses virtualization to share each server for multiple customers
– Economy of scale lowers prices
– Can create VM with push of a button

Smallest Medium Largest

VCPUs 1 5 33.5
RAM 613MB 1.7GB 68.4GB
Price $0.02/hr $0.17/hr $2.10/hr

Storage $0.10/GB per month

Bandwidth $0.10 per GB

18

Computer Science Lecture 24, page 17

PaaS: Google App Engine

• Provides highly scalable execution platform
– Must write application to meet App Engine API
– App Engine will autoscale your application
– Strict requirements on application state
• “Stateless” applications much easier to scale

• Not based on virtualization

– Multiple users’ threads running in same OS
– Allows Google to quickly increase number of “worker threads”
running each client’s application

• Simple scalability, but limited control

– Only supports Java and Python

Computer Science Lecture 24, page 18

 Public or Private
• Not all enterprises are comfortable with using public cloud
services
– Don’t want to share CPU cycles or disks with competitors
– Privacy and regulatory concerns

• Private Cloud
– Use cloud computing concepts in a private data center
• Automate VM management and deployment
• Provides same convenience as public cloud
• May have higher cost

• Hybrid Model
– Move resources between private and public depending on load
Computer Science Lecture 24, page 19

Programming Models

• Client/Server
– Web servers, databases, CDNs, etc

• Batch processing
– Business processing apps, payroll, etc

• MapReduce
– Data intensive computing
– Scalability concepts built into programming model

Computer Science Lecture 24, page 20

 Cloud Challenges

• Privacy / Security
– How to guarantee isolation between client resources?

• Extreme Scalability
– How to efficiently manage 1,000,000 servers?

• Programming models
– How to effectively use 1,000,000 servers?

Computer Science Lecture 24, page 21

Challenge: Memory Efficiency

• May be running multiple virtual machines on a single server
that have a lot of data in common

• For example, ten copies of Linux in separate VMs

– Many customers running an Apache webserver

• Can we eliminate duplicated memory?

– Fit more virtual machines with the same physical resources

Computer Science Lecture 24, page 22

 Content Based Page Sharing
! Approach: eliminate identical
pages of memory across
multiple VMs Physical
RAM
Hypervisor

A
FREE
A D
B
! Virtual VM pages mapped to D VM 2
physical pages FREE
B
Page Table

A
A B
! Hypervisor detects duplicates B C
VM 1
C Page Table

! Replaced with copy-on-write

references

Computer Science Lecture 24, page 23

Challenge: Dynamic Workloads

• Server workloads change over time

• Time of day variations

• Flash crowds

• Example: social media on election day

Number of Users

! Workload changes may

require more resources!

Time

Computer Science Lecture 24, page 24

 Virtual Machine Migration
• Approach: move (migrate) a virtual machine from one
physical server to another (with more available resources)

Workload 1

Workload 2

Workload 5

Workload 6
Server A

Server B
Tenant 1 Tenant 2 Tenant 5 Tenant 6
Database Database Database Database

Tenant 3 Tenant 4 Tenant 4

Database Database Database

Workload to
Workload 3

Workload 4

Workload 4
database 4
increases!

• Nice, but incurs downtime!

Computer Science Lecture 24, page 25

Live Migration
• Migrate without stopping Server Clients

! (1) Copy pages of memory

• Continue handling workload A
F B G
C D H
E
Source Server
! (2) Update changed pages
• Multiple rounds

I
! (3) Switch workload to target
• Brief downtime Target Server

Computer Science Lecture 24, page 26

 Summary
• Many services moving to the cloud
– Remotely available
– Pay-as-you-go
– High scalability

• Operating in large, shared data centers

• Data centers use virtualization to increase utilization and

decrease costs

• Many challenges in resource management using virtualized

data centers

Computer Science Lecture 24, page 27