Openstack Super Bootcamp PDF
Openstack Super Bootcamp PDF
Openstack Super Bootcamp PDF
Super
Bootcamp
Mirantis, 2012
Agenda
OpenStack Essex architecture recap
Folsom architecture overview
Quantum vs Essex's networking model
Initial State
Tenant is created, provisioning
quota is available, user has an
access to Horizon/CLI
Horizon
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
nova-compute fetches
information about VM from DB,
creates a command to
hypervisor and delegates VM
rendering to hypervisor.
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
Initial State
Tenant is created, provisioning
quota is available, user has an
access to Horizon/CLI
Horizon
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
CLI
Keystone
Glance
endpoint
glance-api
glance-registry
scheduler
nova: Compute
nova-db
nova-compute
nova-network
nova-volume
que
ue
Shared Storage
hypervisor
Swift
proxy-server
object
store
Limitations?
Nova is "overloaded" to do 3 things
Compute
Networking
Block Storage
Keystone
keystone
server
nova
compute node
controller
queu
e
nova-api
nova:
novaCompute
Hypervisor
compute
Quantum
V
M
scheduler
keystone
-db
quantum
server
nova-db
Network
quantum
-db
Cinder
endpoint
scheduler
cinder-db
queu
e
cinder-vol
block
storage
node
glance-api
storage
glance-registry
Glance
quantum
plugin
Swift
proxy-server
glance
db
object
store
Keystone
keystone
server
nova
compute node
controller
queu
e
nova-api
nova:
novaCompute
Hypervisor
compute
Quantum
V
M
scheduler
keystone
-db
quantum
server
nova-db
Network
quantum
-db
Cinder
endpoint
scheduler
cinder-db
queu
e
cinder-vol
block
storage
node
glance-api
storage
glance-registry
Glance
quantum
plugin
Swift
proxy-server
glance
db
object
store
router
router: 10.0.0.1
(def. gateway for VMs)
VM
10.0.0.2
nova-compute host
VM
10.0.0.4
VM
10.0.0.3
VM
10.0.0.5
linux
bridge
linux
bridge
eth0
eth0
switch
nova-compute host
Single-host networking
eth1
public ip
routing/NAT
eth0
nova-network host
eth0 IP:10.0.0.1
(def. gateway for VMs)
VM
10.0.0.3
VM
10.0.0.4
VM
10.0.0.2
nova-compute host
VM
10.0.0.5
linux
bridge
linux
bridge
eth0
eth0
switch
nova-compute host
No route to host
routing/NAT
eth0
VM
10.0.0.2
nova-compute host
VM
10.0.0.4
VM
10.0.0.3
VM
10.0.0.5
linux
bridge
linux
bridge
eth0
eth0
switch
nova-compute host
Multi-host networking
Move routing from the central server
to each compute node independently to
prevent SPOF.
eth1
routing/NAT
eth0
public ip
public ip
eth1
eth1
routing/NAT
routing/NAT
VM
10.0.0.2
nova-compute &
nova-network host
VM
10.0.0.4
VM
10.0.0.3
VM
10.0.0.5
linux
bridge
linux
bridge
eth0
eth0
switch
nova-compute &
nova-network host
Multi-host networking
Compute servers maintain Internet access independent from each other. Each of
them runs nova-network & nova-compute components.
public ip
public ip
eth1
eth1
routing/NAT
routing/NAT
VM
10.0.0.2
10.0.0.1(gw)
VM
10.0.0.4
VM
10.0.0.3
linux
bridge
10.0.0.6(gw)
linux
bridge
eth0
eth0
nova-compute &
nova-network host
VM
10.0.0.5
switch
nova-compute &
nova-network host
Routing:
Kernel routing tables are checked to decide if the packet
should be NAT-ed to eth1 or sent via eth0
Routing:
Kernel routing tables are checked to decide if the packet
should be NAT-ed to eth1 or sent via eth0
IP address management:
Network manager
Determines network layout of the cloud
infrastructure
Capabilities of network managers
FlatManager
Features:
eth1
VM
VM
/etc/network/interfaces:
"address 10.0.0.2
gateway 10.0.0.1"
linux
bridge
linux
bridge
eth0
nova-compute &
nova-network
eth0
nova-compute &
nova-network
10.0.0.1(gw)
FlatDHCPManager
Features:
VM
eth1
VM
obtain dhcp static lease:
ip: 10.0.0.2
gw: 10.0.0.1
dnsmasq
eth0
nova-compute &
nova-network
linux
bridge 10.0.0.1(gw)
eth0
nova-compute
& nova-network
eth1
VM
ip: 10.0.0.6
gw: 10.0.0.3
linux
bridge
10.0.0.3(gw)
VM
ip: 10.0.0.5
gw: 10.0.0.1
VM
ip: 10.0.0.2
gw: 10.0.0.1
eth0
linux
bridge
eth0
nova-compute &
nova-network
nova-compute
& nova-network
switch
10.0.0.1(gw)
VlanManager
Features:
eth1
VM_net2
VM_net1
VM_net1
VM_net2
br100
br200
dnsmasq_net1
eth0
nova-compute &
nova-network
dnsmasq_net2
eth0.100
nova-compute
& nova-network
eth0 eth0.200
eth1
VM_net1
VM_net2
VM_net2
VM_net1
br100
br200
br100
br200
eth0.100
nova-compute
& nova-network
eth0.200
eth0 eth0.200
nova-compute
& nova-network
tagged traffic
eth0
eth0.100
FlatManager
Limitations
FlatDHCPManager
VlanManager
Inter-tenant traffic
Compute node's routing table
consulted to route traffic
between tenants' networks
(based on IPs of the linux
bridges)
public ip
eth1
VM_net1
VM_net2
routing
10.100.0.1
eth0.100
nova-compute
& nova-network
br200
10.200.0.1
eth0 eth0.200
Accessing internet
eth1 address is set as the
public ip
eth1
VM_net1
VM_net2
routing/NAT
br100
10.100.0.1
eth0.100
nova-compute
& nova-network
br200
10.200.0.1
eth0 eth0.200
Floating IPs:
Floating IPs
User associates a floating
IP with VM:
floating IP is added as a
secondary IP address on
compute node's eth1 (public
IF)
DNAT rule is set to redirect
floating IP -> fixed IP
(10.0.0.2)
floating IP
added as a
secondary IP
on eth1
vm_float_ ip: 92.93.94.95
public ip
eth1
floating IP DNAT:
-d 92.93.94.95/32 -j DNAT -to-destination 10.0.0.2
VM
10.0.0.2
VM
10.0.0.3
linux
bridge
eth0
nova-compute &
nova-network host
Limitations
Networking management is available only for
admin
Implementation is coupled with networking
abstractions
E
C
I
Presents a logical API and a corresponding
plug-in
V
R
architecture that separates the description
of network
E
S
connectivity from its implementation.
N
O
I
T
Offers an API that isCextensible
and evolves
independently R
ofA
the compute API
T
S
B
Provides
A a platform for integrating advanced networking
solutions
N
A
Quantum Overview
quantum abstracts
quantum architecture
quantum Open vSwitch plugin
quantum L3 agents
external net
172.24.0.0/16
NAT
NAT
router1
10.0.0.1
GW
router2
192.168.0.1
10.23.0.1
GW
vm
vm
vm
vm
10.0.0.2
192.168.0.2
10.23.0.2
10.23.0.3
local nets
QUANTUM - abstracts
DC net
vm
compute node
vm
vm
vm
vm
remote
DC
tunnel
DC DMZ
compute node
(another DC)
internet
vm
vm
vm
QUANTUM - abstracts
virtual L2 networks
IP pools & DHCP
virtual ports & routers
"local" & "external" networks
Quantum - architecture
source: http://openvswitch.org
source: http://openvswitch.org
source: http://openvswitch.org
source: http://openvswitch.org
Linux Bridge
OpenVSwitch
Nicira NVP
Cisco (UCS Blade + Nexus)
Ryu OpenFlow controller
NEC ProgrammableFlow Controller
single integration
bridge "br-int"
compute node
vm
vm
LV_1
LV_2
br-int
ovs
daemon
breth0
q-agt
eth0
compute node
openvswitch
daemon accepts
calls from Quantum
agent & reconfigures
network
Quantum agent
accepts calls from
the central quantum
server via plugin
vm
vm
LV_1
LV_2
br-int
ovs
daemon
breth0
quantum server
qplugi
n
q-agt
eth0
quantum server
qplugi
n
compute node
vm
vm
LV_1
LV_2
br-int
ovs
daemon
breth0
q-agt
eth0
Single L2 bcast
domain
openvswitch
FLAT:
br-int
br-eth0
eth0
802.1Q VLANs
openvswitch
VLAN:
br-int
br-eth0
eth0
openvswitch
GRE:
VM
br-int
br-tun
eth0
2^12 = 4096
vm
compute node
vm
DC net
vm
vm
vm
remote
DC
tunnel
DC DMZ
compute node
(another DC)
internet
vm
vm
vm
dedicated per-NIC
bridge
vlan range:
401-800
tunnel ID
range:
50-600
vm
vm
vm
vm
vm
vm
vm
vm
10.1.0.0/24
vm
vm
10.0.0.0/24
vm
vm
vm
vm
10.2.0.0/24
vm
We need IP addresses
vm
vm
10.1.0.0/24
vm
vm
10.0.0.0/24
vm
vm
vm
vm
10.2.0.0/24
vm
We need routers
vm
vm
10.1.0.0/24
vm
vm
10.0.0.0/24
vm
vm
vm
vm
10.2.0.0/24
vm
We need external
access/NAT
vm
vm
Quantum vs L3 services
dhcp-agent &
quantum db
for IP address
mgmt
10.1.0.0/24
vm
vm
10.0.0.0/24
vm
vm
vm
vm
10.2.0.0/24
vm
vm
l3-agent for
routing & NAT
vm
IPAM
DHCP
dhcp-agent: aims to manage different dhcp backends to
provide dhcp services to openstack instances.
Routing
l3-agent:
NAT
l3-agent: creates NAT-ed connections to "external" networks
quantum server
dhcp host
gateway host
dnsmasq
routing/NAT
dhcp-agent
l3-agent
flat
vlan
CISCO
gre
nexus
UCS
NICIRA
RYU
NEC
OTHER?
NVP
Open
Flow/O
VS
Progra
mmabl
eFlow
???
QUANTUM
dnsma
sq
DHCP
AGENT
NAT
router
L3
AGENT
iptable
s
???
FIREWALL
AGENT
HApro
xy
F5
L-B
AGENT
???
tunnel IDs
disjoint concepts of "network" and "IP pool"
tenant networks connected with one another by
"virtual routers"
internal vs external networks
Quantum vs nova-network
NOVA-NETWORK
QUANTUM
multi-host
Yes
No
VLAN networking
Yes
Yes
Flat(DHCP)
networking
Yes
Yes
Tunneling (GRE)
No
Yes
many bridges
No
Yes
SDN
No
Yes
IPAM
Yes
Yes
dashboard support
No
Limited - no floating
IPs
Yes
security groups
Questions?
kishanov@mirantis.com