June 2012 @Dubai

IBM Power Academy

IBM PowerVM
network virtualization
Luca Comparini
STG Lab Services Europe
IBM FR

June,13th 2012
@IBM Dubai
 Objective of the session: understand this chart

VIOS 1 VIOS 2 Client 1 Client 2

en3 en3
Primary (if) (if) Backup

ent3 ent2 ent2 ent3 en0 en1 en0 en1

(SEA) (Vir) (Vir) (SEA) (if) (if) (if) (if)

ent0 ent1 ent1 ent0 ent0 ent1 ent0 ent1

(Phy) (Vir) (Vir) (Phy) (Vir) (Vir) (Vir) (Vir)

VID PVID PVID=99 VID PVID PVID PVID PVID PVID

Hypervisor

2 1 2 1 1 2 1 2

VLAN 1
VLAN 2

Untagged Untagged
VLAN ID 2 VLAN ID 2
Untagged
VLAN ID 2

Ethernet Switch Ethernet Switch

Active
Passive
2
 Agenda

 PowerVM Virtual Ethernet Switch

 Shared Ethernet Adapter

 Dual VIOS configuration

3
 Introduction on VIOS – concepts of Virtual I/O Server - Client

Power Hypervisor

4
 Bridging, Routing and Switching

Bridge Router

Bridge Router

Ethernet TCP/IP Ethernet TCP/IP

Data Data
Header Header Header Header

Switch (Multi-port Bridge) VLAN Switch (Smart Multi-port Bridge)

Switch Switch

Ethernet TCP/IP Ethernet V TCP/IP

Data Data
Header Header Header L Header

5
 Power VIO Server Switch concepts

Traditional Switches Virtual I/O Server Switch

Client Client LPAR
en0 en0
(if) (if)

PowerVM Server
ent0 ent0
(Virt)

Switch VIOS ent1

Port
(Virt)

ent2
Logic
(SEA)

ent0
Port
(phy)

To other device or switch To other switch

6
 Etherchannel and link aggregation

• Ethernet Link Aggregation

– Group Ethernet adapters together to act as a Power Server
single pseudo network adapter
– One common hardware (MAC) and IP address
– Adapters must be the same speed and full
duplex
– All adapters (except backup) must be connected NIC NIC NIC
to the same network switch
– PowerVM client - only one active virtual Ethernet
adapter is supported
Ethernet Switch
• Sometimes called NIC teaming, port teaming, or NIC
bonding

• Link Aggregation Benefits

– Greater reliability
– Greater total network bandwidth

7
 Etherchannel and link aggregation

• Standard Algorithm
– Standard algorithm uses the last byte of the
destination IP to determine the outbound NIC.
– All traffic to the same host goes out the same
NIC.

• Hash Mode (src_dst_port)

– Hashes the source and designation TCP or
UDP port values to determine the outbound
NIC.
– Probably best initial choice for mode.

• Round Robin (EtherChannel only)

– Traffic spread evenly across all adapters.
– Ideal when there are no intervening switches
due to increased risk of out-of-order packets.

• Incoming Traffic from Switch

– Governed by the algorithm of the switch.

8
 Shared Ethernet Adapter

VIOS provides virtual networking to client partitions, including IBM i, by bridging

a physical Ethernet adapter and one or more virtual Ethernet adapters.

The virtualization object that provides this Ethernet bridge is called a Shared
Ethernet Adapter (SEA).

The SEA forwards network packets from any client partitions on a VLAN to the
physical LAN through the physical Ethernet adapter. Because the SEA creates
a Layer-2 bridge, the original MAC address of the virtual Ethernet adapter in
IBM i is used on the physical LAN.

The CMNxx communications port that represents the virtual Ethernet adapter in
IBM i is configured with an externally routable IP address and a standard
network configuration is used. The physical adapter bridge by the SEA can be
any network adapter supported by VIOS, including Integrated Virtual Ethernet
(IVE) ports, also known as Host Ethernet Adapter (HEA) ports.

9
 Shared Ethernet Adapter

VIOS 1 Client 1 Client 2

In most cases, it is
unnecessary to create more en8 en7
than one Virtual Ethernet (if) (if)
adapter for a SEA.
ent6 ent8 2 1 ent7 en0 en1 en0 en1
Think simple! (LA) (SEA) (SEA) (if) (if) (if) (if)

Multiple VLANs can be ent2 ent1 ent4 ent5 ent0 ent3 ent0 ent1 ent0 ent1
added to a single SEA. (Phy) (Phy) (Vir) (Vir) (Phy) (Vir) (Vir) (Vir) (Vir) (Vir)

VID PVID VID 300 PVID 100 PVID PVID PVID

200 2 ( PVID 3 ) 300 2 200

PVID 2 PVID 100

VID 200,300

1 mkvdev -sea ent0 -vadapter ent3 -default ent3 -defaultid 100

2 mkvdev -sea ent6 -vadapter ent4,ent5 -default ent4 -defaultid 2

Physical Ethernet Virtual Ethernet Virtual Ethernet that Default

adapter or link adapters in the VIOS will contain the VLAN
aggregation device that will be used with default VLAN
this SEA

10
 Virtual I/O Network terms

VLAN Device
Shared Ethernet Adapter
Virtual I/O Server Client 1 Client 2
(Acts as a layer 2 bridge
to the outside world )

en4 en1
Link Aggregation Adapter (if)
(if)
(Used to combine
physical Ethernet
adapters) ent2 ent4 ent1 en0 en0 en1
(LA) (SEA) VLAN (if) (if) (if) Interface
Virtual
ent1 ent0 ent3 ent0 ent0 ent1 Ethernet
Physical Ethernet (Phy) (Phy) (Vir) (Vir) (Vir) (Vir)
Adapter Adapter
(Will hurt if it falls on your VID PVID PVID VID PVID PVID
Hypervisor

foot) 2 1 1 2 1 2

Virtual LAN (VLAN)

(Mechanism to allow VLAN 1
multiple networks to
share the same VLAN 2
physical link)

IEEE 802.3ad Link Aggregation

or Cisco EtherChannel VLAN ID (VID) Port VLAN ID (PVID)
(VLAN tag information. (The default VID for a port.
Packets are tagged.) Packets are delivered untagged.)
Ethernet Switch

11
How to set it up - IVM

1. Create a SEA Adapter (L2 Bridge)

How to set it up - IVM

2. Associate SEA Adapter to Virtual I/O Adapter

How to set it up - IVM

3. Assign Virtual I/O Adapter to client LPAR

(while creating partition, or just after)
How to set it up - HMC

VIOS 1 Client 1 Client 2

en4
(if)

ent3 ent4 en0 en1 en0 en1

(LA) (SEA) (if) (if) (if) (if)

ent1 ent0 ent2 ent0 ent1 ent0 ent1

(Phy) (Phy) (Vir) (Vir) (Vir) (Vir) (Vir)

VID PVID PVID PVID PVID PVID

100 1 1 100 1 100

Untagged (PVID 1)
VID 100

1. Create Virtual Ethernet adapters for the clients

2. Create the Virtual Ethernet adapter for the VIOS
3. Create a Link Aggregation device in the VIOS (if required)
4. Create the Shared Ethernet Adapter (SEA) in the VIOS
How to set it up – HMC – Create Virtual Ethernet Adapter PVID 1

2
1

Do this for both client 1 and client 2 LPARs

How to set it up – HMC – Create Virtual Ethernet Adapter PVID 100

3
100

3
100

Do this for both client 1 and client 2 LPARs

How to set it up – HMC – VIOS Virtual Ethernet PVID 1, VID 100

100


How to set it up – HMC – VIOS link aggregation

VIOS 1 Client 1 Client 2

en4
(if)

Link ent3 ent4 en0 en1 en0 en1

Aggregation (LA) (SEA) (if) (if) (if) (if)
Device
ent1 ent0 ent2 ent0 ent1 ent0 ent1
(Phy) (Phy) (Vir) (Vir) (Vir) (Vir) (Vir)

VID PVID PVID PVID PVID PVID

100 1 1 100 1 100

Untagged (PVID 1)
VID 100

• Create the Link Aggregation device on the VIOS

– $ mkvdev –lnagg ent0,ent1
– ent3 Available
– en3
– et3
How to set it up – HMC – SEA

VIOS 1 Client 1 Client 2

en4
(if)

ent3 ent4 en0 en1 en0 en1

(LA) (SEA) (if) (if) (if) (if)

ent1 ent0 ent2 ent0 ent1 ent0 ent1

Shared (Phy) (Phy) (Vir) (Vir) (Vir) (Vir) (Vir)

Ethernet VID PVID PVID PVID PVID PVID

100 1 1 100 1 100
Adapter

Untagged (PVID 1)
VID 100

• Create the Shared Ethernet Adapter (SEA)

$ mkvdev -sea ent3 -vadapter ent2 -default ent2 -defaultid 1
ent4 Available
en4
et4
High Availability VIOS options

• Network Interface Backup • Shared Ethernet Adapter Failover

– Must be set up in each client. – Set up in the VIOS’s only
– Needs to ping outside host from – Optional ping is done in VIOS on
each client to initiate NIB failover. behalf of all clients
– Load share clients across SEAs but – Cannot load-share clients between
LPAR to LPAR communications will the primary and backup SEA
happen through external switches – VLAN-tagged traffic is supported
– VLAN-tagged traffic is not – Supported on all AIX, IBM i, Linux
supported.
– AIX only.

AIX Client VIO Client

LA
NIB NIC
NIC NIC

VIOS VIOS VIOS VIOS

SEA SEA SEA SEA

NIC NIC NIC NIC

High Availability VIOS options: NIB

• Complexity
– Requires specialized setup on client (NIB)
– Needs to ping outside host from the client to initiate
NIB failover

• Resilience
– Protects against single VIOS, switch port, switch,
and Ethernet adapter failures

• Throughput / Scalability
– Allows load-sharing between VIOS’s

• Notes
– NIB does not support tagged VLANs on physical
LAN
– Must use external switches not hubs
– Only supported on AIX
High Availability VIOS options: NIB

VIOS 1 VIOS 2 AIX 1 AIX 2

en2 en2 en2 en2

(if) (if) (if) (if)

ent2 ent2 ent2 ent2

(SEA) (SEA) (LA) (LA)
NIB NIB
ent0 ent1 ent1 ent0 ent0 ent1 ent0 ent1
(Phy) (Vir) (Vir) (Phy) (Vir) (Vir) (Vir) (Vir)

PVID PVID PVID PVID PVID PVID

Hypervisor

1 2 1 2 2 1

VLAN 1
VLAN 2

Untagged Untagged Note: If you split the active

client interfaces across VIOS’s,
those LPARs will talk to each
other through the external
switches no the Hypervisor.
Untagged
Ethernet Switch Ethernet Switch

Active
Passive
High Availability VIOS options: NIB

VIOS 1 VIOS 2 AIX 1 AIX 2

en2 en2 en2 en2

(if) (if) (if) (if)

ent2 ent2 ent2 ent2

(SEA) (SEA) (LA) (LA)
NIB NIB
ent0 ent1 ent1 ent0 ent0 ent1 ent0 ent1
(Phy) (Vir) (Vir) (Phy) (Vir) (Vir) (Vir) (Vir)

PVID PVID PVID PVID PVID PVID

Hypervisor

1 2 1 2 2 1

VLAN 1
VLAN 2

Untagged Untagged Note: If you split the active

client interfaces across VIOS’s,
those LPARs will talk to each
other through the external
switches no the Hypervisor.
Untagged
Ethernet Switch Ethernet Switch

Active
Passive
High Availability VIOS options: SEA Failover

• Complexity
– Specialized setup confined to VIOS

• Resilience
– Protection against single VIOS, switch port,
switch, and Ethernet adapter failure

• Throughput / Scalability
– Cannot do load-sharing between primary and backup
SEA
– SEA failure initiated by:
• Backup SEA detects the active SEA has failed.
• Active SEA detects a loss of the physical link
• Manual failover by putting SEA in standby mode
• Active SEA cannot ping a given IP address.

• Notes
– Can be used on AIX, IBM i, Linux
– Outside traffic may be tagged
High Availability VIOS options: SEA Failover

VIOS 1 VIOS 2 Client 1 Client 2

en3 en3
Primary (if) (if) Backup

ent3 ent2 ent2 ent3 en0 en0

(SEA) (Vir) (Vir) (SEA) (if) (if)

ent0 ent1 ent1 ent0 ent0 ent0

(Phy) (Vir) (Vir) (Phy) (Vir) (Vir)

PVID PVID=99 PVID PVID PVID

Hypervisor

1 1 1 1

VLAN 1

Untagged Untagged

Untagged
Ethernet Switch Ethernet Switch

Active
Passive
High Availability VIOS options: SEA Failover with LA

VIOS 1 VIOS 2 Client 1 Client 2

en5 en5
Primary (if) (if) Backup

ent4 ent5 ent3 ent3 ent5 ent4 en0 en0

(LA) (SEA) (Vir) (Vir) (SEA) (LA) (if) (if)

ent1 ent0 ent2 ent2 ent0 ent1 ent0 ent0

(Phy) (Phy) (Vir) (Vir) (Phy) (Phy) (Vir) (Vir)

PVID PVID=99 PVID PVID PVID

Hypervisor

1 1 1 1

VLAN 1

Untagged Untagged

Ethernet Switch Untagged Ethernet Switch

Active
Passive
High Availability VIOS options: SEA Failover with LA, with VLAN

VIOS 1 VIOS 2 Client 1 Client 2

en3 en3
Primary (if) (if) Backup

ent3 ent2 ent2 ent3 en0 en1 en0 en1

(SEA) (Vir) (Vir) (SEA) (if) (if) (if) (if)

ent0 ent1 ent1 ent0 ent0 ent1 ent0 ent1

(Phy) (Vir) (Vir) (Phy) (Vir) (Vir) (Vir) (Vir)

VID PVID PVID=99 VID PVID PVID PVID PVID PVID

Hypervisor

2 1 2 1 1 2 1 2

VLAN 1
VLAN 2

Untagged Untagged
VLAN ID 2 VLAN ID 2
Untagged
VLAN ID 2

Ethernet Switch Ethernet Switch

Active
Passive
 CREDITS to

Questions? John Banchy

System Architect
IBM US

Luca Comparini
STG Lab Services Europe
IBM FR

THANKS