Vmware Architecture

Virtual Machines: Architectures, Implementations and Applications
HOTCHIPS 17 Tutorial 1, Part 1

J. E. Smith University of Wisconsin-Madison Rich Uhlig Intel Corporation August 14, 2005
INTRODUCTION
Introduction
Why are virtual machines interesting? They allow transcending of standard interfaces (which often seem to be an obstacle to innovation) They enable innovation in flexible, adaptive software & hardware, security, network computing (and others) They involve computer architecture in a pure sense Virtualization will be a key part of future computer systems A fourth major discipline? (with HW, System SW, Application SW)
August 2005
VM Intro (c) 2005, J. E. Smith
Abstraction
Software
Computer systems are built on levels of abstraction Higher level of abstraction hide details at lower levels Example: files are an abstraction of a disk
Application Programs
fileLibraries abstraction
Drivers Operating System Scheduler
file
Memory Manager
Execution Hardware Memory Translation
System Interconnect (bus) Controllers I/O devices and Networking
Controllers
Main Memory
Hardware
August 2005 VM Intro (c) 2005, J. E. Smith
4
Virtualization
Similar to abstraction
Except Details not necessarily hidden
virtualization file file
Construct Virtual Disks

As files on a larger disk Map state Implement functions
VMs: do the same thing with the whole machine
August 2005
The Machine
Different perspectives on what the Machine is: OS developer
Application Programs Libraries Operating System
Execution Hardware
Instruction Set Architecture

ISA Major division between hardware and software
System Interconnect (bus) I/O devices and Networking
Memory Translation
Main Memory
August 2005
The Machine
Different perspectives on what the Machine is: Compiler developer
Execution Hardware
Application Binary Interface

ABI User ISA + OS calls
Memory Translation
Main Memory
August 2005
The Machine
Different perspectives on what the Machine is: Application programmer
Execution Hardware
Application Program Interface

API User ISA + library calls
Memory Translation
Main Memory
August 2005
Virtual Machines
add Virtualizing Software to a Host platform and support Guest process or system on a Virtual Machine (VM) Example: System Virtual Machine
Applications Applications OS
Guest
OS Virtualizing Software Hardware "Machine"
VMM Host
Virtual Machine
August 2005
The Family of Virtual Machines
Lots of things are called virtual machines

IBM VM/370 Java VMware Some things not called virtual machines, are virtual machines IA-32 EL Dynamo Transmeta Crusoe
August 2005
10
Taking a Unified View

The subjects of virtual machines and emulators have been treated as entirely separate. they have much in common. Not only do the usual implementations have many shared characteristics, but this commonality extends to the theoretical concepts on which they are based
-- Efrem G. Wallach, 1973
August 2005
11
System Virtual Machines
Provide a system environment Constructed at ISA level Persistent Examples: IBM VM/360, VMware, Transmeta Crusoe
guest guest process process Guest OS VMM
guest process
guest guest guest process process process Guest OS2 VMM
HOST PLATFORM
virtual network communication
August 2005
12
Process VMs

Execute application binaries with an ISA different from hardware platform Couple at ABI level via Runtime System Not persistent
Guest Runtime
Application Process Virtualizing Software OS
Application Process
Host
Machine Hardware
Virtual Machine
August 2005
13
Process Virtual Machines
Guest processes may intermingle with host processes As a practical matter, guest and host OSes are often the same Same-ISA Dynamic optimizers are a special case Examples: IA-32 EL, FX!32, Dynamo
host process
guest process
runtim e
guest process
runtim e
guest process
runtim e
host process
create
HOST OS
file sharing
Disk
network communication
August 2005
14
HLL VMs

Java and CLI are recent examples Binary class files are distributed ISA is part of binary class format OS interaction via APIs (part of VM platform)
Java Binary Classes
Java VM Architecture
VM implementation Sparc Workstation VM implementation x86 PC VM implementation Apple Mac
August 2005
15
Co-Designed VMs

Perform both translation and optimization VM provides interface between standard ISA software and implementation ISA Primary goal is performance or power efficiency Use proprietary implementation ISA Transmeta Crusoe and IBM Daisy best-known examples
X86 Apps Windows
VLIW
August 2005
16
Composition
apps 1 OS 1 apps 2 OS 1
ISA 2
August 2005
17
Composition: Example
Java application JVM Linux x86 VMware Windows x86 Code Morphing Crusoe VLIW
August 2005
18
Summary (Taxonomy)
VM type (Process or System) Host/Guest ISA same or different
Process VMs System VMs
same ISA Multiprogrammed Systems HP Dynamo
different ISA IA-32 EL FX!32
same ISA IBM VM/370 VMware
different ISA Virtual PC for Mac Transmeta Crusoe
Java VM MS CLI
August 2005
19
Tutorial Topics

Introduction & VM Overview Emulation: Interpretation & Binary Translation Process VMs & Dynamic Translators HLL VMs Co-Designed VMs System VMs
August 2005
20
Emulation: Interpretation and Binary Translation
Key VM Technologies

Emulation: binary in one ISA is executed on processor supporting a different ISA Dynamic Optimization: binary is improved for higher performance

May be done as part of emulation May optimize same ISA (no emulation needed)
X86 apps Windows
HP UX HP Apps.
Alpha
HP PA ISA
Emulation
August 2005
Optimization
22
Definitions
NOTE -- there are no standard definitions Emulation:

A method for enabling a (sub)system to present the same interface and characteristics as another. E.g. the execution of programs compiled for instruction set A on a machine that executes instruction set B. Interpretation: relatively inefficient instruction-at-a-time Binary Translation: block-at-a-time optimized for repeated instruction executions
Ways of implementing emulation

August 2005
23
Definitions
Guest
Environment that is being supported by underlying platform Underlying platform that provides guest environment
Guest
Host
supported by
Host
August 2005
24
Definitions
Source ISA or binary
Original instruction set or binary I.e. the instruction set to be emulated Instruction set being executed by processor performing emulation I.e. the underlying instruction set Or the binary that is actually executed
Source
Target ISA or binary
emulated by
Sometimes Confusing terminology, e.g. shade:

Target -> Host
Target
Source/Target refer to ISAs; Guest/Host refer to platforms.

25
August 2005
Interpreters
HLL Interpreters have a very long history

Lisp Perl Forth (notable for its threaded interpretation model)
Binary interpreters use many of the same techniques

Often simplified Some performance tradeoffs are different E.g. the significance of using an intermediate form
August 2005
26
Interpreter State
Hold complete source state in interpreters data memory

Program Counter
Code
Condition Codes Reg 0 Reg 1
. . .
Data
Reg n-1
Stack
Interpreter Code
August 2005
27
Decode-Dispatch Interpretation
while (!halt) { inst = code(PC); opcode = extract(inst,31,6); switch(opcode) { case LdWordAndZero:LdWordAndZero(inst); case ALU: ALU(inst); case Branch: Branch(inst); . . .} } Instruction function list
August 2005
28
Instruction Functions: Load

LdWordAndZero(inst){ RT = extract(inst,25,5); RA = extract(inst,20,5); displacement = extract(inst,15,16); source = regs[RA]; address = source + displacement ; regs[RT] = data[address]; PC = PC + 4; }
August 2005
29
Decode-Dispatch: Efficiency
Decode-Dispatch Loop

Mostly serial code Several jumps/branches (some hard-to-predict) Approximately 20 target instructions Several loads/stores Several shift/mask steps Example: DEC/Compaq FX!32 Software pipelined decode-dispatch loop
Executing an add instruction

Hand-coding can lead to better performance
August 2005
30
Binary Translation
Generate custom code for every source instruction
Get rid of repeated instruction parsing and jumps altogether
August 2005
31
Optimization: Register Mapping

source ISA target ISA
Reduces loads/stores significantly Easier if

#target regs > #source regs
Source Register Block
R1
R2
Source Memory Image
Register mapping may be on a per-block basis

If #target registers not enough
program counter
R3
stack pointer reg 1 reg 2
R2 R5 R6
reg n
RN+4
August 2005
32
Binary Translation Example

x86 Source Binary
addl movl add %edx,4(%eax) 4(%eax),%edx %eax,4
Translate to PowerPC Target

r1 r2 r3 r4 r7
August 2005
points to x86 register context block points to x86 memory image contains x86 ISA PC value holds x86 register %eax holds x86 register %edx etc.
33
Binary Translation Example

x86 Source Binary
addl movl add %edx,4(%eax) 4(%eax),%edx %eax,4
PowerPC Target
addi lwzx add stwx mr addi
August 2005
r16,r4,4 r17,r2,r16 r7,r17,r7 r7,r2,r16 r4,r16 r3,r3,9
;add 4 to %eax ;load operand from memory ;perform add of %edx ;store %edx value into memory ;move update value into %eax ;update PC (9 bytes)
34
The Code Discovery Problem
In order to translate, emulator must be able to discover code
Easier said than done; especially w/ x86

source ISA instructions inst. 1 inst. 3 reg. data pad for instruction alignment inst. 2 jump data in instruction stream
inst. 5 inst. 6 uncond. brnch pad jump indirect to??? inst. 8
August 2005
35
Dynamic Translation
First Interpret
And perform code discovery as a byproduct Incrementally, as it is discovered Place translated blocks into Code Cache Save source to target PC mappings in lookup table Execute translated block to end Lookup next source PC in table
If translated, jump to target PC Else interpret and translate
Translate Code

Emulation process

August 2005
36
Dynamic Translation
source binary
SPC to TPC Lookup Table
miss
Interpreter Emulation Manager
translator
hit
Translation Mem ory
August 2005
37
Flow of Control
Control flows between translated blocks and Emulation Manager

translation block
Emulation Manager
translation block
translation block
August 2005
... VM Intro (c) 2005, J. E. Smith ...
38
Tracking the Source PC

Can always update SPC as part of translated code Better to place SPC in stub
Emulation Manager
Code Block
General Method:

Hash Table
Branch and Link to EM Next Source PC
Translator returns to EM via BL Source PC placed in stub immediately after BL EM can then use link register to find source PC and hash to next target code block
... VM Intro (c) 2005, J. E. Smith ...
Code Block
August 2005
39
Example
x86 Binary
4FD0: addl movl sub jz 4FDC: add jmp %edx,(%eax) (%eax),%edx %ebx,1 51C8 %eax,4 4FD0 ;load and accumulate sum 9AC0: ;store to memory ;decrement loop count ;branch if at loop end ;increment %eax ;jump to loop top
9AE4: lwz add stw addic. beq bl 4FDC bl 51C8 stw xor bl 6200
PowerPC Translation
r16,0(r4) r7,r7,r16 r7,0(r5) r5,r5,-1 cr0,pc+12 F000 F000 r7,0(r6) r7,r7,r7 F000 ;load value from memory ;accumulate sum ;store to memory ;decrement loop count, set cr0 ;branch if loop exit ;branch & link to EM ;save source PC in link register ;branch & link to EM ;save source PC in link register ;store last value of %edx ;clear %edx ;branch & link to EM ;save source PC in link register
51C8:
movl xorl jmp
(%ecx),%edx %edx,%edx 6200
;store last value of %edx ;clear %edx ;jump elsewhere
9C08:
August 2005
40
Example
PowerPC Translation
9AC0:
HASH TABLE
SPC TPC link
1 2
9AE4:
lw z add stw addic. beq bl 4FDC bl 51C8 stw xor bl 6200
r16,0(r4) r7,r7,r16 r7,0(r5) r5,r5,-1 cr0,pc+12 F000 F000
3 4
;load value from memory ;accumulate sum ;store to memory ;decrement loop count, set cr0 ;branch if loop exit ;branch & link to EM ;save source PC in link register ;branch & link to EM ;save source PC in link register
51C8
6 8
9C08
//////
Emulation Manager
;retrieve address in link register ;load SPC from stub ;perform halfword shift left ;perform XOR hash ;finish hash - logical shift ;access at hash address w/update ;r30 points to map table base ;compare for hit ;use target address ;else follow hash chain
9C08:
10
r7,0(r6) r7,r7,r7 F000
F000: ;store last value mflr of %edx r20 lwz r20,0(r20) ;clear %edx slwi r21,r20,16 5 ;branch & link toxor EM r21,r21,r20 ;save source PC in link register srwi r21,r21,12 lwzux r26,r21,r30 CR0,r26,r20 CR0, run lookup_translate cmpw beq b
1 2 3 4 5 6 7 8 9 10
Translated basic block is executed 9 Branch is taken to stub Stub BAL to Emulation Mgr. EM loads SPC from stub, using link EM hashes SPC and does lookup EM loads SPC from hash tbl; compares Branch to transfer code Load TPC from hash table Jump indirect to next translated block Continue execution
7
run: lwz mtlr blr r27,4(r21) r27 ;read target address from table ;branch to next translated block
lookup_translate: follow hash chain, if hit, branch to TPC If miss, branch to translate
August 2005
41
Translation Chaining
Jump from one translation directly to next

Avoid switching back to Emulation Mgr. Without Chaining: With Chaining:
translation block
translation block
EM
translation block
EM
translation block
translation block
translation block
translation block
August 2005
42
Software Jump Prediction

Form of Inline Caching Example Code: Say Rx holds source branch address
addr_i are predicted addresses (in probability order) Determined via profiling target_i are corresponding target code blocks target_1 goto target_2 goto target_3 ; do it the slow way
If Rx == addr_1 goto Else if Rx == addr_2 Else if Rx == addr_3 Else hash_lookup(Rx)
August 2005
43
Source/Target ISA Issues

Register architectures Condition codes
Lazy evaluation as needed Floating point Decimal MMX Byte vs Word addressing Natural vs arbitrary Big/Little endian
44
Data formats and operations

Address resolution
Address alignment
Byte order
August 2005
Emulation Summary
Decode/Dispatch Interpretation

source code "data" accesses
interpreter routines
Memory Requirements: Low Startup: Fast Steady State Performance: Slow Portability: Excellent
dispatch loop
August 2005
45
Emulation Summary
Binary Translation

binary translated target code source code
Memory Requirements: High Startup: Very Slow Steady State Performance Fast Portability: Poor
binary translator
August 2005
46

Perform guest/host mapping at ABI level Encapsulate guest process in processlevel runtime Issues

host process
guest process
runtim e
guest process
runtim e
guest process
runtim e
host process
Memory Architecture Exception Architecture OS Call Emulation Overall VM Architecture High Performance Implementations System Environments
create
HOST OS
file sharing
Disk
network communication
August 2005
48
Process VM Architecture
Application Mem ory Im age
Initialization
Em ulation Engine Code Cache Translator
Initialize signals
Profile Data
Interpreter
Code Cache Manager
OS Call Em ulator
Exception Em ulation
Exception Side Tables
Host Operating System
August 2005
49
Runtime Components
Initialization

Allocate Memory Initialize runtime data structures Initialize all signals Implement replacement algorithm when cache fills Flush when required (e.g. self-modifying code) Translate OS Calls Translate OS Responses Handle signals If registered by source code, pass to emulated source handler If not registered emulate host response Form precise state
50
Code Cache Manager

OS Call Emulator

Exception Emulator
August 2005
State Mapping
For best performance
Host Registers Guest Registers
Guest register space fits inside host register space Guest memory space fits inside host memory space Best case does not always happen But often does (x86 on RISC)
Host Register Space
Runtime Data Runtime Code
Guest Data
Host ABI Address Space
Guest Code
August 2005
51
Software Mapping
Runtime Software maintains mapping table
Similar to hardware page tables/TLBs Slow, but can always be made to work
Guest Application Address Space
Host Application Address Space
mapping table
Runtime Software
August 2005
52
Direct (Hardware) Mapping
VM software mapping is slow
Several instructions per load/store If guest address space + runtime fit within host space
Runtime Software
Use underlying hardware
Runtime Software
+base addr
August 2005
(a)
(b) VM Intro (c) 2005, J. E. Smith
53
Guest Memory Protection

Runtime must be protected from Guest process VM software mapping can be easily used
Place (and check) protection info in mapping table Runtime must be able to set privileges Protection faults should be reported to Runtime (So it can respond as guest OS would later) Requires some support from Host OS
Better: use underlying hardware

August 2005
54
Host OS Support

A system call where runtime can set protection levels A signal mechanism where protection faults trap to handler in runtime SimOS Example

Map guest space to a file (map, unmap, read-only mapping) Signal (SIGSEGV) delivered to VM software on fault
references succeed references cause page faults
Physical Memory File (VM Memory)
Virtual Machine's Virtual Address Space
references succeed
Free Pages
writes cause protection faults
Read-Only Mappings
August 2005
55
Self-Modifying Code

Write-Protect original code to catch code self-modification Exception handler invalidates old translations Be sure to make forward progress original code Pseudo self-modifying code may require optimizations

data
translated code
Data mixed in with code Implement software-supported fine-grain checking (Transmeta)
translator
Sometimes can rely on source binary to indicate self modification e.g. SPARC flush
write protected
August 2005
56
Self-referencing code

Original copy is maintained by translator All reads are with respect to original copy correct data is returned
data
original code self reference
translated code
translator
August 2005
57
Exceptions: Interrupts
Application may register some interrupts
Precise state easier than traps (because there is more flexibility wrt location)
Problem: Translated blocks may executed for an unbounded time period Solution:

Interrupt signal goes to runtime Runtime unchains translation block currently executing (eliminates loops) Runtime returns control to current translation Translation soon reaches end (and precise state is available)
If interrupts are common, runtime may inhibit all chaining

58
August 2005
Exceptions: Traps

Can be detected directly via interpretation
or explicit translated code checks
Can be detected indirectly via target ISA trap and signal
Runtime registers all trap conditions as signals If trap is architecturally similar in target in source then trap/signal may be used Otherwise interpretive method must be used
Semantic matching

Generally, more difficult than interrupts wrt precise state
August 2005
59
Precise State: Program Counter

Interpretation: Easy source PC is maintained Binary translation: more difficult source PC only available at translation block boundaries

Trap PC is in terms of target code Target PC must be mapped back to correct source PC Use side table and reverse translate Can be combined with PC mapping table Requires search of table to find trapping block Reconstruct block translation to identify specific source PC of trapping instruction
Solution

August 2005
60
PC Side Table
source code code cache
block A
signal returns target PC 2 trap occurs 1
find corresponding source PC 5 Re-analyze source code
block B 3
side table
target PCs
Start PC A Block Formation Info Block Formation Info
4 find source code start information
binary search side table
Start PC B
. . .
block N
Start PC N
August 2005
61
Recovering Register State
Simple if target code updates register state in same order as source code
Register state mapping can be used to generate source register values Implement software version of reorder buffer or checkpoints
More difficult if optimizations reorder code
August 2005
62
Recovering Memory State
Simple if target code updates memory state in same order as source code

Restricts optimizations (more difficult to back-up than register state) Most process VMs maintain original store order
August 2005
63
OS Call Emulation
Wrapper or Jacket code converts source call to target OS call(s)

Source code segment . . s_inst1 s_inst2 s_system_call X s_inst4 s_inst5 . . Target code segment . . t_inst1 t_inst2 jump runtime t_inst4 t_inst5 . . Runtime wrapper code copy/convert arg1 copy/convert arg2 . . t_system_call X copy/convert return val return to t_inst4
Binary Translation
August 2005
64
OS Call Emulation
Same source and target OSes (different ISAs)

Syntactic translation only E.g. pass arguments in stack rather than registers Semantic translation/matching required Similar to inter-OS porting May be difficult (or impossible) OS deals with real world What if source OS supports a type of device that the target does not?
Different source and target OSes
August 2005
65
High Performance Emulation
Important tradeoff

Startup time -- Cost of converting code for emulation Steady state -- Cost of emulating Low startup, high steady state cost High startup, low steady state cost
2500 2000 1500 1000 interpretation 500 0 10 20 30 40 50 60 70 80 90 100 N - Number of Times Emulated binary translation
Interpretation:
Binary translation
August 2005
Total Emulation Time
66
Staged Emulation

Adjust optimization level to execution frequency Tradeoff

Total
runtime = program runtime + translation overhead Higher optimization shorter program runtime Lower optimization lower overhead
Interpreter
Profile Data
Binary Memory Image
Code Cache
Emulation Manager
Translator/ Optimizer
August 2005
67
Staged Emulation
General Strategy
1. 2.
3.
Begin interpreting For code executed above a threshold Use simple translation/optimization For translated code executed above a threshold Optimize more etc. Shade uses 1 and 2 Wabi uses 2 and 3 FX!32 uses 1 and 3 IA32-EL, UQDBT use 2 and 3
Specific Strategies may skip some of the steps

August 2005
68
Code Cache Management
Code Cache is different from typical hardware cache

Variable sized blocks Dependences among blocks due to linking No backing store; re-generating is expensive LRU replacement is typically not used (fragmentation problems)
These factors affect replacement algorithm
August 2005
69
Flush When Full

Simple, basic algorithm Gets rid of stale links if control flow changes High overhead for re-translating after flush
August 2005
70
Pre-emptive Flush
Flush when program phase change is detected
Many new translations will be needed, anyway
Detect when there is a burst of new translations Dynamo does this
new translations
detect working set change and flush
time
August 2005 VM Intro (c) 2005, J. E. Smith
71
Coarse-Grain FIFO
Code Cache Backpointer Tables
Replace many blocks at once

FIFO block A
Large fixed-size blocks Only backpointers among replacement blocks need to be maintained OR linking between large blocks can be prohibited.
FIFO block B
. . .
FIFO block D
August 2005
72
System Environment
High level of interoperability Seamless access to both guest and host processes Works best with same OS
host process guest process

runtim e
guest process
runtim e
guest process
runtim e
host process
create
HOST OS
file sharing
Disk
August 2005
73
Encapsulation
Guest code is encapsulated

Host Process create Host DLL
At creation by loader DLLs at load time Host can create guest Guest can create host Guest can use guest or host Host uses only host
Guest Process create Guest DLL Host Process create Host DLL
Creation

DLLs

Host Process create
Guest Process
August 2005
74
Loaders
Requires two loaders

One for host processes One for guest processes Modify kernel loader Identifies type of binary, calls correct loader Requires modification of kernel loader Add code to guest binary when installed Invokes guest loader Requires local installation of guest binary Modify host process create_process API Invokes guest loader for guest binaries Modifies create_process in host binaries Used in FX!32
Approaches
August 2005
75
Persistence
How long do translations last?
One ABI instantiation Re-translate each time an ABI is initiated Multiple ABI instantiations Save translation/profile data on disk Is it faster to optimize or read from disk?
A lot of instructions can execute in a few milliseconds
August 2005
76
Example: FX!32

x86/Windows ABIs on Alpha/Windows Runtime software

Follows typical model But, translations/optimizations are done between executions First execution of binary: interpret and profile Translate and optimize off line Later execution(s): use translated version, continue profiling Translations and profile data are saved on disk between runs Very time consuming optimization with x86 source Hybrid static/dynamic binary translation
Persistence
August 2005
77
Performance

(comparing 200 MHz Pentium Pro and 500 MHz 21164) Goal: same as high-end x86 Byte benchmark integer 40% faster than Pentium Pro Flt point 30% slower than Pentium Pro Achieves 70% of native alpha performance
August 2005
78
Dynamic Binary Optimization
Optimization Example
Basic Block 1 ... ... R3 <- R7 <- ... R1 <- R2 + R3 Br L1 if R3 ==0
Superblock ... ... R3 <- R7 <- ... Br L2 if R3 !=0 R1 <- 0 ... Compensation code R1 <- R2 + R3
Basic Block 2 ... R6 <- R1 + R6 ...
Basic Block 3 L1: R1 <- 0 ...
Basic Block 2 L2: . . . R6 <- R1 + R6 ... (b)
(a)
August 2005
80
Profiling
Collect statistics about a program as it runs

Branches (taken, not taken) Jump targets Data values
Predictability allows these statistics to be used for optimizations to be used in the future Profiling in a VM differs from traditional profiling used for compiler feedback
E.g. cant do overall analysis before inserting probes
August 2005
81
Types of Profiles
Block or node profiles

Identify hot code blocks Fewer nodes than edges Give a more precise idea of program flow Block profile can be derived from edge profile (not vice versa)
A
65
Edge profiles

A 50 C
15
15 C 12 13 D 17 2 15
50
B 48
25
38 10
48
17
August 2005
82
Collecting Profiles
Instrumentation-based
Software probes Slows down program more Requires less total time Hardware probes Less overhead than software Less well-supported in processors Typically event counters Interrupt at random intervals and take sample Slows down program less Requires longer time to get same amount of data Not useful during interpretation
Sampling based
August 2005
83
Improving Locality: Procedure Inlining
User partial inlining

Unlike static full inlining Follow dominant flow of control A

Call proc xyz B
A X Y B Proc xyz X Y Z Return
. . .
K Call proc xyz L
. . .
K X Z L
August 2005
84
Improving Locality: Traces
Proposed by Fisher (Multiflow)
3 A
Trace 2 Trace 1
Used overall profile/analysis
30
70 D 29 C 68 2 E 68
Join points sometimes inhibit optimizations Join points detected incrementally

bookkeeping
B
Trace 31
F 1
29 G15
Typically not used in optimizing VMs
97
August 2005
85
Improving Locality: Superblocks

One entry multiple exits May contain redundant blocks (tail duplication) Commonly used in optimizing VMs
A A
G15
G15
August 2005
86
Superblock Formation
Start Points

When block use reaches a threshold Profile all blocks (UQDBT) Profile selected blocks (Dynamo) Profile only targets of backward branches (close loops) Profile exits from existing superblocks Use hottest edges above a threshold (UQDBT) Follow current control path (most recent edge) (Dynamo) Start point of this superblock Start point of some other superblock When a maximum size is reached When no edge above threshold can be found (UQDBT) When an indirect jump is reached (depends on whether inlining is enabled)
Continuation

End Points

August 2005
87
Dynamic Optimization Overview

Original source code Intermediate form Optimized target code
A B A B C
opt. A B C
comp comp
C
Collect basic blocks using profile information August 2005 Add compensation Schedule and Generate Convert to target code code; place in code intermediate optimize cache form; place in buffer VM Intro (c) 2005, J. E. Smith
88
Case Study: Intel IA-32 EL
Software method for running IA-32 binaries on IPF
Previous approach was in hardware OS independent section (BTgeneric) OS dependent section (BTlib) Fast binary translation (cold code) Optimized binary translation (hot code)
Runs with both Windows and Linux

Two stages

Precise traps are an important consideration
August 2005
89
Operating System Interaction

Process level runtime Supports Windows and Linux

BTlib is implementation dependent part About 1% of total code
Initializes structures Translates OS calls Handles exceptions
August 2005
90
IA-32 Optimizations
Floating point/MMX
IPF uses large flat register file IA-32 uses stack register file IA-32 TAG indicates valid entries IA-32 aliases MMX regs to FP regs
Speculate common case usage and put guard code at beginning of block Examples:
TOS (Top of Stack) same for all block executions No invalid accesses (indicated by TAG) 99-100% accurate
Data Misalignment
Similar to FX!32 See paper for details
August 2005
91
IA-32 EL Performance
Comparison with native IPF performance
Provides 65% performance (Gmean) mcf performs better because it has a 32-bit data footprint rather than 64-bits
August 2005
92
IA-32 EL Performance
Where the time is spent
SPEC mostly in hot code; very little overhead Sysmark only 45% hot code; 22% in OS (IPF code)
August 2005
93
Same-ISA Optimization
Objective: optimize binaries on-the-fly
Many binaries are un-optimized or are at a low optimization level Translation at basic block level is identity translation Initial sample-based profiling is attractive Original code can be used, running at native speeds Patch code cache regions into original code Replace original code with branches into code cache (saves code some code duplication) Can avoid hash table lookup on indirect jumps
Initial emulation can be done very efficiently

Code patching can be used

Can bail-out if performance is lost
August 2005
94
Code Patching Example

Source Binary Superblock Cache
A B patch
link
G patch Y patch indirect jum p
August 2005
95
Case Study: HP Dynamo

Maps HP-PA ISA onto itself Improved optimization is goal

native instruction stream
no
interpret until taken branch lookup branch target in cache m iss start-of-trace condition?
hit
jum p to top of fragm ent in cache
yes
increm ent counter assoc. w ith branch target addr
no
counter value exceeds hot threshold?
signal handler
OS signal
yes Fragment Cache

interpret + codegen until taken branch
em it into cache, link w ith other fragm ents & recycle the associated counter
create new fragm ent and optim ize it
yes
end-of-trace condition?
no
August 2005
96
Superblock Selection

Does not use hardware counters, PC sampling, or path sampling Interpreter performs MRET

Most Recently Executed Tail Associate a counter with superblock-start points If counter exceeds threshold then trigger instruction collection At superblock-end, collected instructions are hot superblock Concept: when an instruction becomes hot, the very next sequence will also be hot Simple, small counter overhead No overheads Problem if branch behavior changes Fragment cache is occasionally flushed
97
No profiling on fragments

August 2005
Prototype Implementation

Conservative optimizations
Allow recovery of state for synchronous traps Do not allow recovery of state Include Dead code removal Code sinking Loop invariant code motion
Aggressive optimizations

Start in aggressive mode, switch to conservative mode if suspicious code sequence is encountered Bail out for ill-behaved code

Unstable working sets Thrashing in the Fragment Cache

98
August 2005
Performance

Compare with +o2 Biggest gain from inlining and improved code layout Conservative opts help about as much as aggressive Some benchmarks bail-out
25 Percent speedup relativ e to nativ e +O2 execution aggressive conservative no optimization
20 15
10
5 0
ks im
eg
pr es s
i se
go
ue l ta bl de
li
rl
pe
rte
88
-10
August 2005
co
Av
-5
vo
er ag
ijp
bo
99
Performance
Outperforms +O2; +O4, but not +O4 plus profiling

This
may be due to code layout Many app developers do not profile

600
500 Native +O2 Native +O3 300 Native +O4 Native +O4 +P Dynamo +O2 Dynamo +O3 Dynamo +O4 Dynamo +O4 +P
Run Time (sec.)
400
200
100
ks im
eg
pr es s
go
i se
ue
li
rl
pe
vo rte
88
l ta
August 2005
co
de
Av
er ag
ijp
bo
bl
100
Performance Conclusions

Mostly useful for code optimized at low levels Dynamo ran on processor that stalled indirect jumps

Baseline is slow compared with most superscalar processors Dynamo removes indirect jumps via procedure inlining and inlined software jump prediction
On other modern processors there is a significant performance loss due to indirect jumps

See Dynamo/RIO (x86) RIO project targets security, not performance
August 2005
101
High Level Language VMs
HLL VMs

Goal: complete platform independence for applications Similar to Process VMs
Major difference is specification level: Virtual instruction set + libraries Instead of ISA and OS interface
HLL Program Compiler front-end Intermediate Code Compiler back-end Object Code (ISA) Loader Memory Image Traditional HLL Program Compiler Portable Code (Virtual ISA ) VM loader Virt. Mem. Image VM Interpreter/Translator Host Instructions HLL VM
August 2005
103
UCSD P-Code

Popularized HLL VMs Provided highly portable version of Pascal Consists of

Primitive libraries Machine-independent object file format Stack-based ISA A set of byte-oriented pseudo-codes Virtual machine definition of pseudo-code semantics
August 2005
104
Modern HLL VMs
Superficially similar to P-code scheme

Stack-based ISA Standard libraries BUT, Objective is application portability, not compiler portability Untrusted software (this is the internet, after all) Robustness (generally a good idea) => object-oriented programming Bandwidth is a consideration Good performance must be maintained Java VM Microsoft Common Language Infrastructure (CLI)
Network Computing Environment

Two major examples

August 2005
105
Terminology

Java Virtual Machine Architecture CLI
Analogous to an ISA Analogous to a computer implementation
Java Virtual Machine Implementation CLR
Java bytecodes Microsoft Intermediate Language (MSIL), CIL, IL
The instruction part of the ISA ISA + Libraries; a higher level ABI
Java Platform .NET framework
August 2005
106
Modern HLL VMs
Compiler forms program files (e.g. class files)

Standard format In theory any compiler can be used

Virtual Machine Implementation
Program files contain both code and metadata

Machine Independent Program File Loader Metadata Interpreter Code Translator Native Code
Internal Data Structures
August 2005
107
Robustness: Object-Orientation
Objects

Data carrying entities Dynamically allocated Must be accessed via pointers or references Procedures that operate on objects Method operating on an object is like sending a message A type of object and its associated methods Object created at runtime is an instance of the class Data associated with a class may be dynamic or static
Methods

Classes

August 2005
108
Security

Remote System
A key aspect of modern network-oriented VMs Rely on protection sandbox Must protect:

Other File Public File
Remote resources (files) Local files Runtime from user process
Sandbox Boundary Network
This is the first generation security method still the default
application
Accessible Local File
VMM User Process Other Local File
Local System
August 2005
109
Protection Sandbox
class file class file class file class file
Remote resources
Protected by remote system Protected by security manager Protected via static/dynamic checking
local file local file
Network, File System
Local resources
loaded method
loaded method
loaded method
VM software
loaded method lib. method lib. method
loaded method
native method
standard libraries
native method
loaded method
trusted
security agent
trusted
Emulation Engine trusted
loader
trusted
August 2005
110
Garbage Collected Heap
Objects are created and float in memory space

Tethered by references In architecture, memory is unbounded in size In reality it is limited During program execution, many objects are created then abandoned (become garbage) Due to limited memory space, Garbage should be collected so memory can be re-used Forcing programmer to explicitly free objects places more burden on programmer Can lead to memory leaks, reducing robustness To improve robustness, have VM collect garbage automatically
111
Garbage creation
Collection

August 2005
Network Friendliness
Support dynamic class file loading on demand

Load only classes that are needed Spread loading out over time Use stack-oriented ISA (as in Pascal) Metadata also consumes bandwidth, however Overall, it is probably a wash
Compact instruction encoding

August 2005
112
Java ISA
Includes

Bytecode (instruction) definitions Metadata: data definitions and inter-relationships
Formalized in class file specification
August 2005
113
Java Architected State
Implied Registers

PC Stack Pointer etc. Locals Operands Objects Arrays (intrinsic objects) Constant pool holds immediates and other constant information
Stack

Heap

Class file contents
August 2005
114
Data Accessing
index Instruction stream opcode opcode opcode opcode opcode opcode opcode opcode operand operand operand operand operand operand operand implied implied index index
CONSTANT POOL
HEAP
Array
Object
STACK FRAME
Locals
Operands Object
Object
August 2005
115
Instruction Set

Defined for class file, not memory image Bytecodes

opcode
One byte opcode Zero or more operands Opcode indicates how many Instruction Current constant pool Current frame local variables Values on operand stack Distinguish storage types and computation types
opcode
index
Can take operands from

opcode
index1
index2
opcode
data
opcode
data1
data2
August 2005
116
Instruction Types

Pushing constants onto the stack Moving local variable contents to and from the stack Managing arrays Generic stack instructions (dup, swap, pop & nop) Arithmetic and logical instructions Conversion instructions Control transfer and function return Manipulating object fields Method invocation Miscellaneous operations Monitors
117
August 2005
Data Movement
All data movement takes place through stack

CONSTANT POOL STACK FRAME
Instruction stream locals
GLOBAL STORAGE
opcode opcode opcode
operand operand
operand
operand stack
ALU
August 2005
118
Bytecode Example
public int perimeter();
PC
0: 1: 2: 5: 6: 7: 8: 11: 12: 13: 14: 15:
August 2005
instruction
iconst_2 aload_0 getfield #2; iconst_0 iaload aload_0 getfield #2; iconst_1 iaload iadd imul ireturn //Field: sides reference
//Field: sides reference
119
Stack Tracking
Operand stack at any point in program has:

Same number of operands Of same types In same order Regardless of control flow path getting there
Helps with static type checking
August 2005
120
Exception Table

Exceptions identified by table in class file Address Range where checking is in effect Target if exception is thrown
Operand stack is emptied Pop stack frame and check calling method Default handlers at main
To 12 Target 96 Type Arithmetic Exception
If no table entry in current method

From 8
August 2005
121
Binary Classes

Magic Number Version Information Const. Pool Size Constant Pool
Formal ISA Specification Magic number and header Major regions preceded by counts

Access Flags This Class Super Class Interface Count Interfaces Field count Field Information Methods count
Constant pool Interfaces Field information Methods Attributes
Methods
Attributes Count
Attributes
August 2005
122
Java Virtual Machine

An abstract entity that gives meaning to class files Has many concrete implementations

Hardware Interpreter JIT compiler An instance is created when an application starts Terminates when the application finishes
Persistence

August 2005
123
Structure of Virtual Machine

class files
Class Loader Subsystem
Memory
method area Java stacks native method stacks
heap
Garbage Collector
addresse s
PCs & implied regs
data & instructions

native method interface
Execution Engine
native method libraries

124
August 2005
Structure of Virtual Machine, contd.

Method Area
Type information provided by class loader Contains objects created by program Every created thread gets a set Every created thread gets one Divided into Frames Contains state of method invocations for the thread Local variables, parameters, return value, operand stack Special area for implementation-dependent native methods
125
Heap Area
PC Register & Implied Registers
Java stacks

Native method stacks
August 2005
Class Loader Subsystem

Primordial loader Other loaders that are part of apps Tasks:
Finds and imports binary information describing type Verifies correctness of type Allocates and initializes memory for class variables Resolves symbolic references to direct references Invokes initialization code
August 2005
126
Protection Sandbox: Security Manager

A trusted class containing check methods Attached when Java program starts
Cannot be removed or changed Files, types of access, etc. Native methods that involve resource accesses (e.g. I/O) first call check method(s)
User specifies checks to be made
Operation
August 2005
127
Verification

Class files are checked when loaded
To ensure security and protection Checks for magic number Checks for truncation or extra bytes Each component specifies a length Make sure components are well-formed Check valid opcodes Perform full path analysis Regardless of path to an instruction contents of operand stack must have same number and types of items Checks arguments of each bytecode Check no local variables are accessed before assigned Makes sure fields are assigned values of proper type
128
Internal Checks

Bytecode checks

August 2005
Java Native Interface (JNI)
Allows Java and native SW to interoperate
Java Side
Jav a HLL Program
Native Side
C Program
E.g. Java can call C program (and vice versa) Native routines allow access of Java data
Compile and Load
Compile and Load
Bytecode Methods getfield/ putfield
inv oke nativ e method Nativ e Machine Code load/store Nativ e Data Structures
JNI get/put
obj ect
obj ect
array
August 2005
129
JVM Bytecode Emulation

Interpretation
Simple, fast startup, but slow Compile each method when first touched Simple, static optimizations Find frequently executed code Apply more aggressive optimizations on that code Typically phased with interpretation or JIT Based on Hot-Spot compilation Use runtime information to optimize
130
Just-In-Time (JIT) Compilation

Hot-Spot Compilation

Dynamic Compilation

August 2005
Microsoft CLI

Common Language Infrastructure Part of .NET framework Allows multiple HLLs and multiple Platforms Allows both verifiable and unverifiable modules (class files)

Verifiability is different from validity Unverifiable modules must be trusted by user Verifiable and unverifiable modules can be mixed (but the program becomes unverifiable)
August 2005
131
Microsoft CLI Interoperability

C# program Java Program Visual Basic.Net Managed C++ program
Compile
Compile
Compile
Compile
Verifiable Module
Verifiable Module
Verifiable Module
Unverifiable Module
Common Language Runtime
Common Language Runtime
X86 Platform
IA-64 Platform
August 2005
132
Microsoft CLI and MSIL
Similar to Java and JVM

Object oriented Stack-based ISA Broader in scope ISA not designed for interpretation Module can be valid (but not verifiable), verifiable, or invalid Support for C-like pointers and un-typed memory blocks (not verifiable)
Some differences

August 2005
133
Summary: HLL VMs vs. Process VMs
Memory architecture
Object model is less implementation-dependent No compatibility problems due to size limitations/differences Pointers very carefully controlled No rogue load/stores Exception checking is explicit (no masks) Operand stack imprecise within a method Locals imprecise if exception goes to higher level
134
Memory protection
Precise Exceptions

August 2005
Summary: HLL VMs vs. Process VMs
Instruction set dependences

No registers No condition codes Restricted, explicit control flow All code can be discovered at method entry Simply doesnt exist
Code discovery

Self Modify-Referencing Code
August 2005
135
Co-Designed VMs
Co-Designed VMs
Design hardware and VM software concurrently and cooperatively Use proprietary target ISA
Applications
Or modified ISA
OS
No native OS or applications Goal is performance or power efficiency
Emulation/Translation Software
Cached Translated Code
Source ISA Hidden Software Target ISA
Not compatibility
VMM
Techniques also applicable to HW support for other VMs
Hardware
August 2005
137
Concealed Memory
VM
software resides in memory concealed from all conventional software
concealed memory
Translation Cache VM Code VM Data Source ISA Code
ICache Hierarchy Processor Core DCache Hierarchy
conventional memory
Source ISA Data
August 2005
138
Precise Exceptions
Traps must be precise wrt original binary

All conventional software is unaware of underlying VM Code may undergo heavy duty re-organization E.g. CISC VLIW Have VMM periodically checkpoint state Consistent with a point in original binary On fault, rollback and interpret original binary Keep out-of-order results in scratch registers, update architected registers in-order I.e. software renaming
Checkpoint and rollback

In-order state update
August 2005
139
Checkpoint and Rollback

set checkpoint
Translation Block A
set checkpoint
Translation Block B
Translation Block A interpret
restore checkpoint
Translation Block B
Source Code
trap
set checkpoint
Translation Block N
August 2005
140
Precise Interrupts via checkpoint/rollback

As in Transmeta Crusoe Shadow copies of registers Gated store buffer Code divided into translation groups
Crusoe
x86
X86 regs
shadow
Precise state between groups
Commit when trans. group is exited

scratch spec. results constants
At commit point make shadow copy, release gated stores & establish new gate stores
Release gated store buffer Copy current registers into shadow
gated store buffer August 2005 VM Intro (c) 2005, J. E. Smith

141
Precise Interrupts via checkpoint/rollback
When a trap occurs

Flush store buffer Backup with shadow registers Interpret forward until trap occurs Larger precise interrupt units => coarser grain optimizations, dead code elimination, etc. Store buffer size limits translation unit size
Crusoe
x86
X86 regs
shadow
Advantage:
scratch spec. results constants
On exception restore from shadow copy, squash gated stores & establish new gate for stores
Disadvantage:
gated store buffer
August 2005
142
Page Fault Compatibility

Major difference wrt Process VMs All page faults in guest must be accurately emulated Data accesses no problem
Detected via page table/TLB Fetches are from code cache, not guest memory Code cache pages are not related to guest pages
Instruction accesses more difficult

August 2005
143
Page Crossings
code cache guest pages
A B C D E D E probe page table F G H I J H I J page correctly mapped? yes no F G page correctly mapped? yes A B C
no
jump to VMM
continue execution
K L
probe page table K L continue execution
jump to VMM
August 2005
144
Input/Output

VMM itself uses no I/O Run guest I/O drivers as-is
Let I/O drivers directly control I/O signals

Use access-protect in TLB to detect accesses to volatile pages De-optimize code that accesses volatile pages Enhance ISA w/ load/store opcodes that over-ride access-protect
Problems w/ Memory-Mapped I/O

August 2005
145
Case Study: Transmeta Crusoe
x86 to VLIW (4-way)
decompress
512 KBytes compressed VMM 2 MBytes VMM 8KB Local Inst. Mem.
Specialized Fields (ALU, LD/ST, FP, Br)
16M Translation Cache 8K bytes VMM local inst. mem.
8KB Local Prog. Mem.
Reduces I cache pollution Reduces D cache pollution

14 MBytes Crusoe Data & Translations 64 KB I-Cache
8K bytes VMM local data mem.
64 KB D-Cache
August 2005
146
Transmeta Crusoe Block Diagram

Shadow FPRs 32 FPRs B r FPU
L1 I-Cache 64B LInes 8-w ay 64 Kbytes Local Program Mem ory 8Kbytes
F P
Gated Store Buffer Ld St ALU1 A L U
L1 D-Cache 32B LInes 16-w ay 64 Kbytes
L2 Cache 256Kbytes 4-w ay
Shadow GPRs
ALU0
64 GPRs
August 2005
147
Crusoe Translation
Staged optimization

Interpretation (with profiling) Simple translation Highly optimized translation
Algorithm translates multiple basic blocks
August 2005
148
Alias Hardware

To allow re-scheduling of memory ops Removal of redundant loads load-and-protect

Special load opcode records load address and loaded data size in table Special store opcode Checks specified (via mask) loads in table if conflict, triggers re-do of loads
store-under-alias-mask

August 2005
149
Alias Hardware Example

Original Code st (W) . . . ld (X) . . . st (Y) . . . ld (Z) Optimized Code ldp (X) x . . . ldp (Z) x x . . . stam (W) . . . stam (Y)
stam(W) traps if W==X or W==Z stam(Y) traps if Y ==Z
August 2005
150
Another Example: IBM AS/400

A very early (and successful) co-designed VM Goals
Hardware independence Demonstrated by move to PowerPC Support robust/well integrated software Re-define conventional software boundaries Divided OS into implementation independent/dependent parts Architect object-orientation
August 2005
151
IBM AS/400 Platforms

System /38 Proprietary implementation ISA AS/400 First, extend proprietary ISA Then transition to PowerPC ISA
OS/400 User Applications MI & LIC Translator; Implementation-Dep. OS IMPI Proprietary Platform PowerPC Platform Translator; Implementation-Dep. OS PowerPC OS/400 User Applications MI & LIC
August 2005
152

Vmware Architecture

Uploaded by

Copyright:

Available Formats

Vmware Architecture

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vmware Architecture

Uploaded by

Copyright:

Available Formats

Virtual Machines: Architectures, Implementations and Applications

HOTCHIPS 17 Tutorial 1, Part 1

VM Intro (c) 2005, J. E. Smith

Execution Hardware Memory Translation

System Interconnect (bus) Controllers I/O devices and Networking

Construct Virtual Disks

As files on a larger disk Map state Implement functions

VMs: do the same thing with the whole machine

VM Intro (c) 2005, J. E. Smith

Different perspectives on what the Machine is: OS developer

Application Programs Libraries Operating System

Instruction Set Architecture

ISA Major division between hardware and software

System Interconnect (bus) I/O devices and Networking

VM Intro (c) 2005, J. E. Smith

Different perspectives on what the Machine is: Compiler developer

Application Programs Libraries Operating System

Application Binary Interface

ABI User ISA + OS calls

System Interconnect (bus) I/O devices and Networking

VM Intro (c) 2005, J. E. Smith

Different perspectives on what the Machine is: Application programmer

Application Programs Libraries Operating System

Application Program Interface

API User ISA + library calls

System Interconnect (bus) I/O devices and Networking

VM Intro (c) 2005, J. E. Smith

VM Intro (c) 2005, J. E. Smith

The Family of Virtual Machines

Lots of things are called virtual machines

VM Intro (c) 2005, J. E. Smith

Taking a Unified View

VM Intro (c) 2005, J. E. Smith

System Virtual Machines

guest guest process process Guest OS VMM

guest guest guest process process process Guest OS2 VMM

virtual network communication

VM Intro (c) 2005, J. E. Smith

Application Process Virtualizing Software OS

VM Intro (c) 2005, J. E. Smith

Process Virtual Machines

VM Intro (c) 2005, J. E. Smith

VM Intro (c) 2005, J. E. Smith

X86 Apps Windows

VM Intro (c) 2005, J. E. Smith

VM Intro (c) 2005, J. E. Smith

VM Intro (c) 2005, J. E. Smith

same ISA Multiprogrammed Systems HP Dynamo

different ISA IA-32 EL FX!32

same ISA IBM VM/370 VMware

different ISA Virtual PC for Mac Transmeta Crusoe

VM Intro (c) 2005, J. E. Smith

Emulation: Interpretation and Binary Translation

Ways of implementing emulation

VM Intro (c) 2005, J. E. Smith

VM Intro (c) 2005, J. E. Smith

Source ISA or binary

Target ISA or binary

Sometimes Confusing terminology, e.g. shade:

Source/Target refer to ISAs; Guest/Host refer to platforms.