EC792

High Performance Computing Architecture

Lecture 3
Performance measures

M S Bhat
msbhat@nitk.edu.in
 Performance Measures
❑ Benchmark suites (SPEC - Standard Performance Evaluation
Corporation SPEC CPU2006 – 26 programs gives one SPEC rating,
ESPRESSO….)
❑ Performance is the result of executing a workload on a configuration
❑ Workload = program + input
❑ Configuration = CPU + cache + memory + I/O + OS + compiler +
optimizations
◼ Compiler optimizations can make a huge difference

Choosing Programs to evaluate Performance

❑ Real programs – C compilers, TeX, CAD tools etc
❑ Kernels – Loops, LINPACK (e.g., Solving Ax=b using Gaussian
elimination )
❑ Toy Benchmarks – solving Puzzles, quicksort etc.
❑ Synthetic Benchmarks – Whetstones (floating point operations),
Dhrystones etc.
Measurement Tools
❑ Benchmarks, Traces, Mixes
❑ Hardware: Cost, delay, area, power estimation
❑ Simulation (many levels)
◼ ISA, RT, Gate, Circuit

❑ Queuing Theory
Benchmark Suites
❑ Performance is measured with benchmark suites: a collection of
programs that are likely relevant to the user
◼ SPEC CPU 2006: cpu-oriented programs (for desktops)

◼ SPECweb, TPC: throughput-oriented (for servers)

◼ EEMBC: for embedded processors/workloads Whetstone

(1976) -- designed to simulate arithmetic-intensive scientific
programs.
◼ Dhrystone (1984) -- designed to simulate systems
programming applications. Structure, pointer, and string
operations are based on observed frequencies, as well as
types of operand access (global, local, parameter, and
constant), (no floating point op.)
◼ Whetstone – another synthetic benchmark (floating point op.)

◼ Linpack – FP benchmark (Ax=b)

◼ PC Benchmarks – aimed at simulating real environments

⚫ Business Winstone – navigator + Office Apps
SPEC CPU Benchmarks www.spec.org
Integer benchmarks FP benchmarks
gzip compression wupwise Quantum chromodynamics
vpr FPGA place & route swim Shallow water model
gcc GNU C compiler mgrid Multigrid solver in 3D fields
mcf Combinatorial optimization applu Parabolic/elliptic pde
crafty Chess program mesa 3D graphics library
parser Word processing program galgel Computational fluid dynamics
eon Computer visualization art Image recognition (NN)
perlbmk perl application equake Seismic wave propagation
simulation
gap Group theory interpreter facerec Facial image recognition
vortex Object oriented database ammp Computational chemistry
bzip2 compression lucas Primality testing
twolf Circuit place & route fma3d Crash simulation fem
sixtrack Nuclear physics accel
Gaming softwares apsi Pollutant distribution
Programs in SPECCPU2000 benchmark suites.
Example SPEC CPU Ratings

• Higher Spec ratio number indicates faster performance

than a Sun Ultra 5_10 with a 300 MHz processor
• In this graph performance scales almost linearly with
clock rate increase, but this is often not the case.
Programs in SPEC as of 2017

Fig. Active Benchmarks from SPEC as of 2017

Comparing Performance
Total execution time (implies equal mix in workload)
◼ Just add up the times

Arithmetic average of execution time

◼ To get more accurate picture, compute the average of several
runs of a program

Weighted execution time (weighted arithmetic mean)

◼ Program p1 makes up 25% of workload (estimated), P2 75%
then use weighted average

Geometric mean consistent but cannot predict execution

time
◼ Nth root of the product of execution time ratios

Example: Let’s look at an example

The Bottom Line: Performance (and Cost)

New Delhi Throughput

Plane Speed Passengers (in passenger mile per
to Paris hour)

Boeing 747 6.5 hours 610 mph 470 286,700

Concorde 3 hours 1350 mph 132 178,200

• Time to run the task (ExTime)

– Execution time, response time, latency
• Tasks per day, hour, week, sec, ns … (Performance)
– Throughput, bandwidth
Defining (Speed) Performance
❑ Normally interested in reducing
◼ Response time (aka execution time) – the time between the start
and the completion of a task
⚫ Important to individual users
◼ Thus, to maximize performance, need to minimize execution time
performanceX = 1 / execution_timeX

If X is n times faster than Y, then

performanceX execution_timeY
-------------------- = --------------------- = n
performanceY execution_timeX

Throughput – the total amount of work done in a given time

- Important to data center managers
Decreasing execution time almost always improves throughput
Performance Factors
❑ Want to distinguish elapsed time and the time spent on
our task
❑ CPU execution time (CPU time) : time the CPU spends
working on a task
◼ Does not include time waiting for I/O or running other programs

CPU execution time = # CPU clock cyclesx clock cycle time

for a program for a program
or
CPU execution time = #-------------------------------------------
CPU clock cycles for a program
for a program clock rate
❑ Can improve performance by reducing either the length
of the clock cycle or the number of clock cycles required
for a program
Review: Machine Clock Rate
❑ Clock rate (MHz, GHz) is inverse of clock cycle time (clock
period)
CC = 1 / CR

one clock period

10 nsec clock cycle => 100 MHz clock rate

1 nsec clock cycle => 1 GHz clock rate
500 psec clock cycle => 2 GHz clock rate
200 psec clock cycle => 5 GHz clock rate
Clock Cycles per Instruction
❑ Not all instructions take the same amount of time to
execute
◼ One way to think about execution time is that it equals the
number of instructions executed multiplied by the average time
per instruction
# CPU clock cycles # Instructions Average clock cycles
= for a program x
for a program per instruction

❑ Clock cycles per instruction (CPI) – the average number

of clock cycles each instruction takes to execute
A way to compare two different implementations of the same ISA

CPI for this instruction class

A B C
CPI 1 2 3
Effective CPI
❑ Computing the overall effective CPI is done by looking at
the different types of instructions and their individual cycle
counts and averaging n
Overall effective CPI =  (CPIi x ICi)
i=1
Where ICi is the count (percentage) of the number of instructions
of class i executed
CPIi is the (average) number of clock cycles per instruction for
that instruction class
n is the number of instruction classes

❑ The overall effective CPI varies by instruction mix – a

measure of the dynamic frequency of instructions across
one or many programs
The Performance Equation
❑ Our basic performance equation is then
CPU time = Instruction count x CPI x cycle time
or
Instruction count x CPI
CPU time = -----------------------------------------------
clock rate
❑ These equations separate the three key factors that
affect performance
Can measure the CPU execution time by running the program
The clock rate is usually given
Can measure overall instruction count by using profilers/
simulators without knowing all of the implementation details
CPI varies by instruction type and ISA implementation for which
we must know the implementation details
 CPU Performance Equation

CPU time = CPU Clock Cycles  Clock cycle time

CPU time = Instruction Count  Cycles Per Instruction  Clock cycle time

CPU time = 
Seconds Instructions Clock Cycles
=  Seconds
Program Program Instruction Clock Cycle

ISA, Organization, Hardware

Compiler Technology Technology,
ISA
Organization

A.K.A. The “iron law” of performance

CPI example
❑ Computer A: Cycle Time = 250ps, CPI = 2.0
❑ Computer B: Cycle Time = 500ps, CPI = 1.2
❑ Same ISA, which computer is faster?

CPU Time A = Instruction Count CPIA Cycle Time A

= I 2.0  250ps = I 500ps

CPU TimeB = Instruction Count CPIB Cycle TimeB

= I1.2  500ps = I 600ps

CPU TimeB I 600ps

= = 1.2
A is faster by this much
CPU Time A I 500ps
Determinates of CPU Performance
CPU time = Instruction count x CPI x cycle time

Instruction CPI cycle time

count
Algorithm

Programming
language
Compiler

ISA

Processor
organization
Technology
Determinates of CPU Performance
CPU time = Instruction count x CPI x cycle time

Instruction CPI cycle time

count
Algorithm
X X

Programming X X
language
Compiler X X

ISA X X X

Processor
X X
organization
Technology
X
A Simple Example
Op Freq CPIi Freq x CPIi
ALU 50% 1 .5
Load 20% 5 1.0
Store 10% 3 .3
Branch 20% 2 .4
Overall effective CPI = 2.2
A Simple Example
Op Freq CPIi Freq x CPIi Q1
ALU 50% 1 .5 .5
Load 20% 5 1.0 .4
Store 10% 3 .3 .3
Branch 20% 2 .4 .4
Overall effective CPI = 2.2 1.6

❑ How much faster would the machine be if a better data cache

reduced the average load time to 2 cycles?

CPU time new = 1.6 x IC x CC so 2.2/1.6 means 37.5% faster

(Notice that 2.2 x IC x CC / 1.6 x IC x CC = 2.2/1.6, since the IC and CC
remain unchanged by adding a bigger cache)
A Simple Example
Op Freq CPIi Freq x CPIi Q2
ALU 50% 1 .5 .5
Load 20% 5 1.0 1.0
Store 10% 3 .3 .3
Branch 20% 2 .4 .2
Overall effective CPI = 2.2 2.0

❑ How does this compare with using branch prediction to save a

cycle off the branch time?

CPU time new = 2.0 x IC x CC so 2.2/2.0 means 10% faster

A Simple Example
Op Freq CPIi Freq x CPIi Q3
ALU 50% 1 .5 .25
Load 20% 5 1.0 1.0
Store 10% 3 .3 .3
Branch 20% 2 .4 .4
Overall effective CPI = 2.2 1.95

❑ What if two ALU instructions could be executed at once?

CPU time new = 1.95 x IC x CC so 2.2/1.95 means 12.8% faster

A Simple Example
Op Freq CPIi Freq x CPIi Q1 Q2 Q3
ALU 50% 1 .5 .5 .5 .25
Load 20% 5 1.0 .4 1.0 1.0
Store 10% 3 .3 .3 .3 .3
Branch 20% 2 .4 .4 .2 .4
Overall effective CPI = 2.2 1.6 2.0 1.95

❑ How much faster would the machine be if a better data cache

reduced the average load time to 2 cycles?
CPU time new = 1.6 x IC x CC so 2.2/1.6 means 37.5% faster
❑ How does this compare with using branch prediction to remove
a cycle off the branch time?
CPU time new = 2.0 x IC x CC so 2.2/2.0 means 10% faster
❑ What if two ALU instructions could be executed at once?
CPU time new = 1.95 x IC x CC so 2.2/1.95 means 12.8% faster
CPU Performance Equation

CPU time = CPU Clock Cycles  Clock cycle time

 n 
CPU time =   IC i  CPIi   Clock cycle time
 i =1 

How many cycles it

For each kind takes to execute an
of instruction instruction of this
kind
How many
instructions of this
kind are there in the
program
CPU performance w/ different instructions

Instruction Frequency CPI

 n 
Type CPU time =   IC i  CPIi   Clock cycle time
 i =1 
Integer 40% 1.0
Branch 20% 4.0
Load 30% 2.0
Store 10% 3.0

Total Instructions = 50M, Clock speed = 2 GHz

Comparing and Summarizing Performance
❑ How do we summarize the performance for benchmark
set with a single number?
The average of execution times that is directly proportional to
total execution time is the arithmetic mean (AM)
n
AM = 1/n  Timei
i=1
Where Timei is the execution time for the ith program of a total of
n programs in the workload
A smaller mean indicates a smaller average execution time and
thus improved performance
❑ Guiding principle in reporting performance measurements is
reproducibility – list everything that another experimenter would need
to duplicate the experiment
◼ E.g., Version of the operating system, compiler settings, input set used, specific
computer configuration (clock rate, cache sizes and speed, memory size and
speed, etc.)
 Comparing Performance
• Consider 3 programs from a benchmark set – how do we capture the
behavior of 3 processors A, B & C for all 3 programs with a single
number? Assume Sys-A to be a reference machine.

ExecTime: Prgm1 Prgm2 Prgm3 SET SWET GM

Sys-A 10 8 25 43 3 12.6
Sys-B 12 9 20 41 3.125 12.9
Sys-C 8 8 30 46 3 12.43

➢ Sum of execution times (SET) (Arithmetic Mean: AM)

➢ Sum of weighted execution times (SWET) (AM) w.r.t. Sys-A
➢ Geometric mean of execution times (GM)
➢ Let E1,E2,E3 be the exec times of Sys-A for Prgm1, Prgm2, Prgm3
GM = 3
( E1* E 2* E 3)
Sum of Weighted Exec Times – Example

• Let us fix a reference machine , X and run 4 programs A, B, C, D on

it such that each program will run for 1 second (Total time = 4s.)

• The exact same workload (the four programs execute the same
number of instructions that they did on machine X) is run on a new
machine Y and the execution times for each program are 0.8, 1.1,
0.5, 2 (Total time = 4.4s. )

• With AM of normalized execution times, we can conclude that Y is

1.1 times slower than X – perhaps, not for all workloads, but atleast
for this workload
GM Example
Computer-A Computer-B Computer-C
P1 1 sec 10 secs 20 secs
P2 1000 secs 100 secs 20 secs
GM 31.62 31.62 20

Conclusion with GMs: (i) A=B

(ii) C is ~1.6 times faster

• For (i) to be true for AM too, P1 must occur 100 times for every
occurrence of P2 i.e., Consider : P1x100+P2 . Now take GM for this
scenario

• With the above assumption, (ii) is no longer true for AM. Hence, GM
can lead to inconsistencies
 GM Example
Computer-A Computer-B Computer-C
P1 1 sec 10 secs 20 secs
P2 1000 secs 100 secs 20 secs
GM 31.62 31.62 20

Spec ratio A Exec timeB 10 Spec ratioA Exec timeC 20

SpecB1 = = = SpecC1 = = =
Spec ratioB Exec timeA 1 Spec ratioC Exec timeA 1

Spec ratioA Exec timeB 100 Spec ratioA Exec timeC 20

SpecB 2 = = = SpecC 2 = = =
Spec ratioB Exec timeA 1000 Spec ratioC Exec timeA 1000

SpecB1 * SpecB 2 GM B
=
SpecC1 * SpecC 2 GM C
Summarizing Performance

GM: Does not require a reference machine, but does not predict
performance very well. So we multiplied execution times and
determined that sys-C is 1.6x faster…but on what workload?

AM: Does predict performance for a specific workload, but that

workload was determined by executing programs on a reference
machine

Every year or so, the reference machine will have to be updated

CPU Performance Equation
• Clock cycle time = 1 / clock speed

• CPU time = Instruction count x CPI x clock cycle

• Influencing factors for each:

➢ Instruction count: instruction set design and compiler
➢ CPI: architecture and instruction set design
➢ Clock cycle time: technology and pipeline

• CPI (cycles per instruction) or IPC (instructions per cycle) can not
be accurately estimated analytically
An Alternative Perspective - I
• Each program is assumed to run for an equal number of cycles, so
we’re fair to each program

• The number of instructions executed per cycle is a measure of how

well a program is doing on a system

• The appropriate summary measure is sum of IPCs or

AM of IPCs = 1.2 instr + 1.8 instr + 0.5 instr ( = 3.5 instr)
cyc cyc cyc 3 cyc

This measure implicitly assumes that 1 instr in prog-A has the same
importance as 1 instr in prog-B
An Alternative Perspective - II
• Each program is assumed to run for an equal number of
instructions, so we’re fair to each program

• The number of cycles required per instruction is a measure of

how well a program is doing on a system

• The appropriate summary measure is sum of CPIs or

AM of CPIs = 0.8 cyc + 0.6 cyc + 2.0 cyc ( = 3.4 cyc)
instr instr instr 3 instr

• This measure implicitly assumes that 1 instr in prog-A has the

same importance as 1 instr in prog-B
 Problem 2
• My new laptop has an IPC that is 20% worse than my old laptop. It
has a clock speed that is 30% higher than the old laptop. I’m running
the same binaries on both machines. What speedup is my new
laptop providing?

Exec time = cycle time * CPI * instrs

Perf = clock speed * IPC / instrs
Speedup = new perf / old perf
= new clock speed * new IPC / old clock speed * old IPC
= (new clock speed / old clock speed )*( new IPC/old IPC)
= 1.3 * 0.8 = 1.04
Speedup Vs. Percentage

• “Speedup” is a ratio = old exec time / new exec time

• “Improvement”, “Increase”, “Decrease” usually refer to percentage

relative to the baseline = (new perf – old perf) / old perf

• A program ran in 100 seconds on my old laptop and in 70 seconds

on my new laptop
1 1
▪ What is the speedup? (100/70) = 1.42 −
70 100 = 42%
▪ What is the percentage increase in performance? 1
▪ What is the percentage reduction in execution time?
100
100 – 70 = 30%
100
Improve Performance
❑ At Software level, change
◼ algorithm
◼ data structures
◼ programming language
◼ compiler
◼ OS parameters
❑ Take advantage of parallelism:
◼ At system level – Data Parallelism
⚫ Multiple processors, multiple disks : Scalability
◼ At processor level – Instruction Level Parallelism
⚫ Pipelining
◼ At block level - e.g., Set associative cache, carry look ahead
adder etc.
❑ Principle of locality: 10% code 90% of the time
◼ locality of reference – Temporal and spatial – Levels of Cache
memory
❑ Make common case fast
Quantitative principle of computer design

❑ Make the common case fast

◼ Improve the execution time of frequently occurring operations.

❑ Make design decisions that make frequently used

operations fast, even if they make some infrequently
used operations slow.

❑ Limits of improvement
◼ Improvement is limited by how frequent the frequent case is!
Make the Common Case Fast!
Most pervasive principle in design
Common case
◼ Need to validate what is common or uncommon
◼ H/W that isn’t used still costs you
◼ S/W done right that isn’t used probably doesn’t cost you

It is often the case that

◼ Common cases are simpler than uncommon ones
◼ Truly simple is often both cheap and fast.
Amdahl’s Law
Quantification of the diminishing return principle
Defines speedup gained from a particular feature
Execution time without using the enhancement
Speedup =
Execution time using the enhancement

Exec-time = 1/performance
Depends on 2 factors
◼ Fraction of original computation time that can take advantage
of the enhancement
◼ Level of improvement of the enhancement
 Amdahl’s Law
Execution Time without Enhancemen t Execution Time old
Speedup = =
Execution Time with Enhancemen t Execution Time new

What if enhancement does not enhance everything?

Execution Time without using Enhancemen t at all

Speedup =
Execution Time using Enhancemen t when Possible

 
= Execution Time old   (1 − Fraction Enhanced ) +
Fraction Enhanced
Execution Time new 

 SpeedupEnhanced 

1
Overall Speedup =
 
 (1 − Fraction Enhanced ) +
Fraction Enhanced

 Speedup Enhanced 
Caution: fraction
of What?
 Amdahl’s Law

S = Speedupenhanced
F = Fractionenhanced
 Amdahl’s Law
❑ Make the Common Case Fast
1
Overall Speedup =
 
 (1 − Fraction Enhanced ) +
Fraction Enhanced

 Speedup Enhanced 

Speedup Enhanced = 20
SpeedupEnhanced = 1.2
Fraction Enhanced = 0.1 VS
FractionEnhanced = 0.9
1
Speedup = = 1.105 Speedup =
1
= 1.176
 (1 − 0.1) + 0.1   0 .9 

 20 
  (1 − 0.9) + 
 1 . 2 

Important: Principle of locality

In general , Approx. 90% of the time spent in 10% of the code
 Amdahl’s Law (3)
❑ Diminishing Returns

Generation 1
Total Execution Time SpeedupGreen = 2
Green Phase Blue Phase FractionGreen =
1
2
Generation 2 SpeedupOverall = 1.33 over Generation 1
Total Execution Time SpeedupGreen = 2
Green Blue FractionGreen =
1
3
Generation 3 SpeedupOverall = 1.2 over Generation 2
Total Execution Time
Blue
Simple Example
In an application:
◼ FPSQRT is 20%
◼ FP instructions account for 50% of instructions
◼ Other 30%
The following are possible (at the same cost)
◼ Improve FPSQRT by 40x
◼ improve all FP by 2x
◼ Improve other (other than FP instructions) by 8x
Which one should you do?
And the answer for Speedup is…

1
FPSQRT = = 1.242
(1-0.2) + 0.2
40

1
FP = = 1.538
(1-0.7) + 0.7
2

1
= = 1.356
OTHER 0.3
(1-0.3) +
8
Another Amdahl’s Example
Suppose application is “almost all” parallel: 90%
◼ What is the speedup using 10, 100, and 1000
processors?

# of processors = P 1
Speedup p =
Fraction enhanced = 0.9 0.9
0.1 +
P

1
1
Speedup 10 = = = 5.3
0.9 0.19
0.1 +
10
1 1
Speedup 100 = = = 9.1
0.9 0.109
0.1 +
100

Speedup 1000 = 9.9

Example contd.
1 1
Speedup 100 = = = 9.1
0.9 0.109
0.1 +
100

What if you speed up the non-parallel part by a factor of 2?

1
Speedup 100+2 = = 1
= 16.95
0.1 + 0.9 0.05 + 0.009
2 100
Return on Investment
❑ What does it cost me to add some performance
enhancement?
❑ How much effective performance do I get out of it?
◼ 100% speedup for small fraction of time wasn’t a big win

❑ How much more do I have to charge for it?

◼ Extra development, testing, marketing costs

❑ How much more can I charge for it?

◼ Does the market even care?

❑ How does the price change affect volume?

Summary: Evaluating ISAs
❑ Design-time metrics:
◼ Can it be implemented, in how long, at what cost?
◼ Can it be programmed? Ease of compilation?

❑ Static Metrics:
◼ How many bytes does the program occupy in memory?

❑ Dynamic Metrics:
◼ How many instructions are executed? How many bytes does the
processor fetch to execute the program?
◼ How many clocks are required per instruction? CPI
◼ How "lean" a clock is practical?
Best Metric: Time to execute the program!

Depends on the instructions set, the

processor organization, and compilation Inst. Count Cycle Time
techniques.