Computer Architecture Essesntials by

ARM
Module 1, Intro
[music] I'm Richard Grisenthwaite, Chief Architect at Arm. What does a Chief
Architect do? The Chief Architect is responsible for the evolution of the Arm
architecture. One of the things about an architecture is, you want your
software to continue running on future products. So it's a very long term
commitment, and the role of the chief architect is essentially to curate the
architecture over time, adding new features as needed by the partner. Why are
microprocessors so important and ubiquitous? I think the fundamental reason
is that they are a general purpose device. They describe a basic instruction
set that does the stuff that you can construct pretty much any program out
of. Indeed, they're Turing complete, which means, by definition, you can
create any computable problem, solving a computable problem with the
processor. They're not customized to any particular usage, but they do two
things really rather well, which are data processing and decision making. In
other words, having, for example, added two numbers and compared them, you're
then taking a branch and making a decision based off that. Now in reality, an
awful lot of general purpose problems are ones that involve: work out some
value or some criteria, compare against that, and then make a decision on it.
And essentially that allows you to solve a huge number of different problems
and that's why the microprocessor has become so ubiquitous. What is the
difference between architecture and microarchitecture? Architecture is what
it has to do, and microarchitecture is how it does it. So for example, we
will define in the architecture a set of instructions that do "add" or "load"
or "branch", but it says nothing about whether you've got a ten-stage
pipeline or a three-stage pipeline. It says nothing about branch prediction.
All of those sort of features which actually determine the performance of the
device; all of those are part of the microarchitecture. The key point is that
software that is written to be compliant on the architecture will actually
run on lots of different implementations, lots of different
microarchitectures that essentially implement that architecture in different
ways, choosing a trade-off between the power, the performance and the area of
the design. What does it take to build a commercially successful processor
today? If you actually start from scratch with an established architecture,
but we want to create a new microarchitecture, we reckon, for a high end
processor, you're talking about three-to-four hundred person-years' worth of
work in order to create a properly competitive multi-issue out-of-order
machine compared with the state-of-the-art that you can get from Arm. In
addition to that — and that's just to come up with the RTL — in addition to
that, you've then got to do the implementation. If you're going to build that
on a three-nanometer process, the leading edge to get the best possible
performance, you're talking about tens of millions of dollars for the mask
sets. There's a whole bunch of software you've got to go and build on top of
that and do all of that work. When we've looked at companies that are
interested in actually starting up, taking an Arm architecture license — say
I want to go and build my own business — we reckon that you need to be
prepared to spend getting on for half a billion dollars before you're
actually going to be successful because it takes time. Your first product is
not necessarily going to be fully competitive because it would be slightly
surprising if the first thing that you built was as good as what people have
been building for many years. It takes time to build up that expertise and
skills. And so you're going to see a couple of iterations before even the
best teams end up really being competitive. And so as I say, I think if you
went from nothing and wanted to essentially create something, and that's
using the Arm architecture with all of its existing software and so on,
you're talking the best part of half a billion. What is the Arm business
model? The Arm business model fundamentally is the licensing of semiconductor
IP to as wide a range of companies as possible and in as many ways as
possible, in order to maximize the uptake of the IP. When we talk about IP,
we're talking about essentially designs for processors and we license that
both as an architecture license, where you effectively gain the right to
build your own processor to the specification of the Arm architecture, or an
implementation license where we are licensing, actually, an implementation
that is compliant with the Arm architecture in the form of some richer
transfer-level RTL. What makes Arm different from its competitors? So if you
go back to where we were when Arms started out in the early 90s, there were
many, many different architectures available and they were all kind of doing
the same thing but slightly differently. They would have different
instructions, different instruction sets, and so software needed to be
ported. A huge part of Arm's success actually comes from the fact that we
created a business model of licensing the IP to make it very easy for people
to build processors, to incorporate the designs into their SoCs, into their
systems. And that then made it very straightforward for people to be able to
use the Arm architecture. Now what this then meant was, people said: I will
port more software to this because it's more widely available and you get
this positive feed-forward effect whereby more availability of hardware
encourages more software, encourages more hardware, and so on. And
essentially that meant that a lot of people said: there's no point in me
having a different architecture. I'm not getting a valuable difference from
doing that. All I've got is, kind of, a needless change to the software that
I need to make. So actually the whole model, we went on at Arm, which was:
let's license our IP to a great number of different players to come up with
different solutions to meet the form factor of a camera or a PDA, or whatever
it was back in the day. Those things made it much more straightforward for
people to incorporate our technology. [music]

Module 1, Video 1
Computer architecture is the study of tools and techniques that help us to
design computers. More precisely, it helps us understand how to meet the
needs of particular markets and applications, using the technology and
components that are available. For example, we might need to produce the chip
at the heart of a smartphone using 10 billion transistors and a power of less
than two Watts. How do we achieve the performance a customer wants? The
challenge is a fascinating one, and one that requires a broad understanding.
For example, what target market are we designing for? What are the
characteristics of the applications to be run? How will programming languages
and compilers interact with the microprocessor? How best to craft the narrow
interface between the hardware and software? How to organize the components
of our microprocessor? And how to design a circuit, given the characteristics
of individual transistors and wires? Like many design problems, computer
architecture requires many trade-offs to be made and evaluated. Each design
decision will impact trade-offs between size, performance, power, security,
complexity, and cost. Trade-offs must be re-evaluated regularly, due to
advances in fabrication technology, applications, and computer architecture.
Computer architecture must be grounded in quantitative techniques and
experimentation, but the endless number of possible designs means that the
field depends on a high degree of human ingenuity and art. Perhaps
surprisingly, the earliest computers and today's most advanced machines have
much in common. They both execute a stored program constructed from machine
instructions. These instructions perform simple operations such as adding two
numbers. Nevertheless, greater numbers of faster transistors, and the
application of a relatively small number of computer architecture concepts,
have enabled us to construct machines that can perform billions of
instructions per second, and shrink these machines to fit in hand-held
battery-powered devices. It is this rapid progress that has supported
breakthroughs in machine learning, drug discovery, climate modeling, and
supports our modern world where computation and storage are almost free. The
task of designing a microprocessor is split into different levels of
abstraction: "Architecture;" "Microarchitecture;" and "Implementation."
"Architecture" focuses on the contract between programmers and hardware. It
allows compatible families of microprocessor products to be built. The ARMv8-
A architecture is an example of this. Architecture includes the "Instruction
Set Architecture," or ISA, which defines what instructions exist. It also
defines precisely the behavior of memory and other features needed to build a
complete processor. "Microarchitecture" focuses on the organization and
structure of the major components of a microprocessor. It has to match the
rules set by the architecture. The microarchitecture still has flexibility
though; and so the implementation specifies the circuit detail precisely.
This culminates in the exact circuit design for manufacture. Each of these
levels is vital, and each comes with its own challenges and opportunities.

Module 1, Video 2
At the beginning of the 20th century, "computers" were people employed to
perform calculations. These computers used mechanical calculators to help
them perform arithmetic. They followed instructions to decide what
calculation to perform next. These instructions defined the algorithm or
program they were executing. They would consult paper records to find the
inputs to their calculations, and would store their intermediate results on
paper so they could refer back to them later. A modern electronic computer is
organized in a similar way. We will look in more detail at these components
of a microprocessor in the next video, but for now let's look at how it
operates. Microprocessors also follow instructions one by one, and then
perform relevant calculations. This idea of fetching instructions from memory
and executing them is called the "Fetch-Execute Cycle." In the "Fetch" stage,
the computer reads the next instruction from the program. This instruction is
encoded in binary as ones and zeroes, so it must be decoded to understand
what it means. This is done in the "Decode" stage. Once it is clear what to
do, we move to the "Execute" phase. This can involve different tasks such as
reading memory, performing a calculation, and storing a result. Once done,
the computer is then ready to begin the cycle again, by fetching the next
instruction. Instructions are normally fetched sequentially in order, but
some special instructions called "branches" can change which instruction will
be executed next. For branches, a calculation determines the next
instruction. This can mean evaluating a condition, or reading a register to
determine the next instruction's location. Branches allow computers to make
decisions, and to re-use instruction sequences for common tasks. A modern
computer program, like a web browser, contains millions of instructions, and
computers execute billions of instructions per second, but they all
conceptually follow this "Fetch-Execute Cycle."

Module 1, Video 3
Modern microprocessors are circuits built using anywhere from 1,000 to 100
billion tiny transistors. The key to designing circuits with such huge
numbers of parts is to build them from re-usable blocks. Let's take a look at
some of these. Each transistor is an electrically-controlled switch. When
there is too little voltage at the gate, the switch is off, so the electrical
signal cannot propagate from the drain to the source. However, when there is
sufficient voltage at the gate, the switch is on, so the signal does
propagate. When designing processors, we use digital electronics. The only
voltage or current values we consider represent zero and one, enabling us to
build robust circuits from imprecise components, even in the event of
electrical noise or manufacturing imperfections. This means our circuits
represent binary numbers, since they only have two states: zero and one. We
use transistors to build increasingly complex circuits. We can design
circuits that can remember binary values, or select between multiple inputs,
or even do arithmetic, like addition. We can then use those circuits as
building blocks in even larger designs. When designing a digital system, we
must keep everything synchronized to control when its behavior occurs. To do
this, we use a "Clock Signal," which is a wire in the circuit whose signal
cycles between zero and one. We measure the rate in Hertz. For example, if
you hear a processor has a clock speed of two Gigahertz, it means 2 billion
ones and zeroes per second. The maximum speed of the clock signal is
determined by the longest, and therefore slowest, path in the circuit between
2 clocked flip-flops. This is referred to as the "Critical Path." The signal
must have time to propagate all the way along the critical path before the
clock cycle completes. A microprocessor needs to do many types of arithmetic.
For this, we build an "Arithmetic Logic Unit" or "ALU." This circuit receives
2 numbers as input, as well as an indication of what operation to perform,
for example, addition or subtraction. In addition to logic, a microprocessor
needs memory. Memory is organized as arrays of memory cells that are able to
store many "Words" of data. A specific word, commonly 32 bits in length, can
be accessed by specifying its "Address." Each address is a number that
indicates the location in the memory that should be read or written. Memory
cells range from hundreds of bits to millions of bits in size, but larger
ones are slower to access, as signals and their long internal wires take
longer to propagate. For that reason, almost all microprocessors include at
least two types for storing data: a big slow "Data Memory," and a small fast
memory called a "Register File." In reality, the "Data Memory" may be
implemented using many different sizes of memory, as we'll see in Module 4.
As well as storing data in memory, we also use some memory to store the
instructions. We need a way to keep track of which instruction we will fetch
next, so we have a "Program Counter." This stores the address in Instruction
Memory of the next instruction to be accessed. Since instructions are encoded
in binary, we also have "Instruction Decode Logic" that converts that binary
to the various signals needed to control the microprocessor.

Module 1, Video 4
We've already seen how a microprocessor is controlled by instructions. But
what are they really? An instruction is a simple command that the
microprocessor hardware can perform directly. We write them as text like this
to make them easier to read. But for the microprocessor we encode them in
binary. We use a program called an assembler to translate between the human
text and the binary. In this video we'll be looking specifically at an Arm
instruction set called A64 but other instruction sets follow similar
principles. Arithmetic and logic instructions are the simplest type of
instruction. The first word tells us what operation will be performed such as
addition or multiplication. The values after this tell the processor where to
put the result and where to get the inputs. Values starting with X are
addresses in the register file. Arithmetic instructions read one or two
registers and then put the result into a third register. Branch instructions
are used to make decisions and to repeat instructions. Normally the
microprocessor executes instructions in sequential order but branches change
that, and explicitly tell the microprocessor the address of the instruction
to run next. This is done by giving the address of the next instruction in
the instruction memory. Some branches are unconditional, meaning they always
occur and always affect the next instruction address. Other branches are
conditional, meaning the processor will perform a calculation to decide
whether to follow the branch or to continue executing instructions
sequentially following the branch. These are preceded by comparison
instruction to calculate the condition. Loads and stores are the instructions
for accessing the data memory. Loads copy values from memory to the register
file. Stores do the opposite. In both cases, the instruction needs to know
the address in the data memory and the location in the register file to copy
between. For data memory, loads and stores read an address from a base
register. They can also optionally add to this base address by reading
another register, or by simply specifying a number in the instruction itself.
Using sequences of instructions we can build programs. Here is an example of
a small program that implements Euclid's greatest common divisor algorithm.
Let's take a look at it working, one instruction at a time. To start with,
the input stored in X1 and X2 are compared. If they're equal, because we have
found the greatest common divisor a conditional branch instruction moves to
instruction 7. If they're not equal, another conditional branch instruction
can be used to determine whether X1 is smaller. Finally, we use an arithmetic
instruction to subtract either X1 or X2 depending on which was larger and
then unconditionally branch back to the start. Here we can see an example of
the program running. One instruction executes at a time. As we step through
the program, the values in the registers X1 and X2 are shown. And we see
these are modified each time we reach instruction 3 or 5. The instructions
repeat in a loop, as is common for many different types of program. Each time
round the loop instruction 0 and 1 are checking whether or not to continue
the loop. When they detect X1 and X2 have become equal, the program finishes
and the processor moves on to the next task.

Module 1, Lab
In this exercise, we're going to be using ASim. ASim is a behavioral
simulator for a subset of the Arm AArch64 instruction set. Behavioral
simulator means that it's not simulating real circuit level details of a
microprocessor it's just simulating the behavior of each instruction as it
runs. Nevertheless, behavioral simulators are vital tools for computer
architects. We use them to check the designs we've built, match with the
behavior we intended and we can also use them to explore new ideas, for
example adding new instructions to the instruction set. We're just going to
be using it though to get familiar with the basics of Arm AArch64. So we can
create a new file which will allow us to type in some Arm AArch64
instructions and when we press assemble that will cause ASim to begin the
simulation of the instructions. If we scroll down we can see that below we've
put a little cheat sheet of Arm AArch64 instructions that you can use as a
reference. Do note that these are just examples though. So for example this
first one is mov X1 X2 and the descriptive text says that this copies the
value of register X2 to register X1. But equally that would imply that if we
use mov X3, X4 instead that would copy the value of X4 to the register X3. So
we don't have to just use these exact instructions, we can tweak them as we
need to, to achieve our goals. So going back to the ASim, let's say that we
wanted to add the number two to the number three. In our cheat sheet we could
see that there is an 'add' instruction but it requires that the inputs for
the addition are stored in registers. So first of all, we're going to need to
load up two registers with the numbers that we want to add in this case two
and three. So looking at our cheat sheet again we can see that there's an
instruction mov that allows us to do that. So if I write mov X2, #3 this
would cause the register X2 to be loaded with the value 3 and similarly I
could write mov X1,#2 and this would cause the register X2 X1 to be loaded
with the value two. Last but not least, we could then do the addition we
wanted to do, such as add X3 X1 X2. This will cause X1 to be added to X2 and
the results stored in X3. If I press assemble now, we will see that the UI
changes slightly. Here we can now see the simulated state of the processor.
ASim is only going to, because it's a behavioral simulator it's only going to
show the state before and after each instruction. And so right now we are
before executing the first instruction. That's the yellow highlighting there
and we can also see it in the simulated memory. If I press step we can now
see that that instruction has completed, and it's before executing the mov X1
X2. And notably we can see that X2 has been loaded with the number 3 which is
exactly what we would expect. If I press step again, we can see that mov X1
X2 has now occurred. MOV X1 #2, sorry has now occurred which has loaded the
value 2 into the register X one. And last but not least, if we step again we
see that X3 has become equal to five which is exactly what we would expect if
registry X1 was added to registry X2. So this allows us to get a feel for
what it's like for a machine to execute these instructions. We can reset back
to the beginning if we want to watch it go through again. If we want to do an
edit for example adding a new instruction we can just do that. Press assemble
and that will add the new instruction to our program, and we can see its
effects by simulating. Now in the exercise you will be invited to take an
existing program and add 1 new instruction to it at the indicated position.
Do note that when you assemble the exercise program, you'll be taken to a
gray bit of code, which is our testing code. But as you step, you'll see that
the simulator flicks between the testing code and the code that we wrote the
exercise one code. You can also click between them using this. Feel free to
try to work out what our testing code does but you can just ignore it if you
want to. The point of the exercise is just to add the instruction in the
indicated position. When you're comfortable you've got the right instruction
you can press run to get all the way to the end. And if you really think
that's correct if you scroll down to the bottom of the page you'll see the
submit button which you can use to send in your answer. Best of luck.

Module 2, Intro
[music] Hello, my name is Martin Weidmann. I'm an engineer and product
manager with Arm's Architecture and Technology Group. I look after the A-
profile architecture and I maintain Arm's Interrupt Controller
specifications. Computer architecture is sometimes called a science of trade-
offs. Why is everything a trade-off when it comes to designing a processor?
Let's take an example of where we've had to make trade-offs when developing
process architecture. So the Arm architecture has a feature called Pointer
Authentication. This is often abbreviated to PAC for Pointer Authentication
Code. What this feature is trying to do is protect against a form of attack
called ROP and JOP. These are Return Orientated and Jump Orientated
Programming, and it's where an attacker tries to subvert things like the call
stack to run legitimate code, but in ways that weren't expected by the
programmer or the compiler. PAC or Pointer Authentication tries to defend
against this kind of attack by using part of an address to provide an
encrypted signature. So we can check the signature and the address match and
if they don't, we can spot an attack in progress. So why is this a trade-off?
Well, because to add security, we want that signature to be as big as
possible. The bigger the signature, the more bits we use for that, the
stronger cryptographically that signature is. The trade-off is: the more bits
we use for the signature, the fewer bits we have available for other things,
such as the address. So you can have a big signature with a small address,
but if you want the address to get bigger, then you get a smaller signature,
and that's then cryptographically weaker. So the trade-off we have to make
when designing a technology like that is: What's the right amount of bits for
the signature? What's the strength of cryptography we need from that
signature in order to get the design goal, which is to defeat these attacks
and give us more robust computing? What sort of guiding principles do you use
when designing a processor? So when you're designing a processor, the key
thing you have to bear in mind is: What's it going to be used for? What you
don't want to end up with is a very expensive paperweight. So we need to
understand the requirements that the processor has to meet. We have to
understand the design trade-offs we're making and how they work into meeting
those requirements. We also have to consider not just the processor itself,
but how we're going to show that that processor is correct. We'll put as much
time, if not more, into testing and validating the design as we do into
designing it. How do you design a new microprocessor? If you wanted to create
a new processor from scratch, the first thing you're going to have to do is
understand the market that that processor is going to address and to then
build a team to design that processor. There isn't such a thing as one
processor for every possible market. The requirements for something like an
embedded microcontroller are going to be very different to what you want from
the processor in your mobile phone, your laptop, or your server. So you need
to understand those requirements as the first step into building a new
processor. What determines the best design for a microprocessor? So when
you're designing a processor, you need to work out what the best design for a
given market or application is going to be. There's no magic formula for
this. It's going to depend a lot on what you're trying to achieve with that
processor. You need to understand things like the power requirements, the
performance requirements. Is it going to work in a highly noisy electrical
environment? There's a big difference between the reliability requirements
you need from something like a watch versus a satellite. So you would take
those requirements and you'd work out what the best set of trade-offs is
going to be, and that's an art more than it is a science. How do the
underlying technologies contribute to this best design? A lot of technologies
go into our processor. There's the design of the microarchitecture, the
implementation of the processor. There's the silicon process you're going to
use, how you're going to integrate that processor into an SoC or ASIC. Is it
a single die, or is it going to be using chiplets or multiple sockets? All of
those different technologies are going to be factors into how you design the
process or what trade-offs you make, and what performance and power you get
out of the design once you're done. In reality there may be many different
'best' designs, so how do you pick one? So when you're designing a processor,
what you want is the best design. But often there isn't "a" best design,
there's just different trade-offs. You have to decide what the best set of
trade-offs is for the particular use case you're going for. And that's also
going to depend on: Is this a device that will be off the shelf, used for
lots of different applications—a general purpose processor? Or is this being
designed for one specific use case? Again, there isn't really a magic bullet
or single answer for this type of question. You need to understand how the
processor is going to be used and then use your experience to judge the
trade-offs, and what will give you the best mix of power, performance, area,
cost, reliability for your target use case.

Module 2, Video 1
[music] In this module, we're going to explore how to improve the simple
microprocessor design from Module 1 in order to allow it to execute programs
more efficiently. First, let's find out how long a program takes to execute.
The time taken to perform the average instruction is equal to the number of
clock cycles taken to perform an instruction multiplied by the duration of
one clock cycle. The time taken to run our program is found by multiplying
the average time to perform an instruction by the number of instructions in
our program. How could we make this faster? One thing we could try is to
reduce the number of instructions in a program. We might be able to optimize
the code removing unnecessary and repeated work and selecting instructions to
minimize code size and maximize performance. We could give our microprocessor
the ability to perform more operations in order to help programmers or
compilers further reduce the number of instructions in their program. For
example, allowing the loading of two data values at the same time might allow
fewer instructions to be used in the program. The downside to this approach,
is that adding more instructions will require extra circuitry in the
processor and therefore we likely increase the clock period. If the extra
instructions are rarely used this could even mean an overall decrease in
performance. We see this theme often in computer architecture trade-offs that
we have to carefully balance. Another approach is to use faster transistors
perhaps constructed from a more recent fabrication technology. This would
reduce the clock period but may increase costs. The rest of this module
focuses on an optimization to reduce the clock period called pipelining. This
is the most important optimization we use when designing processors. It uses
a similar concept to an assembly line in a factory where work can start on
the next item before the previous one finishes. Let's take a closer look.
Imagine that each instruction has to go through four circuits in a processor.
If we attempt to do all of these in one clock cycle this means our clock
period is the latency of all four circuits added together. If we were to
pipeline this, we would add a pipeline register in the middle. This divides
the circuit into two sections called stages. Notice that although each
instruction takes a similar amount of time to travel down the whole pipeline,
the pipeline design can execute nearly twice as many instructions per second.
The throughput has doubled. This is because we can set the clock period much
shorter. It's now the maximum latency of the two stages. We can pipeline into
many stages and this allows for much faster execution of programs.
Unfortunately, though, pipelining a real microprocessor design is not quite
as simple because the processor has various feedback signals and loops in the
circuit. In the next video, we'll take a look at the challenges of pipelining
in practice. [music]

Module 2, Video 2
[music] In this video, we're going to look at applying the pipeline
optimization to a realistic microprocessor design. In the first module, we
met the components of a microprocessor, so let's look at how these are really
connected. This diagram shows all the connections needed for a real,
unpipelined microprocessor. Each clock cycle, the processor starts by
fetching an instruction from the instruction memory. Once the instruction
reaches the decode logic, it is decoded to produce the control signals
necessary to execute it. The exact control signals vary depending on the type
of instruction. For example, arithmetic instructions access the register file
and interact with the ALU. Ultimately, no matter how the instruction was
executed, the last step of each clock cycle is to update the program counter.
This is done by the branch unit. For non-branch instructions, this just means
incrementing the program counter. However, for branch instructions, the
branch unit has to do some calculations. When we apply our pipeline
optimization to this design, we face some challenges. The design has several
loops because instructions have dependencies. How can we break these cycles?
The key observation is that not every instruction is the same. In real
programs, branch instructions usually make up less than 20 percent of the
program. For non-branches, the branch unit doesn't actually need to wait for
the ALU before calculating the result. Let's look at how we can use this fact
to pipeline the processor. Once the first instruction shown in yellow reaches
the pipeline register, we're ready to begin fetching the next instruction,
shown in blue. The yellow instruction can be in the execute stage whilst the
blue instruction is being fetched. Once the yellow instruction is finished,
the blue instruction is ready to enter the execute stage and a new green
instruction enters the fetch stage. What about the branches though? Let's
imagine this next yellow instruction is a branch. The fetch stage works
normally until the branch unit, but the branch unit cannot proceed.
Consequently, the pipeline stalls. The fetch stage spends a cycle waiting
whilst the execute stage executes the branch. Finally, once the ALU is done,
the branch unit can proceed and the next instruction, in this case blue, can
be fetched. Overall, this means that the processor wasted one cycle stalling
due to the branch. Since only 20 percent of instructions are branches, this
means that each instruction would require on average 1.2 cycles. The same
idea of stalling the pipeline can be used to create even longer pipeline
designs. This diagram shows a typical five-stage processor pipeline. In the
next video, we'll look at how we can manage or prevent some of the stalls in
a design like this. [music]

Module 2, Video 3
[music] Instructions within a program may be dependent on each other. That
is, one instruction may produce a value that a subsequent instruction
consumes. Data values may be communicated through registers or memory. The
simple program shown has a number of so-called true data dependencies. This
means we must take care to execute these instructions in order, and make sure
results are correctly communicated. Additionally, the outcomes of branch
instructions may affect the path taken through the program, and consequently,
this affects whether an instruction is actually executed. This sort of
dependency is known as a control dependency. In the previous video, we met a
realistic processor pipeline with five stages. Circumstances that prevent an
instruction making progress in our pipeline are known as pipeline hazards.
Let's take a look at how dependencies cause hazards. This program has a true
data dependency. The first instruction writes to register one, which is then
read by the second instruction. If we send this down our pipeline, we see
that the second instruction must stall, waiting for register one to be
written, before it can read and proceed. This is a data hazard.
Unfortunately, dependent instructions are common and stalling in this way
would significantly increase the average cycles per instruction. Let's take a
closer look at the hazard though. The ADD instruction is in the execute
stage, meaning its result is being computed. The SUB instruction needs that
result to proceed. Rather than waiting for the ADD to reach the writeback
stage, we could add an extra path into our pipeline to carry the output of
one stage to a later instruction, making the result available straight away.
We call this a forwarding path. In this case, the ALU result is forwarded to
the SUB instruction to be used as X1. This small piece of extra circuitry
allows this data hazard to be eliminated completely. Unfortunately, even if
we add forwarding paths everywhere, it's not possible to eliminate all data
hazards. For example, this program has a data hazard due to the load
instruction. There are other types of hazard too. This program contains a
control hazard. We cannot be sure which instruction to fetch until after the
branch instruction executes. Consequently, this program has two stall cycles.
We will look in more detail at how control hazards can be mitigated in the
next module. Another class of hazards, called "structural hazards" occur when
two instructions require the same resources simultaneously. For example, if
instructions and data were stored in the same memory, and this could only be
accessed once per cycle, we would have to very frequently stall our pipeline
to let these stages access memory one-by-one. [music]

Module 2, Video 4
[music] In the previous videos, we explored how pipelining could improve
performance by reducing our clock period and by overlapping the execution of
different instructions. We also saw that it was sometimes necessary to stall
our pipeline to ensure that instructions were executed correctly. Ideally,
our average cycles per instruction, or CPI, will remain at 1.0. If we must
stall, however, this will increase. For example, if 20 percent of our
instructions were loads and each of these caused one stall cycle, our CPI
would be 1.2. If a further 20 percent of instructions were branches, and each
of these caused two stall cycles, our CPI would be 1.6. The longer we make
our pipeline, the more stall cycles there will be, and eventually the cost of
stalls may outweigh the benefit of the faster clock period. For example,
let's imagine we added a stage to our five-stage pipeline from the previous
video. Now the number of stalls after a branch instruction increases to
three, hurting our CPI. On the other hand, our clock period would improve. So
whether or not this helps speed program execution would depend on the exact
details. It may eventually become more difficult to reduce our clock period
by adding further pipelining stages. This is because it becomes harder to
perfectly balance the logic between stages and because of the constant delays
associated with clocking and our pipeline registers. To mitigate these
issues, we will need to invest in more transistors and our design will
require more area and power. The deeper our pipeline gets, the greater the
investment we need to make in terms of area and power for the same
incremental improvement. Commercial processes today have anywhere from two to
twenty pipeline stages. The faster, more expensive and power-hungry
processors tend to have longer pipelines than the smaller, cheaper processes
in embedded devices. As with many techniques in computer architecture,
eventually it becomes more profitable to invest our time and resources in an
alternative way of improving performance. In later modules, we'll explore how
we can reduce the CPI, even in heavily pipelined processes. [music]

Module 2, Lab
[music] In this exercise, we're going to be using a model of a processor
pipeline to explore the effect of the pipelining optimization. Computer
architects use models like this to make high level decisions early on about
what parameters they will use for a processor and using a model such as this
saves the burden of actually building the processor to find out its
performance. The model is not simulating accurately the performance of the
processor but rather it's giving us an idea for what performance we might
expect. So what can we do with this model? Well, we can configure the number
of pipeline stages, which we can see affects the diagram. And we can also
turn on or off the forwarding optimization. As we change these numbers notice
that the design parameters change down here. So for example, the clock
frequency is improved by increasing the number of pipeline stages but the
design area will get bigger. And so this may be a consideration depending on
the problem. We can also choose which of two programs we're going to put down
our pipeline. When we press the step forward button the pipeline advances to
the next clock cycle and we can see the instructions have started to flow
down our pipeline and interesting events such as forwarding will be noted in
the simulation. Sometimes the simulation will detect that there will be a
stall for example, in this case, we can see that there is a data hazard
because the instruction in the red memory stage writes to register X13 which
is read by the instruction in the yellow decode stage and therefore a stall
cycle is necessary in order to allow the result to be generated. If we press
the play button, the simulation will proceed automatically and we can see
various stall events happening as the simulation proceeds. But notice that
the the program we're simulating is nearly 1,000,000 cycles long so watching
it play out at this speed is going to take quite a while. So we can use the
fast forward slider to simulate much, much faster. Notice that the statistics
down the bottom have updated depending on the results of the simulation, and
at this point we can see that the program is finished and the simulation of
the program, the simulated program took 3.98 milliseconds to execute. We can
also see that just below, the results of past simulations are stored in
little tables so we can easily refer back to them when we're doing later
experiments. So as an experiment, let's imagine what would happen if we
disabled the forwarding optimization but change nothing else and we'll just
run this program through. What we can see immediately is the design side is
slightly better which is what we would expect. It's got 1% better in fact in
this case because of the lack of the forwarding wires. But now that the
program is finished, we can see that the program execution time is a lot
worse. 6.34 milliseconds is about 50% worse. So again, looking in our table,
we can compare the execution times in the area and we can see that in most
cases the forwarding optimization would be a big optimization here because at
the cost of an increase in the area of about 1%, we've had a improvement in
execution time of about 50% which is likely to be a good trade off, but not
always. It would depend on the exact scenario. Perhaps that 1% area is more
important than the performance of this program. In the exercise, you'll be
invited to design a or suggest a design using the number of pipeline stages
and whether forwarding is enabled that will meet certain criteria. You can
play about and do as many simulations as you wish to figure out what the best
program might be. Once you've got it set up select the processor that you're
happy with at the top and then scroll down to the submit button and press
that. Good luck. [music]

Module 3, Intro
[music] Hi, I'm Nigel Stevens. I'm Lead Instruction Set Architect for the Arm
A-Profile architecture. I've been at Arm for about 14 years and I have
responsibility for the Arm V8-A instruction set including recent developments
such as the Scalable Vector Extension and Scalable Matrix Extension. What do
we mean by the instruction set architecture? The instructions in
architecture, primarily, most people think of I guess as the OP codes, the
encodings of instructions that are executed by an Arm-based processor. But it
also includes other aspects as well such as the exception model, system
programming features, memory management and suchlike. The architecture for
Arm is rather distinct from what other companies may call an architecture.
For Arm, architecture is a legal contract, if you will, between hardware and
software. If software uses only those instructional codes and features of the
ISA that are described by the Arm architecture to perform its work, and the
hardware that it's running on implements all of those op codes and features
exactly as defined by the architecture, then any architecturally compliant
software will run on any architecturally compliant hardware that implements
that Arm architecture. And that doesn't mean just processors from Arm itself,
but also processors that are designed by our partners and which we have
validated are conformant with our architecture. How do you decide which
instructions to include in an ISA? When we are looking at requests from
partners or from internal research to add a new instruction, we go through
quite a long process of trying to justify that instruction, or, quite
commonly, a set of instructions rather than a single instruction. We have to
show that it gives us some real benefit in performance, the performance of
your code running on that CPU. Or maybe not performance. Maybe it's security
you're trying to achieve. But it has to give you some really concrete benefit
that is worth the cost of adding all of the software, the validation
software, the implementation costs for all of the different implementations,
compiler support, so on and so forth. It has to meet the... It has to answer
that cost benefit analysis. What is the difference between an ISA and a
microarchitecture? The difference between an instruction set architecture, or
ISA, and the microarchitecture is that the ISA is an abstract concept. It
defines a set of instruction encodings which software can use, and which
hardware has to recognize and implement. How that is implemented is a choice
for the microarchitecture. So the instruction set architecture is fixed, it's
defined by Arm. The microarchitecture is defined by whatever team of people
is designing that CPU. And there are many different approaches to
implementing the Arm architecture, from very small, efficient cores with in-
order pipelines up to very high-performance, state-of-the-art, out-of-order
execution, and everywhere in between. So the microarchitecture is
implementation-specific, the architecture is generic, and software written
for the architecture should run on any microarchitecture. Why does Arm
produce processors with different instruction sets? Arm supports multiple
instruction sets. Some of that is to do with legacy: you can't abandon your
legacy software, your legacy ecosystem. So as the architecture has advanced
and we've introduced major new instruction sets, we still have to continue to
support old software. It takes years, maybe 10 years to move the software
ecosystem to a major new ISA. So for example, AArch64, which is the 64-bit
architecture that we introduced with Arm V8, also supported the AArch, what
we called AArch32, the old 32-bit architecture that was implemented in the
Arm V7 architecture, and prior to that including the Arm and the Thumb
instruction sets. And we needed to do that because, while some software might
start to migrate to the 64-bit architecture, there's still a lot of software
on the planet which is going to continue using the 32-bit architecture, and
that has to survive. So that's part of the reason: it's about legacy. You
can't just obsolete the whole world when you introduce a new architecture, a
new instruction set architecture in particular. There are other reasons as
well, which is there are certain instruction sets that are different for
reasons of the ecosystem that they're working with. So if you were to
compare, for example, the A-profile architecture that's designed for
application processors that run rich operating systems with virtual memory
supporting SMP, symmetric multi processing operating systems running large
applications, whatever it may be: web browsers on your phone or something, or
a web server in a server farm somewhere. You have your R-profile
architecture, which is designed for high-performance, real-time embedded
systems. The constraints there are somewhat different. The instruction set is
actually fairly similar to the A-profile, but some of the underpinnings of
the architecture, the system side of the architecture, are simplified in
order to give more consistent and predictable real-time response to things
like interrupts or memory translation and suchlike for real-time systems. And
then at the other extreme you have the M-profile architecture which is
designed to be capable of being built in a very simple, ultra-low power
implementation with low code size and again, similar to the R profile, very
predictable real-time performance. So the answer is there are different
instruction sets for reasons of the market that they're trying to address,
and then there are different instruction sets because, well, we have history.
[music]

Module 3, Video 1
[music] In the previous module, we explored how pipelining can be used to
improve performance. We also saw how it is sometimes necessary to stall our
pipeline to ensure our program is executed correctly. In a simple pipeline,
it will be necessary to stall the pipeline whenever we encounter a branch
instruction. This is because we must wait until our branch is executed before
we can be sure which instruction to fetch next. As a recap, branches are
instructions that change which instruction in the program will be executed
next. There are two types of branches: conditional branches and unconditional
branches. Unconditional branches always change which instruction executes
next, whereas conditional ones may or may not, depending on the computations
in the program. In real programs, between approximately one fifth and one
quarter of all instructions are branches, and the majority of these are
conditional. Executing a branch involves calculating the new address to load
into our program counter. This is the branch's "target address." However,
conditional branches have an extra task: we must first determine whether the
branch is taken. If the branch is not taken, we can effectively ignore the
branch and fetch the next instruction as normal. Recall the processor
performance equation from an earlier video. Since we have to wait for
branches to complete before fetching the next instruction, we generate stall
cycles. These increase the average number of "cycles per instruction," which
reduces our microprocessor's performance. The longer our pipeline gets, the
longer it is before each branch is resolved, and the more costly branches
become. Can you think of a way to avoid some of this stalling? One idea is to
evaluate branches earlier in the pipeline, for example in the Decode stage
instead of in the Execute stage. This can indeed help to reduce the number of
stalls, but we may still need to stall if the branch depends on other
instructions that haven't been executed yet. Another idea is to continue
fetching instructions in program order, effectively assuming that each branch
is not taken. The number of stalls in the pipeline for a not-taken branch is
zero in this design. On the other hand, if the branch is in fact taken, the
subsequent instructions that we fetched will be incorrect. So, we must remove
all instructions that have been fetched on this incorrect path from our
pipeline. This is called "flushing" the pipeline. Unfortunately, in real
programs, branches are taken much more than not taken. Could we then simply
assume instead that all branches will be taken? Sadly not, no, because then
we would also need to know the specific "target address" immediately, in
order to know which instruction to fetch next. It may at first seem
impossible to know this before the instruction is decoded. However, computer
architects have found a way to do exactly this. The next video will look at
"dynamic branch prediction:" the idea of predicting the behavior of the
branch instruction before it has even been fetched. [music]

Module 3, Video 2
[music] In this video, we'll explore how to predict the behavior of a branch
instruction. This can sometimes eliminate the cost of branches altogether.
Fundamentally, a branch involves changing the value of the program counter,
which is the address of the next instruction in the program. If we could
predict what this change will be, quickly and accurately, we would have no
need to stall. Precisely, we need to predict that we are fetching a branch,
predict it as taken or not taken, and predict what its target address is. How
could we ever make such predictions? What would we base them on? Well, since
instructions are often executed multiple times, we can accurately make these
predictions based on just the instruction address. If we've previously seen
that a particular address contains a branch, and we see that address again,
we can predict whether that branch will be taken, and its target address,
based on its behavior last time. Amazingly, for real programs, simply
predicting repeating behavior is typically around 90 percent accurate. This
means we could eliminate stalls caused by branch instructions 90 percent of
the time. Let's apply these insights to try to build a branch predictor. We
will add two extra blocks to the Fetch stage of the pipeline we met in Module
2. The first will remember information about recently executed branches. This
will include the program counter values of branches and their target
addresses. This memory is called the "Branch Target Buffer" or BTB. The
second block will make predictions about whether an address containing a
branch is taken or not. We simply call this the "branch predictor." In the
next video, we will look at these in more detail. Combining these two gives
us all the information we need to predict the next value of the program
counter based solely on the current value of the program counter. We don't
even need to decode the instruction to make this prediction. Here we can see
how a running branch predictor behaves for a sample program. Each program
counter is checked in the BTB to see if it's predicted to be a branch and to
identify its predicted target. We also simultaneously check the branch
predictor to see if the branch is predicted to be taken. Based on these
predictions, the branch unit computes the predicted next program counter.
Many cycles after the prediction, feedback will be given by the rest of the
pipeline as to whether or not the prediction was correct. Whenever the
prediction is wrong, we have to flush the pipeline and update the BTB and
branch predictor. The pipeline will then resume fetching from the correct
program counter as computed by the pipeline. The majority of instructions are
not branches, so most of the time the branch unit just adds 1 to the program
counter. The BTB contains the instruction address and target address of some
recently executed branches. The exact number of entries in the BTB varies
considerably. BTBs in large modern processors contain many thousands of
branches. In operation, the BTB checks the supply program counter against its
memory to see whether it has a match, and if so, it returns the target
address. Otherwise, it predicts that the instruction is not a branch. After
each branch executes, the BTB is updated with the true target address. BTB
cannot be arbitrarily large, so it may have to forget an existing branch to
remember a new one. A simple BTB design like this is typically around 90
percent accurate at predicting target addresses in real programs. [music]

Module 3, Video 3
[music] In the previous video, we met the two major components of dynamic
branch prediction, the BTB and the branch predictor. In this video, we'll
take a deeper look at the branch predictor. It predicts whether or not a
branch will be taken, based on the program counter. A simple branch predictor
would try to remember what the branch did last time and predict that the same
behavior will repeat. Let's see how such a predictor might be organized.
Remembering a branch prediction for every possible instruction address would
take up far too much memory, so we reuse the same memory for different
branches via a process called "hashing." We hash the address of the branch to
a smaller number. This does unfortunately lead to a problem called
"aliasing," where two different branches can hash to the same value, but this
is rare in practice. Let's see what happens now, when we execute a simple
loop. We see a misprediction when it encounters our branches for the first
time, and another when we exit the loop. The first case will be dependent on
the value in our predictor's memory, and it may be that we are able to
predict the branch correctly the first time we see it. The second case is
hard to avoid, although some more sophisticated branch predictors will learn
how many iterations a loop will make. A common improvement to this simple
scheme is to avoid instantly flipping our prediction just because the branch
does something unexpected once. This can be achieved with a saturating
counter, which instead remembers how many times the branch has been taken
recently, versus not taken. The counter increments when the branch is taken
and decrements when not taken. It predicts "taken" if the majority of recent
executions of this branch were taken. When building higher-performance
processors, we often have to discard many instructions every time we
mispredict a branch, so accurate branch prediction is very important.
Therefore, branch prediction is still an active area of research. One of the
key ideas used is correlation between branches. In real programs, a common
pattern is for example a pair of branches that always have opposite behavior.
A branch predictor can take advantage of this by remembering a history of
whether or not recent branches were taken or not taken and incorporating this
in the hash. Another idea is to combine multiple different types of predictor
in a "tournament predictor." We use a "meta predictor" to predict which of
two branch predictors will do a better job. As you can see, branch prediction
can get quite complicated. [music]

Module 3, Video 4
[music] In this module, we've met the concept of branch prediction. Even
using relatively small simple circuits, we can accurately predict real branch
behavior more than 95 percent of the time. Could we do better, and do we
really need to? In small, simple processors, these prediction accuracies are
fine, because each misprediction causes only a few stall cycles. Increasing
the accuracy is not that impactful. However, in complex processors with very
long pipelines, the difference between 98 percent and 99 percent prediction
accuracy can be significant for performance as a misprediction could incur
dozens of stall cycles. Accurate prediction really does matter. One of the
problems we face in making accurate predictors is that they need to be small
enough and fast enough to fit in a microprocessor. We can imagine all sorts
of ways to do accurate branch prediction, but if their circuit would be
slower than simply executing the branch, they would not be useful. Modern
high-performance processors will often have multiple branch predictors, for
example a small fast one and a slower complex one. The slower one can
override the prediction of the fast one if it thinks it got it wrong, which
does incur some stall cycles, but fewer than a total misprediction. Another
problem we face is that some branches are just very hard to predict. No
matter what technique is used, there are some branches in real programs that
are effectively "random". For example, when compressing or decompressing
data, the combination of the underlying algorithm and the input data may
provide no clear patterns to a prediction. No matter how hard we try, some
branches will never be predicted correctly 100 percent of the time. A final
problem is that since the predictors work based on observing the program,
there will always be a period of time when the predictors train on a program
to learn its behavior. [music]

Module 3, Lab
[music] In this exercise, we're going to be using a branch predictor
simulator to explore dynamic branch prediction. This simulator will
accurately simulate the details of a branch predictor but it uses a trace of
a real program executing on a real machine to avoid the need to simulate the
rest of the processor. Computer architects use techniques such as this to
quickly explore one area of process design understanding that perhaps the
accuracy may not be completely correct given that we're not simulating the
full process of design. The interface allows us to configure details of our
branch predictor for example the maximum of the saturating counters used in
the branch predictors table or the hash function. And we can see the impact
of changes on the delay of the branch predictor and also the design size
which are two key metrics when designing a branch predictor. Once we've
happily configured the design we want we can press run to simulate a program
and the results of that program's execution will be displayed in the rest of
the stats. So for example, we can see here that the predictor predicted
95.24% of the branches correctly. Which is fairly good and this resulted in
overall execution time of the program of 5.335 milliseconds. Just below we
can see a table that records previous simulation runs as well so we can do
multiple experiments and see which produces the best results. For example, if
we were curious about the effects of using a saturating counter with a
maximum of three rather than one we could change that and notice that the
design size has substantially increased and when we press run, we noticed
that the predictor accuracy, however, has also increased. It's gone up to
96.31% and consequently the execution time the program has fallen slightly
and so we can compare these two designs and see whether or not this
represents a good trade-off for our processor. Perhaps the area cost is
justified or perhaps it's too much. It would depend on the exact processor
we're trying to design In the problems you'll be invited in to come up with
designs that are suitable for particular constraints. For example, particular
constraints on the runtime of the program or the design size. Once you've
configured the branch predictor that you think meets the objectives you can
scroll down to the submit button and click that and you'll be told whether
the answer is great or not. Good luck. [music]

Module 4, Video 1
[music] So far, we've looked at the microprocessor's "datapath"— meaning its
execution units, registers, and control circuitry. We have given less
attention to its memory. We usually implement memory using a different type
of chip than the microprocessor, using a technology called DRAM. It is very
dense, allowing us to store lots of data in a small area. However, one issue
with DRAM is its speed. Since the 1980s, processor performance has increased
very rapidly at roughly 55 percent per year, so CPUs of today are many orders
of magnitude faster than those of 40 years ago. In contrast, memory
performance has grown much more modestly. Whilst memories are also much
faster than they were in previous decades, their performance has not kept
pace with processors. This leads to a processor-memory performance gap, with
it becoming more costly to access memory as time goes on. One of the issues
that makes these memories slow is their size. We can make a memory as fast as
our microprocessor, if it is very small. However, this is the opposite of
what programmers want. Programmers can use extra memory to solve more complex
problems, or to solve existing problems faster. This complicates
microprocessor design because although we want a large memory to hold all our
data, large memories are slow and this slows down our processor. How do we
overcome this? What if we could get the speed benefit of a small memory
alongside the size benefit of a large memory? One solution is to have a
large, slow memory and a small, fast memory in our system. For this to be
useful, we need the small memory to hold the data that we use most often, so
that we often get the speed benefits of accessing it, and only rarely have to
access the slow memory. Whilst there are different arrangements of these two
memories, the configuration that is most often used is an "on-chip cache
memory." The small memory sits inside the processor between the pipeline and
the large memory. We keep all data in the large main memory, and put copies
of often-used data in the small memory, which we call a cache. But we are not
limited to only one cache! Our pipeline reads memory in two places: when
fetching the instructions; and when accessing the data. It makes sense to
have two caches here, each optimized for different purposes. The instruction
cache is optimized for fast reading of instructions at Fetch. The data cache
is optimized for reading and writing data from the memory stage. We will
often put a larger "level 2" cache between these two caches and the main
memory. The L2 cache is a "medium-sized" memory: faster and smaller than main
memory, but slower and larger than the two L1 caches. Using a hierarchy of
caches reduces the bandwidth requirements to main memory and the energy cost
of moving data around. [music]

Module 4, Video 2
[music] We previously looked at the need for a cache, which can be used to
store often-used data for fast access by the processor. But which data is
used often enough that it is worth including in the cache? Programs are
mostly made of loops. Here's a simple one that sums values in memory. It
displays the two characteristics that we can exploit to decide what data to
put into a cache. Let's look briefly at how it works. Each time round the
loop, there are two loads and one store. The first load is to load the data
we're summing. The other load, and the store, are to update the running sum.
Notice two things here. First, we access the running sum over and over again;
each time round the loop. Second, when we access part of the data in one loop
iteration, we've already accessed its predecessor in the previous iteration,
and will access its successor in the next. Caches exploit two types of
"locality" in programs in order to be effective. The first is temporal
locality: if a piece of data is accessed, it is likely that it will be
accessed again in the near future. The running sum has temporal locality. The
second type of locality is spatial locality: If a piece of data is accessed,
then its close neighbors are quite likely to be accessed in the near future.
By close neighbors, we mean data whose memory addresses are not far from each
other. The data accesses have spatial locality. It turns out that most
programs exhibit lots of temporal and spatial locality. We can exploit this
to determine what to put in the cache. Exploiting temporal locality is fairly
easy: we simply see which values have been accessed and place them in the
cache. Exploiting spatial locality is also quite simple: when a piece of data
is accessed, place its neighbors in the cache. We'll see in the next module
how this is actually achieved. We use locality to guess what values will be
accessed next and store them in the cache. If we are correct, we get the
benefit of fast memory, but if we are wrong, we must perform a slow access to
main memory— just as we would if the cache were not present. In real
programs, we see hit rates of 90 percent or more in microprocessor caches,
resulting in vast performance improvements. However, it's worth noting that
some programs have less locality, and for those programs, caches offer little
performance benefit. [music]

Module 4, Video 3
[music] We've looked at the reasons why we build caches, but how do they
actually work? To the outside world, a cache simply takes an address as
input, and either provides the data that is stored at that location at
output, or returns a signal to say that it doesn't have it. If the data is
found, this is called a "cache hit". If the data is not in the cache, this is
called a "cache miss", and it means we must look for the data in main memory
instead. After each miss, we update the contents of the cache. The
fundamental building block of a cache is called the "cache line". It's a
number of data bytes from consecutive addresses in main memory. Cache lines
are typically 32 or 64 bytes long, and a cache typically has an array of
hundreds to many thousands of lines. The line captures spatial locality,
because it is larger than the data read by a single load instruction. When a
cache miss occurs, the whole line containing that data is copied to the cache
from main memory, meaning we have nearby values for future accesses. When a
request comes in, we use some bits from that address to index the line array.
Just like in the last module on branch prediction, this leads to the problem
of aliasing again, since selecting only some of the bits to index into the
array is like a hash. This means that many addresses map to the same line in
the cache, but we can only store one of their lines of data. We need to note
down which addresses' line is currently stored in the data array, in what we
call the "tag array". There is one tag for each cache line. When we access
the cache, we access the tag array with the same index to see if the data we
need is present. This design is called a "direct-mapped cache". Direct-mapped
caches work fairly well, but for some programs, we can be unlucky with the
program accessing to aliasing lines frequently. We can do something about
this by duplicating both the arrays, so that each line of data can now be
stored in one of two places in the cache. When we access the cache, we look
at both arrays and only get a miss if neither of the tags match. This is
called a "2-way set-associative cache", because each line has a set of two
places or "ways" it could reside. The "associativity" of this cache is
therefore two. Set-associative caches introduce a further complication: when
we want to add data, where do we put it? In a 2-way set-associative cache,
there are two choices. How we decide which cache line to evict is called the
"replacement policy". There are many different types of replacement policy. A
simple one is just to make a pseudo-random choice from all the possible cache
lines. Another option is to keep track of when each cache line was last
accessed and to evict the one last used furthest in the past. This is called
a "least recently used policy", and takes advantage of temporal locality. It
does, however, means storing extra information in the tags to track usage.
Many other ideas are possible too. [music]

Module 4, Video 4
[music] Now that we've seen how caches work, let's see how they affect the
performance of a processor. Recall the processor performance equation, where
the processing time is proportional to the average cycles per instruction.
Without a data cache, if 20 percent of instructions are loads, and main
memory takes 20 cycles to access, our CPI figure must be at least 5. However,
if we provide a cache that holds the required data 80 percent of the time...
...and only takes 2 cycles to access, our CPI reduces to 2.2, which is a
significant improvement! We can isolate the memory terms in this equation to
get the average memory access time —abbreviated to AMAT— which allows us to
compare different cache configurations more easily. Changing the cache
configuration will impact the AMAT. There are many different cache parameters
we can change, such as the size, replacement policy, associativity, whether
we put data in the cache for stores or just for loads, and so on. For
example, reducing the size of the cache will improve the access time for a
hit, but will also increase the miss rate. Let's say that we can halve the
access time to 1 with a corresponding halving of the hit rate. This alters
the AMAT to 13, which in this case is worse for performance overall. It's
also useful to look at why an address might miss in the cache. Broadly
speaking, we can divide cache misses into three different categories.
Compulsory misses occur when we attempt to access an address that we have
never seen before and so never had the opportunity to cache it. Capacity
misses occur when there is more data being accessed than the cache could
hold, even if we had complete freedom in where to put each cache block.
Conflict misses occur in caches where there are more addresses hashing to the
same index than arrays to hold the data. We can alter our cache
configurations to lower these misses, but as always, there are trade-offs
involved. Compulsory misses can be reduced by increasing the cache block
size, to take advantage of spatial locality. But for a fixed cache size, this
reduces the number of different addresses or cache lines that can be stored.
A technique called "pre-fetching" can also be used to predict the addresses
that will soon be accessed, and bring their data into the cache early. But
this increases energy consumption, and may make the cache perform worse if
the predictions are not highly accurate. Capacity misses can be reduced
through increasing the size of the cache. Although, as we saw before, this
impacts the number of cycles taken to determine a hit. Conflict misses can be
reduced through increasing the number of cache blocks in each set, with an
increase in energy consumption as a side effect of this. [music]

Module 4, Lab
[music] In this exercise, we're going to be using a cache memory simulator to
explore the effects of cache memories on processor performance. The simulator
accurately simulates the behavior of the cache memory system, but it's using
a model of the rest of the processor to quickly allow us to simulate the
effects of the cache on the processor without needing to simulate the full
processor. We can configure a number of parameters about our cache, for
example, the number of levels in our cache, whether or not the cache
separates instructions and data, or is unified, keeping them both in the same
cache. We can also configure the size, the line size, and the associativity,
and changing these numbers will affect design parameters, such as the access
times in the case of an L1 level one cache hit or a cache miss, and also the
design size. Once we're happy we've found design we'd like to investigate, we
can press "run", at which point the simulator will run through the program.
We can see the hit rates for instructions in the level one cache, and also
data in the level one cache are displayed. And we can also see the average
memory access time, the results from this. And then below everything, we can
see a table of past simulations so that we can quickly refer back to our
previous experiments when we do new ones. So, for example, let's say we were
curious about the effects of increasing the size of the cache. If we change
the parameter and then press "run", we can immediately see that the design
size has substantially increased, which sort of makes sense because we've
doubled the contents of the cache, the size of the contents of the cache, and
therefore we'd expect roughly double the area. And we can also see that the
hit rates have improved. So there's about a 1% improvement to the L1
instruction cache hits and a 1% improvement to the L1 data cache hits, which
has reduced the overall average memory access time. And so we can compare
these two designs to see which of them we think is better. It's a trade-off
of course, though. The larger design has got better performance, but it is
larger, and so depending on the context, we may need to pick the smaller
design or the bigger design depending on our performance goals. In the
exercises, you'll be invited to come up with a series of designs for caches
that meet certain performance goals. For example, you'll have a constraint on
the area of the design or the execution time of the program, and you need to
optimize the cache to meet those goals. Best of luck. [music]

Module 5, Video 1
[music] In this module, we'll look at how to further improve performance by
exploiting "instruction-level parallelism." In Module 2, we explored how
pipelining can improve the performance of our processor. This reduced our
clock period, and allowed execution of instructions to be overlapped,
improving throughput. One way to boost performance further would be to create
a much deeper pipeline. At some point, this would mean even the ALU in our
Execute stage will be pipelined. Consider the simple program shown in the
slide. Some instructions are dependent on a result from the previous
instruction. Remember in our 5-stage pipeline that these dependent
instructions could be executed in consecutive clock cycles with the aid of
data forwarding. If execution takes place over two pipeline stages within the
pipeline, we need to stall if adjacent instruction share a dependency. This
allows time for the result to be computed. The programmer may be able to
rewrite their program to get the same result with fewer stalls, by placing an
independent instruction between our pair of dependent instructions. In this
case, we can move the third and fifth instructions earlier to optimize
performance. The performance of programs that run on our "super-pipelined"
processor would, to some degree, be determined by the availability of
independent instructions that could be executed in parallel in the pipeline.
This is "instruction-level parallelism"—or ILP. Very deep pipelines are
problematic as they would require: a very high-frequency clock to be
distributed across the chip very precisely; careful balancing of logic
between many, very short, pipeline stages; the pipelining of logic that is
difficult to divide further into stages; and the division of logic at points
requiring many pipelining registers to be inserted. A different approach to
exploit ILP is to make our pipeline wider rather than deeper. In this design,
the processor will fetch, decode, and potentially execute multiple
instructions each cycle. Such a design avoids the problems of a super-
pipelined processor, although as we'll see in the next video, it does
introduce some new complications. [music]

Module 5, Video 2
[music] In this video, we are going to explore "superscalar" processors,
which can process multiple instructions in each pipeline stage. In our simple
5-stage pipeline, there is at most one instruction per pipeline stage. At
best, we can complete one instruction per cycle. We call such a design a
"scalar" processor. In a 2-way superscalar version of this processor, we
would extend this design so it is able to fetch, decode, execute and
writeback up to two instructions at a time. In general, superscalar
processors may vary the number of instructions that can be processed together
in each stage. Let's step through the design. Our instruction cache will need
to supply two instructions per cycle. Typical superscalar processors only
ever fetch adjacent instructions on a given cycle. This can lower performance
if, for example, the first instruction fetched is a taken branch, because
then the second would not be required. Note that now every cycle lost due to
control hazards will cost us two instructions rather than one, so accurate
branch prediction matters even more in superscalar designs. The Decode stage
must now decode and read the registers for two instructions simultaneously.
Fortunately, we are able to extend the register file design to read many
register values at the same time. The Decode stage also needs to check
whether the two instructions are independent. If so, and if the functional
units they both need are available, it can "issue" them for execution in
parallel on the next clock cycle. Otherwise, in this simple design, it will
only issue the first, and keep the second back. A simple design such as this—
where two instructions are fetched, decoded and issued— is called a "2-way"
or "dual-issue" processor. In other designs, the width may vary at different
stages of the pipeline. To support the execution of multiple instructions at
the same time, the Execute stage is expanded and contains two execution
pipelines. It's common for these to have slightly different capabilities to
save area. For example, the top pipeline can execute both ALU and memory
instructions, while the second pipeline only executes ALU instructions. To
ensure that dependent instructions can execute on consecutive clock cycles,
we must add data forwarding paths. These data forwarding paths must allow
results stored in either execution pipeline to be forwarded to the input of
either ALU. During writeback, we need to store both results to the register
file. This means the register file must be redesigned to allow two writes per
clock cycle. Overall, these changes typically require 25 percent more logic
circuitry in our processor, compared with a scalar processor. But we'd expect
an improvement in execution time of between 25 and 30 percent for real world
programs. [music]

Module 5, Video 3
[music] We've seen that instruction-level parallelism can be used on
superscalar processors, to run them faster than would be possible on a scalar
processor. But how much of a speedup is this in practice? Ultimately, this
depends on how much instruction-level parallelism is possible in a typical
program. How might we measure this? We can do this initially without
considering any constraints that will be imposed by the processor it will run
on. Let's consider the instructions executed by the program. Let's assume
that we can predict all the branches in the program perfectly. Then we can
ignore branch instructions, as they don't need to flow down our pipeline. Now
let's imagine we can execute any instruction as soon as the data it needs is
ready. That is, we are only restricted by the presence of true data
dependencies. Note that some dependencies are carried through writes and
reads to memory. Rather than considering program order, we can now just look
at the order the dependencies impose on instructions. This is referred to as
"data-flow analysis." Assuming each instruction takes exactly one cycle to
execute, the fastest possible execution time of the whole program in cycles
is given by the longest path in the data-flow graph. The instruction-level
parallelism of this program is the number of instructions divided by this
duration, as this gives the average number of instructions we would need to
be able to execute each cycle to achieve this duration. In real programs,
this can be anywhere from around five, to hundreds or even thousands. An
active area of research and innovation for computer architects is to imagine
processor designs that can expose and exploit as much of this parallelism as
possible. One insight architects have had is that superscaler processors need
to have a fast supply of instructions to be able to analyze dependencies
effectively. This often means that the front end of our processor pipeline is
much wider than the rest of the pipeline, so that it can "run ahead" and see
what behavior the program will have next. Fast and accurate branch prediction
is vital, as we often have to predict multiple branches ahead accurately, to
achieve good performance. Another key insight is that we don't have to wait
to execute the instructions in program order. If all the dependencies of an
instruction are satisfied, the instruction can proceed down the pipeline even
if previous instructions are yet to execute. This can reduce program
execution time by taking advantage of more instruction-level parallelism. In
practice though, this creates extra complications, as we will see in the next
module. [music]
Module 5, Lab
[music] In this exercise, we will be using a simulator to explore superscalar
microprocessor design. The simulator has a number of parameters that we can
configure, such as the number of pipeline stages, the width of the Fetch
stage, the width of the issue, and the number of ALUs in the design. We can
see a diagram of the processor that we've created and we can also see a
number of parameters about that design, for example the clock frequency and
the overall area of the design. When we press step, the simulator will
advance one clock cycle and so for example here we can see that the fetch
stage has fetched the first 4 instructions. However, immediately we see one
of the problems with designs such as this, which is the three of the four
instructions that have been fetched are in fact useless because the first
instruction was an unconditional taken branch and therefore the remaining
three instructions will not be executed by the program and so these will
immediately be discarded on the next cycle. Pressing the run button allows us
to simulate and we can use the fast-forward option to simulate much quicker
in order to get to the end of the long program execution. In this case we can
see that our design achieved an average cycles per instruction less than one,
which is to say that we on average executed more than one instruction per
cycle, which means that our superscaler design has fundamentally worked, and
we can see that, for example, the overall execution time of the program is
1.6 milliseconds. In a table below, we can also see a record of our previous
simulation runs. So let's say for example, we were curious about the effect
of increasing the issue width by one. We can make that change and then press
run again in order to run our new design, and when it finishes we can scroll
down to take a look and we can see that the program execution time has indeed
improved down to 1.51 milliseconds at a cost of only 1% area. So it looks
like this was a very good improvement to our design and it's almost surely
going to be a beneficial trade-off in practice. In the exercise you will be
invited to configure a number of different superscalar processor designs with
various targets in terms of clock frequency, design area and execution time.
Once you're happy that you've configured the process that you think completes
the exercise, you can scroll all the way down to the bottom, where you'll see
the submit button that you can press to have your answer checked. Good luck.
[music]

Module 6, Video 1
Introduction:

So hi, my name is Peter Greenhalgh. I'm Senior Vice President of Technology

and an Arm fellow. I'm responsible for the Central Technology Group at Arm.
We're about 250 people. We work on everything from machine learning to CPU,
GPU, system IP, and the solutions that we create as well. And we basically
path-find future technology at the product level that goes into all of our
products and the IP that we produce. Arm is known for the power efficiency of
its microprocessors. How have you managed to keep a focus on power when
building processors with very complex and power-hungry features? We've got
some really great design teams. In fact, we churn out I think more CPUs than
pretty much anyone else does on the planet. I think we're producing something
like four or five CPUs per year. So we've got a lot of experience in
designing for power efficiency and performance, and in fact we can leverage
the understanding that we have all the way down to microcontrollers through
to the smaller A-class processors, all the way up to the high performance.
There's a lot of sharing between the teams in terms of strong knowledge and
capability, and insight into how to design for both performance and power
efficiency. More specifically, I mean, ultimately, you have a performance
goal that you need to achieve, and then as part of that you have to figure
out how to get the best possible power out of the design when you're
achieving that performance goal. And to do that, there's kind of some
different ways of looking at it. There's the really detailed orientated work
that you need to do around things, like clock gating, data gating, all the
things to try and stop unnecessary power use deep within the
microarchitecture when the instructions are flowing through the pipeline or
data's moving through the pipeline. And then there's essentially the
structure of the design that you've created. And that then dictates
fundamentally what the power of the design is going to be. You can't fix a
design that's got bad structure with improved clock gating, data gating, and
just good low-level design. You have to marry the two together. And that
high-level work that you do is around making sure that the pipeline is well
balanced, that you aren't opening up the pipeline, going too wide too soon;
you're extracting data, you're extracting information as late as you possibly
can and just when you need it, and not just pipelining it down through the
design for the sake of it; and then, fundamentally, good microarchitecture
around branch prediction, which stops you putting things down through the
pipeline that you're just ultimately going to flush; good pre-fetching on the
data side so that you make sure you get the data in the design when you need
it, and you're not sitting around waiting for it. So you have to marry that
altogether, and we've got a lot of great techniques in order to achieve that,
which fundamentally, I say, you need to achieve the performance target, and
then everything else comes together to achieve that performance target in the
best possible energy efficiency. How did Moore's Law affect computer
architectures of the past, and what will its influence be on future designs?
Gordon Moore's influence on the industry has been massive, and the tenets
behind the law still continue today, albeit in a slightly different form. I
mean, I started designing at .18 Micron, essentially 180 nanometers, and here
we are today working on 3 nanometers. So it's a vast difference now compared
to when I started 22 years ago. And there's no way we could have got to where
we are today without the process scaling from all of the foundries out there
and all the companies that provide the foundry technology. So it's a little
bit like magic, all of the work that they do. I can't say I understand it in
detail, but it's incredible technology, and that allows us... If it hadn't
happened, we'd still be stuck in all the designs which were fairly simple.
There's no way that we'd have got to the sort of designs that we have today
of massively out of order, deep, deep amount of instruction and depth, very,
very wide designs. All of that has been made possible by the steady
improvement, a predictable improvement of the foundries. And that's kind of
one of the key points which really was captured by Moore's law or is captured
by Moore's law of that kind of predictable knowledge that you will get an
improvement in the process. Is it 10%, is it 15? Is it 20% on, say, power,
for example? It kind of doesn't matter in a way because you can work with
what you eventually get. You can do things like voltage scaling to be able to
make use of the power that's available to you. Is it 5%? Is it 10% on
frequency? Again, it kind of doesn't matter in a way. But what matters is
when we start designing a processor today and we finish it in 18 months time,
and then two years after that it arrives in the product in the shops that
consumers can buy. We know that over that period, the process improvements
have happened, which allows us to liberate essentially more performance, more
energy efficiency from the design. And we don't mind too much if it takes
another three months or six months to get to the process. We don't mind too
much if the performance or power is not exactly where it was predicted at the
start. But, ultimately, we know we'll get an improvement, and we know
there'll be an improvement in two years, and three years, and four years, and
Moore's Law may have slowed, but it's certainly not stopped.

[music] As we saw in the last module, instruction level parallelism can be

used to improve program execution time in our microprocessor designs. To
enable this, the compiler creates an optimized instruction schedule when the
program is converted into machine code. Unfortunately, the compiler cannot
know precisely what will happen at run-time, so this design is constrained by
the order of instructions in the program. The compiler won't know what the
program's input data will be, whether branches will be mispredicted, or
whether memory accesses hit or miss in our data cache. In contrast, a
superscalar processor with "out-of-order" execution can produce an
instruction schedule at run-time, only constrained by true data dependencies
and its hardware limits. This schedule is produced on demand and so can even
change each time the code runs. To do this, we introduce an "issue window" or
"issue queue" after the Decode stage. This holds instructions until they can
be executed, not necessarily in the order they arrived in. Within this
window, instructions can be issued whenever their dependencies are available,
and when a functional unit is available to process it. To be able to detect
when an instruction is ready to be issued, we must know whether the
instruction's dependencies are ready when it enters the issue window. We must
then update this status as new results are produced. To implement this, the
names of result registers of executed instructions are broadcast to the issue
window. The instructions waiting there compare the register names to the
registers they require. However, this scheme has a problem: A register will
be written multiple times in the program, and since the instructions are
executed out-of-order, the register name alone is not sufficient to record
dependencies. It also means that instructions would have to wait until all
previous reads of a register had finished before executing. These are called
"false dependencies." These problems can be resolved by "renaming" register
names at run-time so that each "in-flight" instruction writes to a unique
destination register. We use a "physical register file" that is large enough
to ensure we don't run out. We keep a "register mapping table" to store the
mapping between architectural, compiler-assigned registers, and physical
registers. Register reads to the same architectural register are renamed
consistently, so that dependencies can be tracked correctly with physical
register names. Physical registers are reused only when they are no longer
used by any instruction currently in-flight or any entry in the register
mapping table. The other big issue with out-of-order execution is memory
dependencies. Load and store instructions can have memory dependencies
because they access the same memory location. To detect this, we need to
compare the computed memory addresses that the instructions access. We thus
split memory operations into two steps: address calculator and memory access.
We issue their address calculation step as soon as the dependencies are
available. Then, the memory access step is placed in a special load-store
queue to be sent to our data cache as soon as possible. We carefully ensure
that operations that access the same address are kept properly ordered, but
independent accesses can be reordered if beneficial. No access can occur
until the addresses of all previous accesses are known. Since memory writes
are irreversible, store instructions must also wait until we are certain that
they will execute. [music]

Module 6, Video 2
[music] In the previous video, we outlined the concepts of out-of-order
execution, and register renaming. The issue window will be filled with
instructions fetched along the path that our branch predictor believes the
program will take. While we hope our branch predictor will be correct in most
cases, it will sometimes be wrong. How do we handle such cases? A simple
approach is to start by recording the original program order of the
instructions, and then to monitor their progress. We call the structure that
stores the instructions the "reorder buffer." As each instruction executes
and produces a result, we can mark it as done. When the oldest instruction
has completed, we can remove it from the end of the reorder buffer, and the
instruction is said to have "committed." This stream of committed
instructions represents how our program would be executed on a simple in-
order pipeline or by an unpipelined processor. It usefully also provides a
point at which we can process exceptions. For example, if the program divides
by zero or attempts to access memory that does not exist. We also check
branch instructions as they complete in order. If they have been
mispredicted, we flush the reorder buffer, our instruction window and any
currently executing instructions and start fetching down the correct path. To
preserve correctness, we must also restore our registers and the register map
table to the values they had when we mispredicted the branch. This can be
done with the aid of a second register map table, updated only when
instructions commit in program order. This can simply be copied to the map
table used by our renaming hardware to "rewind time" for the processor. All
the register values we need will be present, as we don't recycle registers
before we know they will not be needed again. In reality, handling branches
in this way is too slow. Processors instead take many copies of the register
map tables and can handle branches as soon as they are resolved, and we
discover they have been mispredicted. They can also selectively neutralize
the in-flight instructions in the datapath that are on the wrong path, rather
than flushing all of these instructions away. [music]

Module 6, Video 3
[music] We can now bring everything together and look at what a typical
pipeline for an out-of-order superscalar processor might look like. The Fetch
stage is aided by an accurate branch predictor as we met in Module 3. It will
fetch a group of instructions on every clock cycle. This group of
instructions will be requested from the instruction cache, and will be from
consecutive memory locations. Branches may reduce the number of useful
instructions that can, in practice, be fetched in on each cycle. The Decode
stage decodes multiple instructions in parallel. At this point, modern high-
performance processors may also split complex instructions into simpler
operations or "macro-ops." In some cases, there may also be opportunities to
combine simple instructions into a single operation. The next step on an
instruction's journey is renaming to receive a unique destination register.
As we saw in the last video, this increases opportunities for out-of-order
execution. Remember, there are several times more physical registers in our
processor than those available to the compiler. Instructions are placed in
the reorder buffer, and are also "dispatched" to the Issue stage. They will
wait in the window as necessary, and are ready to be issued once all their
operands are available. In the most complex of today's superscalar
processors, there may be hundreds of instructions buffered in the issue
window at the same time. Instructions finally commit in program order. At
this point, any physical registers that are no longer needed can be added
back to the pool of free registers. These are then assigned during the
register renaming step. Once an instruction is issued, it reads its operands
from the physical register file. The Execute stage consists of many
functional units operating in parallel. These may each support different
operations and take different numbers of cycles to execute. A network of
forwarding paths is also provided to ensure we can execute any dependent
instruction on the next clock cycle after the generation of the result. This
requires being able to quickly communicate—or "forward"— a result from the
output of any functional unit, to the input of any other. Some instructions
will need access to memory. After computing their addresses, they are placed
in the processor's load-store queues. "Stores" are sent to memory in program
order, but "loads" can often be sent out of order, and ahead of other older
stores or loads that are not yet ready to be issued to memory. The memory
system reduces the average memory access time by providing numerous levels of
cache memory. After generating results, we write them back to the register
file. This overview is representative of the fastest modern microprocessors
found today in laptops, smartphones and servers. Whilst much extra innovation
goes into real designs, they generally follow the ideas discussed in the
course. [music]

Module 6, Video 4
[music] One question computer architects always ask themselves is: "how much
can we scale up our design?" Let's take a look at some further potential
optimizations to our out-of-order superscalar processor. We could try to make
it wider. For example, by doubling the number of parallel instructions, we
can fetch, decode and execute more instructions per cycle. Would this double
our performance? Sadly, no, things are not that simple! In practice, some
components quickly become very complex, and performance gains may be hard to
extract. For example, today's largest machines fetch at most ten instructions
per cycle from their instruction caches. Fetching more instructions than this
offers minimal performance gain, despite a large hardware cost. If we
increase the number of registers, the size of our issue window, the size of
our load-store queues, or perhaps use a larger and more accurate branch
predictor, our processor's performance will only improve slightly despite a
significant increase in the size of these structures. After a point, the
increase in performance is no longer worth the cost of the extra transistors.
It's also possible that performance might reduce overall as we may need to
lower our clock frequency as the structures get larger. Finally, we could
introduce more pipeline stages, but we know this doesn't necessarily lead to
higher performance, as mispredictions may become more costly. The combination
of these issues means that extracting performance using instruction-level
parallelism alone becomes more expensive as more performance is sought. This
graph shows how the energy cost of executing an instruction grows quickly as
we try to build higher performance processors. Let's look at some example
designs. Suppose we have a core, which requires a certain area. If we double
its area, its performance improves, although there is a small rise in energy
per instruction. If we quadruple its area instead, its performance has now
doubled compared to our original core, while energy has increased by 50
percent. Going further, if we increase our processor's area by a factor of
10, performance is only 2 point 5 times our original core, but energy per
instruction is now 3 times higher. Its performance does not improve as fast
as the cost of running it! Of course, engineers are clever and determined,
and are constantly developing new techniques to bypass many of these issues.
This means the performance of processors—even ones running a single thread or
program— still improves by around 10 to 25 percent each year. Nevertheless,
ultimately we often need more performance than can be provided by
instruction-level parallelism alone. A modern solution is to employ multiple
processor cores on the same chip—called a "multicore" processor. This changes
the task for programmers; they may need to redesign their programs to take
advantage of such parallelism, but if they can, it can give vast performance
benefits. As we've learned throughout the course, every decision involves
trade-offs and compromise. We are faced with a fascinating but often highly-
constrained design problem. We've seen how performance bottlenecks, that at
first seem impassable, can be overcome with innovative designs. What might
the future hold for microprocessors? Can you think of ideas? What would you
design? [music]

Module 6, Lab
[music] In this exercise, we'll be using a simulator to explore an out-of-
order superscalar processor design. The simulator allows us to configure a
number of aspects of our processor, for example, the number of pipeline
stages, the width of the Fetch stage, the size of the issue window, the
number of ALUs, and the size of our re-order buffer. The changes will be
reflected in the pipeline diagram, which we can see below, and also in the
statistics below that, with key design metrics, such as the clock frequency,
clock period, and design size, visible below. When we press "step," the
simulation will advance by 1 clock cycle, and so we can see, for example, on
the first clock cycle the 1st 4 instructions are loaded. Although three of
them are unusable because they follow a taken branch and therefore these will
not be executed and will be discarded on the next clock cycle. We can press
"run" to watch our design in action and in order to quickly get to the end,
we can use the "fast forward" feature to simulate the millions of
instructions in this particular program. After the simulation is complete, we
can check below to see a number of statistics about our pipeline, which are
useful for understanding why the performance is as it is, and in particular
we can see the program execution time. 1.18 milliseconds gives us the overall
execution time of the program that our design achieved, and notably our
average instructions per cycle gives us the number of instructions that we
were able to complete on each clock cycle, in this case well over 1,
indicating that we are taking advantage of instruction level parallelism in
this simulation. Below all that, we can see our completed simulations and we
can check back on these as we explore new designs. So let's say, for example,
we were curious about the effect of increasing the re-order buffer size. We
could change that and then press "run" to quickly run our next experiment.
And then if we scroll down to our table, we can compare the results and see
that actually the program execution time was substantially increased by
increasing the size of the re-order buffer. Although, admittedly, this did
come at a not insignificant increase in the area of our design, and so
whether or not this represents a good trade-off in practice would very much
depend on the problem we're trying to solve. In the exercises, you'll be
given a number of scenarios for processors, which generally revolve around
certain constraints on the design size, or the clock frequency, or the target
program execution time. And once you think you've found a design that meets
the goals required, configure it up using the settings and then scroll down
to the "submit" button at the bottom, and click that to have your answer
checked. Best of luck. [music]