Computer Architecture Essesntials by ARM

Module 1, Intro
[music] I'm Richard Grisenthwaite, Chief Architect at Arm. What does a Chief Architect
do? The Chief Architect is responsible for the evolution of the Arm architecture. One
of the things about an architecture is, you want your software to continue running on
future products. So it's a very long term commitment, and the role of the chief
architect is essentially to curate the architecture over time, adding new features as
needed by the partner. Why are microprocessors so important and ubiquitous? I think
the fundamental reason is that they are a general purpose device. They describe a
basic instruction set that does the stuff that you can construct pretty much any
program out of. Indeed, they're Turing complete, which means, by definition, you can
create any computable problem, solving a computable problem with the processor.
They're not customized to any particular usage, but they do two things really rather
well, which are data processing and decision making. In other words, having, for
example, added two numbers and compared them, you're then taking a branch and making a
decision based off that. Now in reality, an awful lot of general purpose problems are
ones that involve: work out some value or some criteria, compare against that, and
then make a decision on it. And essentially that allows you to solve a huge number of
different problems and that's why the microprocessor has become so ubiquitous. What is
the difference between architecture and microarchitecture? Architecture is what it has
to do, and microarchitecture is how it does it. So for example, we will define in the
architecture a set of instructions that do "add" or "load" or "branch", but it says
nothing about whether you've got a ten-stage pipeline or a three-stage pipeline. It
says nothing about branch prediction. All of those sort of features which actually
determine the performance of the device; all of those are part of the
microarchitecture. The key point is that software that is written to be compliant on
the architecture will actually run on lots of different implementations, lots of
different microarchitectures that essentially implement that architecture in different
ways, choosing a trade-off between the power, the performance and the area of the
design. What does it take to build a commercially successful processor today? If you
actually start from scratch with an established architecture, but we want to create a
new microarchitecture, we reckon, for a high end processor, you're talking about
three-to-four hundred person-years' worth of work in order to create a properly
competitive multi-issue out-of-order machine compared with the state-of-the-art that
you can get from Arm. In addition to that — and that's just to come up with the RTL —
in addition to that, you've then got to do the implementation. If you're going to
build that on a three-nanometer process, the leading edge to get the best possible
performance, you're talking about tens of millions of dollars for the mask sets.
There's a whole bunch of software you've got to go and build on top of that and do all
of that work. When we've looked at companies that are interested in actually starting
up, taking an Arm architecture license — say I want to go and build my own business —
we reckon that you need to be prepared to spend getting on for half a billion dollars
before you're actually going to be successful because it takes time. Your first
product is not necessarily going to be fully competitive because it would be slightly
surprising if the first thing that you built was as good as what people have been
building for many years. It takes time to build up that expertise and skills. And so
you're going to see a couple of iterations before even the best teams end up really
being competitive. And so as I say, I think if you went from nothing and wanted to
essentially create something, and that's using the Arm architecture with all of its
existing software and so on, you're talking the best part of half a billion. What is
the Arm business model? The Arm business model fundamentally is the licensing of
semiconductor IP to as wide a range of companies as possible and in as many ways as
possible, in order to maximize the uptake of the IP. When we talk about IP, we're
talking about essentially designs for processors and we license that both as an
architecture license, where you effectively gain the right to build your own processor
to the specification of the Arm architecture, or an implementation license where we
are licensing, actually, an implementation that is compliant with the Arm architecture
in the form of some richer transfer-level RTL. What makes Arm different from its
competitors? So if you go back to where we were when Arms started out in the early
90s, there were many, many different architectures available and they were all kind of
doing the same thing but slightly differently. They would have different instructions,
different instruction sets, and so software needed to be ported. A huge part of Arm's
success actually comes from the fact that we created a business model of licensing the
IP to make it very easy for people to build processors, to incorporate the designs
into their SoCs, into their systems. And that then made it very straightforward for
people to be able to use the Arm architecture. Now what this then meant was, people
said: I will port more software to this because it's more widely available and you get
this positive feed-forward effect whereby more availability of hardware encourages
more software, encourages more hardware, and so on. And essentially that meant that a
lot of people said: there's no point in me having a different architecture. I'm not
getting a valuable difference from doing that. All I've got is, kind of, a needless
change to the software that I need to make. So actually the whole model, we went on at
Arm, which was: let's license our IP to a great number of different players to come up
with different solutions to meet the form factor of a camera or a PDA, or whatever it
was back in the day. Those things made it much more straightforward for people to
incorporate our technology. [music]

Module 1, Video 1
Computer architecture is the study of tools and techniques that help us to design
computers. More precisely, it helps us understand how to meet the needs of particular
markets and applications, using the technology and components that are available. For
example, we might need to produce the chip at the heart of a smartphone using 10
billion transistors and a power of less than two Watts. How do we achieve the
performance a customer wants? The challenge is a fascinating one, and one that
requires a broad understanding. For example, what target market are we designing for?
What are the characteristics of the applications to be run? How will programming
languages and compilers interact with the microprocessor? How best to craft the narrow
interface between the hardware and software? How to organize the components of our
microprocessor? And how to design a circuit, given the characteristics of individual
transistors and wires? Like many design problems, computer architecture requires many
trade-offs to be made and evaluated. Each design decision will impact trade-offs
between size, performance, power, security, complexity, and cost. Trade-offs must be
re-evaluated regularly, due to advances in fabrication technology, applications, and
computer architecture. Computer architecture must be grounded in quantitative
techniques and experimentation, but the endless number of possible designs means that
the field depends on a high degree of human ingenuity and art. Perhaps surprisingly,
the earliest computers and today's most advanced machines have much in common. They
both execute a stored program constructed from machine instructions. These
instructions perform simple operations such as adding two numbers. Nevertheless,
greater numbers of faster transistors, and the application of a relatively small
number of computer architecture concepts, have enabled us to construct machines that
can perform billions of instructions per second, and shrink these machines to fit in
hand-held battery-powered devices. It is this rapid progress that has supported
breakthroughs in machine learning, drug discovery, climate modeling, and supports our
modern world where computation and storage are almost free. The task of designing a
microprocessor is split into different levels of abstraction: "Architecture;"
"Microarchitecture;" and "Implementation." "Architecture" focuses on the contract
between programmers and hardware. It allows compatible families of microprocessor
products to be built. The ARMv8-A architecture is an example of this. Architecture
includes the "Instruction Set Architecture," or ISA, which defines what instructions
exist. It also defines precisely the behavior of memory and other features needed to
build a complete processor. "Microarchitecture" focuses on the organization and
structure of the major components of a microprocessor. It has to match the rules set
by the architecture. The microarchitecture still has flexibility though; and so the
implementation specifies the circuit detail precisely. This culminates in the exact
circuit design for manufacture. Each of these levels is vital, and each comes with its
own challenges and opportunities.

Module 1, Video 2
At the beginning of the 20th century, "computers" were people employed to perform
calculations. These computers used mechanical calculators to help them perform
arithmetic. They followed instructions to decide what calculation to perform next.
These instructions defined the algorithm or program they were executing. They would
consult paper records to find the inputs to their calculations, and would store their
intermediate results on paper so they could refer back to them later. A modern
electronic computer is organized in a similar way. We will look in more detail at
these components of a microprocessor in the next video, but for now let's look at how
it operates. Microprocessors also follow instructions one by one, and then perform
relevant calculations. This idea of fetching instructions from memory and executing
them is called the "Fetch-Execute Cycle." In the "Fetch" stage, the computer reads the
next instruction from the program. This instruction is encoded in binary as ones and
zeroes, so it must be decoded to understand what it means. This is done in the
"Decode" stage. Once it is clear what to do, we move to the "Execute" phase. This can
involve different tasks such as reading memory, performing a calculation, and storing
a result. Once done, the computer is then ready to begin the cycle again, by fetching
the next instruction. Instructions are normally fetched sequentially in order, but
some special instructions called "branches" can change which instruction will be
executed next. For branches, a calculation determines the next instruction. This can
mean evaluating a condition, or reading a register to determine the next instruction's
location. Branches allow computers to make decisions, and to re-use instruction
sequences for common tasks. A modern computer program, like a web browser, contains
millions of instructions, and computers execute billions of instructions per second,
but they all conceptually follow this "Fetch-Execute Cycle."

Module 1, Video 3
Modern microprocessors are circuits built using anywhere from 1,000 to 100 billion
tiny transistors. The key to designing circuits with such huge numbers of parts is to
build them from re-usable blocks. Let's take a look at some of these. Each transistor
is an electrically-controlled switch. When there is too little voltage at the gate,
the switch is off, so the electrical signal cannot propagate from the drain to the
source. However, when there is sufficient voltage at the gate, the switch is on, so
the signal does propagate. When designing processors, we use digital electronics. The
only voltage or current values we consider represent zero and one, enabling us to
build robust circuits from imprecise components, even in the event of electrical noise
or manufacturing imperfections. This means our circuits represent binary numbers,
since they only have two states: zero and one. We use transistors to build
increasingly complex circuits. We can design circuits that can remember binary values,
or select between multiple inputs, or even do arithmetic, like addition. We can then
use those circuits as building blocks in even larger designs. When designing a digital
system, we must keep everything synchronized to control when its behavior occurs. To
do this, we use a "Clock Signal," which is a wire in the circuit whose signal cycles
between zero and one. We measure the rate in Hertz. For example, if you hear a
processor has a clock speed of two Gigahertz, it means 2 billion ones and zeroes per
second. The maximum speed of the clock signal is determined by the longest, and
therefore slowest, path in the circuit between 2 clocked flip-flops. This is referred
to as the "Critical Path." The signal must have time to propagate all the way along
the critical path before the clock cycle completes. A microprocessor needs to do many
types of arithmetic. For this, we build an "Arithmetic Logic Unit" or "ALU." This
circuit receives 2 numbers as input, as well as an indication of what operation to
perform, for example, addition or subtraction. In addition to logic, a microprocessor
needs memory. Memory is organized as arrays of memory cells that are able to store
many "Words" of data. A specific word, commonly 32 bits in length, can be accessed by
specifying its "Address." Each address is a number that indicates the location in the
memory that should be read or written. Memory cells range from hundreds of bits to
millions of bits in size, but larger ones are slower to access, as signals and their
long internal wires take longer to propagate. For that reason, almost all
microprocessors include at least two types for storing data: a big slow "Data Memory,"
and a small fast memory called a "Register File." In reality, the "Data Memory" may be
implemented using many different sizes of memory, as we'll see in Module 4. As well as
storing data in memory, we also use some memory to store the instructions. We need a
way to keep track of which instruction we will fetch next, so we have a "Program
Counter." This stores the address in Instruction Memory of the next instruction to be
accessed. Since instructions are encoded in binary, we also have "Instruction Decode
Logic" that converts that binary to the various signals needed to control the
microprocessor.

Module 1, Video 4
We've already seen how a microprocessor is controlled by instructions. But what are
they really? An instruction is a simple command that the microprocessor hardware can
perform directly. We write them as text like this to make them easier to read. But for
the microprocessor we encode them in binary. We use a program called an assembler to
translate between the human text and the binary. In this video we'll be looking
specifically at an Arm instruction set called A64 but other instruction sets follow
similar principles. Arithmetic and logic instructions are the simplest type of
instruction. The first word tells us what operation will be performed such as addition
or multiplication. The values after this tell the processor where to put the result
and where to get the inputs. Values starting with X are addresses in the register
file. Arithmetic instructions read one or two registers and then put the result into a
third register. Branch instructions are used to make decisions and to repeat
instructions. Normally the microprocessor executes instructions in sequential order
but branches change that, and explicitly tell the microprocessor the address of the
instruction to run next. This is done by giving the address of the next instruction in
the instruction memory. Some branches are unconditional, meaning they always occur and
always affect the next instruction address. Other branches are conditional, meaning
the processor will perform a calculation to decide whether to follow the branch or to
continue executing instructions sequentially following the branch. These are preceded
by comparison instruction to calculate the condition. Loads and stores are the
instructions for accessing the data memory. Loads copy values from memory to the
register file. Stores do the opposite. In both cases, the instruction needs to know
the address in the data memory and the location in the register file to copy between.
For data memory, loads and stores read an address from a base register. They can also
optionally add to this base address by reading another register, or by simply
specifying a number in the instruction itself. Using sequences of instructions we can
build programs. Here is an example of a small program that implements Euclid's
greatest common divisor algorithm. Let's take a look at it working, one instruction at
a time. To start with, the input stored in X1 and X2 are compared. If they're equal,
because we have found the greatest common divisor a conditional branch instruction
moves to instruction 7. If they're not equal, another conditional branch instruction
can be used to determine whether X1 is smaller. Finally, we use an arithmetic
instruction to subtract either X1 or X2 depending on which was larger and then
unconditionally branch back to the start. Here we can see an example of the program
running. One instruction executes at a time. As we step through the program, the
values in the registers X1 and X2 are shown. And we see these are modified each time
we reach instruction 3 or 5. The instructions repeat in a loop, as is common for many
different types of program. Each time round the loop instruction 0 and 1 are checking
whether or not to continue the loop. When they detect X1 and X2 have become equal, the
program finishes and the processor moves on to the next task.

Module 1, Lab
In this exercise, we're going to be using ASim. ASim is a behavioral simulator for a
subset of the Arm AArch64 instruction set. Behavioral simulator means that it's not
simulating real circuit level details of a microprocessor it's just simulating the
behavior of each instruction as it runs. Nevertheless, behavioral simulators are vital
tools for computer architects. We use them to check the designs we've built, match
with the behavior we intended and we can also use them to explore new ideas, for
example adding new instructions to the instruction set. We're just going to be using
it though to get familiar with the basics of Arm AArch64. So we can create a new file
which will allow us to type in some Arm AArch64 instructions and when we press
assemble that will cause ASim to begin the simulation of the instructions. If we
scroll down we can see that below we've put a little cheat sheet of Arm AArch64
instructions that you can use as a reference. Do note that these are just examples
though. So for example this first one is mov X1 X2 and the descriptive text says that
this copies the value of register X2 to register X1. But equally that would imply that
if we use mov X3, X4 instead that would copy the value of X4 to the register X3. So we
don't have to just use these exact instructions, we can tweak them as we need to, to
achieve our goals. So going back to the ASim, let's say that we wanted to add the
number two to the number three. In our cheat sheet we could see that there is an 'add'
instruction but it requires that the inputs for the addition are stored in registers.
So first of all, we're going to need to load up two registers with the numbers that we
want to add in this case two and three. So looking at our cheat sheet again we can see
that there's an instruction mov that allows us to do that. So if I write mov X2, #3
this would cause the register X2 to be loaded with the value 3 and similarly I could
write mov X1,#2 and this would cause the register X2 X1 to be loaded with the value
two. Last but not least, we could then do the addition we wanted to do, such as add X3
X1 X2. This will cause X1 to be added to X2 and the results stored in X3. If I press
assemble now, we will see that the UI changes slightly. Here we can now see the
simulated state of the processor. ASim is only going to, because it's a behavioral
simulator it's only going to show the state before and after each instruction. And so
right now we are before executing the first instruction. That's the yellow
highlighting there and we can also see it in the simulated memory. If I press step we
can now see that that instruction has completed, and it's before executing the mov X1
X2. And notably we can see that X2 has been loaded with the number 3 which is exactly
what we would expect. If I press step again, we can see that mov X1 X2 has now
occurred. MOV X1 #2, sorry has now occurred which has loaded the value 2 into the
register X one. And last but not least, if we step again we see that X3 has become
equal to five which is exactly what we would expect if registry X1 was added to
registry X2. So this allows us to get a feel for what it's like for a machine to
execute these instructions. We can reset back to the beginning if we want to watch it
go through again. If we want to do an edit for example adding a new instruction we can
just do that. Press assemble and that will add the new instruction to our program, and
we can see its effects by simulating. Now in the exercise you will be invited to take
an existing program and add 1 new instruction to it at the indicated position. Do note
that when you assemble the exercise program, you'll be taken to a gray bit of code,
which is our testing code. But as you step, you'll see that the simulator flicks
between the testing code and the code that we wrote the exercise one code. You can
also click between them using this. Feel free to try to work out what our testing code
does but you can just ignore it if you want to. The point of the exercise is just to
add the instruction in the indicated position. When you're comfortable you've got the
right instruction you can press run to get all the way to the end. And if you really
think that's correct if you scroll down to the bottom of the page you'll see the
submit button which you can use to send in your answer. Best of luck.

Module 2, Intro
[music] Hello, my name is Martin Weidmann. I'm an engineer and product manager with
Arm's Architecture and Technology Group. I look after the A-profile architecture and I
maintain Arm's Interrupt Controller specifications. Computer architecture is sometimes
called a science of trade-offs. Why is everything a trade-off when it comes to
designing a processor? Let's take an example of where we've had to make trade-offs
when developing process architecture. So the Arm architecture has a feature called
Pointer Authentication. This is often abbreviated to PAC for Pointer Authentication
Code. What this feature is trying to do is protect against a form of attack called ROP
and JOP. These are Return Orientated and Jump Orientated Programming, and it's where
an attacker tries to subvert things like the call stack to run legitimate code, but in
ways that weren't expected by the programmer or the compiler. PAC or Pointer
Authentication tries to defend against this kind of attack by using part of an address
to provide an encrypted signature. So we can check the signature and the address match
and if they don't, we can spot an attack in progress. So why is this a trade-off?
Well, because to add security, we want that signature to be as big as possible. The
bigger the signature, the more bits we use for that, the stronger cryptographically
that signature is. The trade-off is: the more bits we use for the signature, the fewer
bits we have available for other things, such as the address. So you can have a big
signature with a small address, but if you want the address to get bigger, then you
get a smaller signature, and that's then cryptographically weaker. So the trade-off we
have to make when designing a technology like that is: What's the right amount of bits
for the signature? What's the strength of cryptography we need from that signature in
order to get the design goal, which is to defeat these attacks and give us more robust
computing? What sort of guiding principles do you use when designing a processor? So
when you're designing a processor, the key thing you have to bear in mind is: What's
it going to be used for? What you don't want to end up with is a very expensive
paperweight. So we need to understand the requirements that the processor has to meet.
We have to understand the design trade-offs we're making and how they work into
meeting those requirements. We also have to consider not just the processor itself,
but how we're going to show that that processor is correct. We'll put as much time, if
not more, into testing and validating the design as we do into designing it. How do
you design a new microprocessor? If you wanted to create a new processor from scratch,
the first thing you're going to have to do is understand the market that that
processor is going to address and to then build a team to design that processor. There
isn't such a thing as one processor for every possible market. The requirements for
something like an embedded microcontroller are going to be very different to what you
want from the processor in your mobile phone, your laptop, or your server. So you need
to understand those requirements as the first step into building a new processor. What
determines the best design for a microprocessor? So when you're designing a processor,
you need to work out what the best design for a given market or application is going
to be. There's no magic formula for this. It's going to depend a lot on what you're
trying to achieve with that processor. You need to understand things like the power
requirements, the performance requirements. Is it going to work in a highly noisy
electrical environment? There's a big difference between the reliability requirements
you need from something like a watch versus a satellite. So you would take those
requirements and you'd work out what the best set of trade-offs is going to be, and
that's an art more than it is a science. How do the underlying technologies contribute
to this best design? A lot of technologies go into our processor. There's the design
of the microarchitecture, the implementation of the processor. There's the silicon
process you're going to use, how you're going to integrate that processor into an SoC
or ASIC. Is it a single die, or is it going to be using chiplets or multiple sockets?
All of those different technologies are going to be factors into how you design the
process or what trade-offs you make, and what performance and power you get out of the
design once you're done. In reality there may be many different 'best' designs, so how
do you pick one? So when you're designing a processor, what you want is the best
design. But often there isn't "a" best design, there's just different trade-offs. You
have to decide what the best set of trade-offs is for the particular use case you're
going for. And that's also going to depend on: Is this a device that will be off the
shelf, used for lots of different applications—a general purpose processor? Or is this
being designed for one specific use case? Again, there isn't really a magic bullet or
single answer for this type of question. You need to understand how the processor is
going to be used and then use your experience to judge the trade-offs, and what will
give you the best mix of power, performance, area, cost, reliability for your target
use case.

Module 2, Video 1
[music] In this module, we're going to explore how to improve the simple
microprocessor design from Module 1 in order to allow it to execute programs more
efficiently. First, let's find out how long a program takes to execute. The time taken
to perform the average instruction is equal to the number of clock cycles taken to
perform an instruction multiplied by the duration of one clock cycle. The time taken
to run our program is found by multiplying the average time to perform an instruction
by the number of instructions in our program. How could we make this faster? One thing
we could try is to reduce the number of instructions in a program. We might be able to
optimize the code removing unnecessary and repeated work and selecting instructions to
minimize code size and maximize performance. We could give our microprocessor the
ability to perform more operations in order to help programmers or compilers further
reduce the number of instructions in their program. For example, allowing the loading
of two data values at the same time might allow fewer instructions to be used in the
program. The downside to this approach, is that adding more instructions will require
extra circuitry in the processor and therefore we likely increase the clock period. If
the extra instructions are rarely used this could even mean an overall decrease in
performance. We see this theme often in computer architecture trade-offs that we have
to carefully balance. Another approach is to use faster transistors perhaps
constructed from a more recent fabrication technology. This would reduce the clock
period but may increase costs. The rest of this module focuses on an optimization to
reduce the clock period called pipelining. This is the most important optimization we
use when designing processors. It uses a similar concept to an assembly line in a
factory where work can start on the next item before the previous one finishes. Let's
take a closer look. Imagine that each instruction has to go through four circuits in a
processor. If we attempt to do all of these in one clock cycle this means our clock
period is the latency of all four circuits added together. If we were to pipeline
this, we would add a pipeline register in the middle. This divides the circuit into
two sections called stages. Notice that although each instruction takes a similar
amount of time to travel down the whole pipeline, the pipeline design can execute
nearly twice as many instructions per second. The throughput has doubled. This is
because we can set the clock period much shorter. It's now the maximum latency of the
two stages. We can pipeline into many stages and this allows for much faster execution
of programs. Unfortunately, though, pipelining a real microprocessor design is not
quite as simple because the processor has various feedback signals and loops in the
circuit. In the next video, we'll take a look at the challenges of pipelining in
practice. [music]

Module 2, Video 2
[music] In this video, we're going to look at applying the pipeline optimization to a
realistic microprocessor design. In the first module, we met the components of a
microprocessor, so let's look at how these are really connected. This diagram shows
all the connections needed for a real, unpipelined microprocessor. Each clock cycle,
the processor starts by fetching an instruction from the instruction memory. Once the
instruction reaches the decode logic, it is decoded to produce the control signals
necessary to execute it. The exact control signals vary depending on the type of
instruction. For example, arithmetic instructions access the register file and
interact with the ALU. Ultimately, no matter how the instruction was executed, the
last step of each clock cycle is to update the program counter. This is done by the
branch unit. For non-branch instructions, this just means incrementing the program
counter. However, for branch instructions, the branch unit has to do some
calculations. When we apply our pipeline optimization to this design, we face some
challenges. The design has several loops because instructions have dependencies. How
can we break these cycles? The key observation is that not every instruction is the
same. In real programs, branch instructions usually make up less than 20 percent of
the program. For non-branches, the branch unit doesn't actually need to wait for the
ALU before calculating the result. Let's look at how we can use this fact to pipeline
the processor. Once the first instruction shown in yellow reaches the pipeline
register, we're ready to begin fetching the next instruction, shown in blue. The
yellow instruction can be in the execute stage whilst the blue instruction is being
fetched. Once the yellow instruction is finished, the blue instruction is ready to
enter the execute stage and a new green instruction enters the fetch stage. What about
the branches though? Let's imagine this next yellow instruction is a branch. The fetch
stage works normally until the branch unit, but the branch unit cannot proceed.
Consequently, the pipeline stalls. The fetch stage spends a cycle waiting whilst the
execute stage executes the branch. Finally, once the ALU is done, the branch unit can
proceed and the next instruction, in this case blue, can be fetched. Overall, this
means that the processor wasted one cycle stalling due to the branch. Since only 20
percent of instructions are branches, this means that each instruction would require
on average 1.2 cycles. The same idea of stalling the pipeline can be used to create
even longer pipeline designs. This diagram shows a typical five-stage processor
pipeline. In the next video, we'll look at how we can manage or prevent some of the
stalls in a design like this. [music]

Module 2, Video 3
[music] Instructions within a program may be dependent on each other. That is, one
instruction may produce a value that a subsequent instruction consumes. Data values
may be communicated through registers or memory. The simple program shown has a number
of so-called true data dependencies. This means we must take care to execute these
instructions in order, and make sure results are correctly communicated. Additionally,
the outcomes of branch instructions may affect the path taken through the program, and
consequently, this affects whether an instruction is actually executed. This sort of
dependency is known as a control dependency. In the previous video, we met a realistic
processor pipeline with five stages. Circumstances that prevent an instruction making
progress in our pipeline are known as pipeline hazards. Let's take a look at how
dependencies cause hazards. This program has a true data dependency. The first
instruction writes to register one, which is then read by the second instruction. If
we send this down our pipeline, we see that the second instruction must stall, waiting
for register one to be written, before it can read and proceed. This is a data hazard.
Unfortunately, dependent instructions are common and stalling in this way would
significantly increase the average cycles per instruction. Let's take a closer look at
the hazard though. The ADD instruction is in the execute stage, meaning its result is
being computed. The SUB instruction needs that result to proceed. Rather than waiting
for the ADD to reach the writeback stage, we could add an extra path into our pipeline
to carry the output of one stage to a later instruction, making the result available
straight away. We call this a forwarding path. In this case, the ALU result is
forwarded to the SUB instruction to be used as X1. This small piece of extra circuitry
allows this data hazard to be eliminated completely. Unfortunately, even if we add
forwarding paths everywhere, it's not possible to eliminate all data hazards. For
example, this program has a data hazard due to the load instruction. There are other
types of hazard too. This program contains a control hazard. We cannot be sure which
instruction to fetch until after the branch instruction executes. Consequently, this
program has two stall cycles. We will look in more detail at how control hazards can
be mitigated in the next module. Another class of hazards, called "structural hazards"
occur when two instructions require the same resources simultaneously. For example, if
instructions and data were stored in the same memory, and this could only be accessed
once per cycle, we would have to very frequently stall our pipeline to let these
stages access memory one-by-one. [music]
Module 2, Video 4
[music] In the previous videos, we explored how pipelining could improve performance
by reducing our clock period and by overlapping the execution of different
instructions. We also saw that it was sometimes necessary to stall our pipeline to
ensure that instructions were executed correctly. Ideally, our average cycles per
instruction, or CPI, will remain at 1.0. If we must stall, however, this will
increase. For example, if 20 percent of our instructions were loads and each of these
caused one stall cycle, our CPI would be 1.2. If a further 20 percent of instructions
were branches, and each of these caused two stall cycles, our CPI would be 1.6. The
longer we make our pipeline, the more stall cycles there will be, and eventually the
cost of stalls may outweigh the benefit of the faster clock period. For example, let's
imagine we added a stage to our five-stage pipeline from the previous video. Now the
number of stalls after a branch instruction increases to three, hurting our CPI. On
the other hand, our clock period would improve. So whether or not this helps speed
program execution would depend on the exact details. It may eventually become more
difficult to reduce our clock period by adding further pipelining stages. This is
because it becomes harder to perfectly balance the logic between stages and because of
the constant delays associated with clocking and our pipeline registers. To mitigate
these issues, we will need to invest in more transistors and our design will require
more area and power. The deeper our pipeline gets, the greater the investment we need
to make in terms of area and power for the same incremental improvement. Commercial
processes today have anywhere from two to twenty pipeline stages. The faster, more
expensive and power-hungry processors tend to have longer pipelines than the smaller,
cheaper processes in embedded devices. As with many techniques in computer
architecture, eventually it becomes more profitable to invest our time and resources
in an alternative way of improving performance. In later modules, we'll explore how we
can reduce the CPI, even in heavily pipelined processes. [music]

Module 2, Lab
[music] In this exercise, we're going to be using a model of a processor pipeline to
explore the effect of the pipelining optimization. Computer architects use models like
this to make high level decisions early on about what parameters they will use for a
processor and using a model such as this saves the burden of actually building the
processor to find out its performance. The model is not simulating accurately the
performance of the processor but rather it's giving us an idea for what performance we
might expect. So what can we do with this model? Well, we can configure the number of
pipeline stages, which we can see affects the diagram. And we can also turn on or off
the forwarding optimization. As we change these numbers notice that the design
parameters change down here. So for example, the clock frequency is improved by
increasing the number of pipeline stages but the design area will get bigger. And so
this may be a consideration depending on the problem. We can also choose which of two
programs we're going to put down our pipeline. When we press the step forward button
the pipeline advances to the next clock cycle and we can see the instructions have
started to flow down our pipeline and interesting events such as forwarding will be
noted in the simulation. Sometimes the simulation will detect that there will be a
stall for example, in this case, we can see that there is a data hazard because the
instruction in the red memory stage writes to register X13 which is read by the
instruction in the yellow decode stage and therefore a stall cycle is necessary in
order to allow the result to be generated. If we press the play button, the simulation
will proceed automatically and we can see various stall events happening as the
simulation proceeds. But notice that the the program we're simulating is nearly
1,000,000 cycles long so watching it play out at this speed is going to take quite a
while. So we can use the fast forward slider to simulate much, much faster. Notice
that the statistics down the bottom have updated depending on the results of the
simulation, and at this point we can see that the program is finished and the
simulation of the program, the simulated program took 3.98 milliseconds to execute. We
can also see that just below, the results of past simulations are stored in little
tables so we can easily refer back to them when we're doing later experiments. So as
an experiment, let's imagine what would happen if we disabled the forwarding
optimization but change nothing else and we'll just run this program through. What we
can see immediately is the design side is slightly better which is what we would
expect. It's got 1% better in fact in this case because of the lack of the forwarding
wires. But now that the program is finished, we can see that the program execution
time is a lot worse. 6.34 milliseconds is about 50% worse. So again, looking in our
table, we can compare the execution times in the area and we can see that in most
cases the forwarding optimization would be a big optimization here because at the cost
of an increase in the area of about 1%, we've had a improvement in execution time of
about 50% which is likely to be a good trade off, but not always. It would depend on
the exact scenario. Perhaps that 1% area is more important than the performance of
this program. In the exercise, you'll be invited to design a or suggest a design using
the number of pipeline stages and whether forwarding is enabled that will meet certain
criteria. You can play about and do as many simulations as you wish to figure out what
the best program might be. Once you've got it set up select the processor that you're
happy with at the top and then scroll down to the submit button and press that. Good
luck. [music]

Module 3, Intro
[music] Hi, I'm Nigel Stevens. I'm Lead Instruction Set Architect for the Arm A-
Profile architecture. I've been at Arm for about 14 years and I have responsibility
for the Arm V8-A instruction set including recent developments such as the Scalable
Vector Extension and Scalable Matrix Extension. What do we mean by the instruction set
architecture? The instructions in architecture, primarily, most people think of I
guess as the OP codes, the encodings of instructions that are executed by an Arm-based
processor. But it also includes other aspects as well such as the exception model,
system programming features, memory management and suchlike. The architecture for Arm
is rather distinct from what other companies may call an architecture. For Arm,
architecture is a legal contract, if you will, between hardware and software. If
software uses only those instructional codes and features of the ISA that are
described by the Arm architecture to perform its work, and the hardware that it's
running on implements all of those op codes and features exactly as defined by the
architecture, then any architecturally compliant software will run on any
architecturally compliant hardware that implements that Arm architecture. And that
doesn't mean just processors from Arm itself, but also processors that are designed by
our partners and which we have validated are conformant with our architecture. How do
you decide which instructions to include in an ISA? When we are looking at requests
from partners or from internal research to add a new instruction, we go through quite
a long process of trying to justify that instruction, or, quite commonly, a set of
instructions rather than a single instruction. We have to show that it gives us some
real benefit in performance, the performance of your code running on that CPU. Or
maybe not performance. Maybe it's security you're trying to achieve. But it has to
give you some really concrete benefit that is worth the cost of adding all of the
software, the validation software, the implementation costs for all of the different
implementations, compiler support, so on and so forth. It has to meet the... It has to
answer that cost benefit analysis. What is the difference between an ISA and a
microarchitecture? The difference between an instruction set architecture, or ISA, and
the microarchitecture is that the ISA is an abstract concept. It defines a set of
instruction encodings which software can use, and which hardware has to recognize and
implement. How that is implemented is a choice for the microarchitecture. So the
instruction set architecture is fixed, it's defined by Arm. The microarchitecture is
defined by whatever team of people is designing that CPU. And there are many different
approaches to implementing the Arm architecture, from very small, efficient cores with
in-order pipelines up to very high-performance, state-of-the-art, out-of-order
execution, and everywhere in between. So the microarchitecture is implementation-
specific, the architecture is generic, and software written for the architecture
should run on any microarchitecture. Why does Arm produce processors with different
instruction sets? Arm supports multiple instruction sets. Some of that is to do with
legacy: you can't abandon your legacy software, your legacy ecosystem. So as the
architecture has advanced and we've introduced major new instruction sets, we still
have to continue to support old software. It takes years, maybe 10 years to move the
software ecosystem to a major new ISA. So for example, AArch64, which is the 64-bit
architecture that we introduced with Arm V8, also supported the AArch, what we called
AArch32, the old 32-bit architecture that was implemented in the Arm V7 architecture,
and prior to that including the Arm and the Thumb instruction sets. And we needed to
do that because, while some software might start to migrate to the 64-bit
architecture, there's still a lot of software on the planet which is going to continue
using the 32-bit architecture, and that has to survive. So that's part of the reason:
it's about legacy. You can't just obsolete the whole world when you introduce a new
architecture, a new instruction set architecture in particular. There are other
reasons as well, which is there are certain instruction sets that are different for
reasons of the ecosystem that they're working with. So if you were to compare, for
example, the A-profile architecture that's designed for application processors that
run rich operating systems with virtual memory supporting SMP, symmetric multi
processing operating systems running large applications, whatever it may be: web
browsers on your phone or something, or a web server in a server farm somewhere. You
have your R-profile architecture, which is designed for high-performance, real-time
embedded systems. The constraints there are somewhat different. The instruction set is
actually fairly similar to the A-profile, but some of the underpinnings of the
architecture, the system side of the architecture, are simplified in order to give
more consistent and predictable real-time response to things like interrupts or memory
translation and suchlike for real-time systems. And then at the other extreme you have
the M-profile architecture which is designed to be capable of being built in a very
simple, ultra-low power implementation with low code size and again, similar to the R
profile, very predictable real-time performance. So the answer is there are different
instruction sets for reasons of the market that they're trying to address, and then
there are different instruction sets because, well, we have history. [music]

Module 3, Video 1
[music] In the previous module, we explored how pipelining can be used to improve
performance. We also saw how it is sometimes necessary to stall our pipeline to ensure
our program is executed correctly. In a simple pipeline, it will be necessary to stall
the pipeline whenever we encounter a branch instruction. This is because we must wait
until our branch is executed before we can be sure which instruction to fetch next. As
a recap, branches are instructions that change which instruction in the program will
be executed next. There are two types of branches: conditional branches and
unconditional branches. Unconditional branches always change which instruction
executes next, whereas conditional ones may or may not, depending on the computations
in the program. In real programs, between approximately one fifth and one quarter of
all instructions are branches, and the majority of these are conditional. Executing a
branch involves calculating the new address to load into our program counter. This is
the branch's "target address." However, conditional branches have an extra task: we
must first determine whether the branch is taken. If the branch is not taken, we can
effectively ignore the branch and fetch the next instruction as normal. Recall the
processor performance equation from an earlier video. Since we have to wait for
branches to complete before fetching the next instruction, we generate stall cycles.
These increase the average number of "cycles per instruction," which reduces our
microprocessor's performance. The longer our pipeline gets, the longer it is before
each branch is resolved, and the more costly branches become. Can you think of a way
to avoid some of this stalling? One idea is to evaluate branches earlier in the
pipeline, for example in the Decode stage instead of in the Execute stage. This can
indeed help to reduce the number of stalls, but we may still need to stall if the
branch depends on other instructions that haven't been executed yet. Another idea is
to continue fetching instructions in program order, effectively assuming that each
branch is not taken. The number of stalls in the pipeline for a not-taken branch is
zero in this design. On the other hand, if the branch is in fact taken, the subsequent
instructions that we fetched will be incorrect. So, we must remove all instructions
that have been fetched on this incorrect path from our pipeline. This is called
"flushing" the pipeline. Unfortunately, in real programs, branches are taken much more
than not taken. Could we then simply assume instead that all branches will be taken?
Sadly not, no, because then we would also need to know the specific "target address"
immediately, in order to know which instruction to fetch next. It may at first seem
impossible to know this before the instruction is decoded. However, computer
architects have found a way to do exactly this. The next video will look at "dynamic
branch prediction:" the idea of predicting the behavior of the branch instruction
before it has even been fetched. [music]

Module 3, Video 2
[music] In this video, we'll explore how to predict the behavior of a branch
instruction. This can sometimes eliminate the cost of branches altogether.
Fundamentally, a branch involves changing the value of the program counter, which is
the address of the next instruction in the program. If we could predict what this
change will be, quickly and accurately, we would have no need to stall. Precisely, we
need to predict that we are fetching a branch, predict it as taken or not taken, and
predict what its target address is. How could we ever make such predictions? What
would we base them on? Well, since instructions are often executed multiple times, we
can accurately make these predictions based on just the instruction address. If we've
previously seen that a particular address contains a branch, and we see that address
again, we can predict whether that branch will be taken, and its target address, based
on its behavior last time. Amazingly, for real programs, simply predicting repeating
behavior is typically around 90 percent accurate. This means we could eliminate stalls
caused by branch instructions 90 percent of the time. Let's apply these insights to
try to build a branch predictor. We will add two extra blocks to the Fetch stage of
the pipeline we met in Module 2. The first will remember information about recently
executed branches. This will include the program counter values of branches and their
target addresses. This memory is called the "Branch Target Buffer" or BTB. The second
block will make predictions about whether an address containing a branch is taken or
not. We simply call this the "branch predictor." In the next video, we will look at
these in more detail. Combining these two gives us all the information we need to
predict the next value of the program counter based solely on the current value of the
program counter. We don't even need to decode the instruction to make this prediction.
Here we can see how a running branch predictor behaves for a sample program. Each
program counter is checked in the BTB to see if it's predicted to be a branch and to
identify its predicted target. We also simultaneously check the branch predictor to
see if the branch is predicted to be taken. Based on these predictions, the branch
unit computes the predicted next program counter. Many cycles after the prediction,
feedback will be given by the rest of the pipeline as to whether or not the prediction
was correct. Whenever the prediction is wrong, we have to flush the pipeline and
update the BTB and branch predictor. The pipeline will then resume fetching from the
correct program counter as computed by the pipeline. The majority of instructions are
not branches, so most of the time the branch unit just adds 1 to the program counter.
The BTB contains the instruction address and target address of some recently executed
branches. The exact number of entries in the BTB varies considerably. BTBs in large
modern processors contain many thousands of branches. In operation, the BTB checks the
supply program counter against its memory to see whether it has a match, and if so, it
returns the target address. Otherwise, it predicts that the instruction is not a
branch. After each branch executes, the BTB is updated with the true target address.
BTB cannot be arbitrarily large, so it may have to forget an existing branch to
remember a new one. A simple BTB design like this is typically around 90 percent
accurate at predicting target addresses in real programs. [music]

Module 3, Video 3
[music] In the previous video, we met the two major components of dynamic branch
prediction, the BTB and the branch predictor. In this video, we'll take a deeper look
at the branch predictor. It predicts whether or not a branch will be taken, based on
the program counter. A simple branch predictor would try to remember what the branch
did last time and predict that the same behavior will repeat. Let's see how such a
predictor might be organized. Remembering a branch prediction for every possible
instruction address would take up far too much memory, so we reuse the same memory for
different branches via a process called "hashing." We hash the address of the branch
to a smaller number. This does unfortunately lead to a problem called "aliasing,"
where two different branches can hash to the same value, but this is rare in practice.
Let's see what happens now, when we execute a simple loop. We see a misprediction when
it encounters our branches for the first time, and another when we exit the loop. The
first case will be dependent on the value in our predictor's memory, and it may be
that we are able to predict the branch correctly the first time we see it. The second
case is hard to avoid, although some more sophisticated branch predictors will learn
how many iterations a loop will make. A common improvement to this simple scheme is to
avoid instantly flipping our prediction just because the branch does something
unexpected once. This can be achieved with a saturating counter, which instead
remembers how many times the branch has been taken recently, versus not taken. The
counter increments when the branch is taken and decrements when not taken. It predicts
"taken" if the majority of recent executions of this branch were taken. When building
higher-performance processors, we often have to discard many instructions every time
we mispredict a branch, so accurate branch prediction is very important. Therefore,
branch prediction is still an active area of research. One of the key ideas used is
correlation between branches. In real programs, a common pattern is for example a pair
of branches that always have opposite behavior. A branch predictor can take advantage
of this by remembering a history of whether or not recent branches were taken or not
taken and incorporating this in the hash. Another idea is to combine multiple
different types of predictor in a "tournament predictor." We use a "meta predictor" to
predict which of two branch predictors will do a better job. As you can see, branch
prediction can get quite complicated. [music]

Module 3, Video 4
[music] In this module, we've met the concept of branch prediction. Even using
relatively small simple circuits, we can accurately predict real branch behavior more
than 95 percent of the time. Could we do better, and do we really need to? In small,
simple processors, these prediction accuracies are fine, because each misprediction
causes only a few stall cycles. Increasing the accuracy is not that impactful.
However, in complex processors with very long pipelines, the difference between 98
percent and 99 percent prediction accuracy can be significant for performance as a
misprediction could incur dozens of stall cycles. Accurate prediction really does
matter. One of the problems we face in making accurate predictors is that they need to
be small enough and fast enough to fit in a microprocessor. We can imagine all sorts
of ways to do accurate branch prediction, but if their circuit would be slower than
simply executing the branch, they would not be useful. Modern high-performance
processors will often have multiple branch predictors, for example a small fast one
and a slower complex one. The slower one can override the prediction of the fast one
if it thinks it got it wrong, which does incur some stall cycles, but fewer than a
total misprediction. Another problem we face is that some branches are just very hard
to predict. No matter what technique is used, there are some branches in real programs
that are effectively "random". For example, when compressing or decompressing data,
the combination of the underlying algorithm and the input data may provide no clear
patterns to a prediction. No matter how hard we try, some branches will never be
predicted correctly 100 percent of the time. A final problem is that since the
predictors work based on observing the program, there will always be a period of time
when the predictors train on a program to learn its behavior. [music]

Module 3, Lab
[music] In this exercise, we're going to be using a branch predictor simulator to
explore dynamic branch prediction. This simulator will accurately simulate the details
of a branch predictor but it uses a trace of a real program executing on a real
machine to avoid the need to simulate the rest of the processor. Computer architects
use techniques such as this to quickly explore one area of process design
understanding that perhaps the accuracy may not be completely correct given that we're
not simulating the full process of design. The interface allows us to configure
details of our branch predictor for example the maximum of the saturating counters
used in the branch predictors table or the hash function. And we can see the impact of
changes on the delay of the branch predictor and also the design size which are two
key metrics when designing a branch predictor. Once we've happily configured the
design we want we can press run to simulate a program and the results of that
program's execution will be displayed in the rest of the stats. So for example, we can
see here that the predictor predicted 95.24% of the branches correctly. Which is
fairly good and this resulted in overall execution time of the program of 5.335
milliseconds. Just below we can see a table that records previous simulation runs as
well so we can do multiple experiments and see which produces the best results. For
example, if we were curious about the effects of using a saturating counter with a
maximum of three rather than one we could change that and notice that the design size
has substantially increased and when we press run, we noticed that the predictor
accuracy, however, has also increased. It's gone up to 96.31% and consequently the
execution time the program has fallen slightly and so we can compare these two designs
and see whether or not this represents a good trade-off for our processor. Perhaps the
area cost is justified or perhaps it's too much. It would depend on the exact
processor we're trying to design In the problems you'll be invited in to come up with
designs that are suitable for particular constraints. For example, particular
constraints on the runtime of the program or the design size. Once you've configured
the branch predictor that you think meets the objectives you can scroll down to the
submit button and click that and you'll be told whether the answer is great or not.
Good luck. [music]

Module 4, Video 1
[music] So far, we've looked at the microprocessor's "datapath"— meaning its execution
units, registers, and control circuitry. We have given less attention to its memory.
We usually implement memory using a different type of chip than the microprocessor,
using a technology called DRAM. It is very dense, allowing us to store lots of data in
a small area. However, one issue with DRAM is its speed. Since the 1980s, processor
performance has increased very rapidly at roughly 55 percent per year, so CPUs of
today are many orders of magnitude faster than those of 40 years ago. In contrast,
memory performance has grown much more modestly. Whilst memories are also much faster
than they were in previous decades, their performance has not kept pace with
processors. This leads to a processor-memory performance gap, with it becoming more
costly to access memory as time goes on. One of the issues that makes these memories
slow is their size. We can make a memory as fast as our microprocessor, if it is very
small. However, this is the opposite of what programmers want. Programmers can use
extra memory to solve more complex problems, or to solve existing problems faster.
This complicates microprocessor design because although we want a large memory to hold
all our data, large memories are slow and this slows down our processor. How do we
overcome this? What if we could get the speed benefit of a small memory alongside the
size benefit of a large memory? One solution is to have a large, slow memory and a
small, fast memory in our system. For this to be useful, we need the small memory to
hold the data that we use most often, so that we often get the speed benefits of
accessing it, and only rarely have to access the slow memory. Whilst there are
different arrangements of these two memories, the configuration that is most often
used is an "on-chip cache memory." The small memory sits inside the processor between
the pipeline and the large memory. We keep all data in the large main memory, and put
copies of often-used data in the small memory, which we call a cache. But we are not
limited to only one cache! Our pipeline reads memory in two places: when fetching the
instructions; and when accessing the data. It makes sense to have two caches here,
each optimized for different purposes. The instruction cache is optimized for fast
reading of instructions at Fetch. The data cache is optimized for reading and writing
data from the memory stage. We will often put a larger "level 2" cache between these
two caches and the main memory. The L2 cache is a "medium-sized" memory: faster and
smaller than main memory, but slower and larger than the two L1 caches. Using a
hierarchy of caches reduces the bandwidth requirements to main memory and the energy
cost of moving data around. [music]
Module 4, Video 2
[music] We previously looked at the need for a cache, which can be used to store
often-used data for fast access by the processor. But which data is used often enough
that it is worth including in the cache? Programs are mostly made of loops. Here's a
simple one that sums values in memory. It displays the two characteristics that we can
exploit to decide what data to put into a cache. Let's look briefly at how it works.
Each time round the loop, there are two loads and one store. The first load is to load
the data we're summing. The other load, and the store, are to update the running sum.
Notice two things here. First, we access the running sum over and over again; each
time round the loop. Second, when we access part of the data in one loop iteration,
we've already accessed its predecessor in the previous iteration, and will access its
successor in the next. Caches exploit two types of "locality" in programs in order to
be effective. The first is temporal locality: if a piece of data is accessed, it is
likely that it will be accessed again in the near future. The running sum has temporal
locality. The second type of locality is spatial locality: If a piece of data is
accessed, then its close neighbors are quite likely to be accessed in the near future.
By close neighbors, we mean data whose memory addresses are not far from each other.
The data accesses have spatial locality. It turns out that most programs exhibit lots
of temporal and spatial locality. We can exploit this to determine what to put in the
cache. Exploiting temporal locality is fairly easy: we simply see which values have
been accessed and place them in the cache. Exploiting spatial locality is also quite
simple: when a piece of data is accessed, place its neighbors in the cache. We'll see
in the next module how this is actually achieved. We use locality to guess what values
will be accessed next and store them in the cache. If we are correct, we get the
benefit of fast memory, but if we are wrong, we must perform a slow access to main
memory— just as we would if the cache were not present. In real programs, we see hit
rates of 90 percent or more in microprocessor caches, resulting in vast performance
improvements. However, it's worth noting that some programs have less locality, and
for those programs, caches offer little performance benefit. [music]

Module 4, Video 3
[music] We've looked at the reasons why we build caches, but how do they actually
work? To the outside world, a cache simply takes an address as input, and either
provides the data that is stored at that location at output, or returns a signal to
say that it doesn't have it. If the data is found, this is called a "cache hit". If
the data is not in the cache, this is called a "cache miss", and it means we must look
for the data in main memory instead. After each miss, we update the contents of the
cache. The fundamental building block of a cache is called the "cache line". It's a
number of data bytes from consecutive addresses in main memory. Cache lines are
typically 32 or 64 bytes long, and a cache typically has an array of hundreds to many
thousands of lines. The line captures spatial locality, because it is larger than the
data read by a single load instruction. When a cache miss occurs, the whole line
containing that data is copied to the cache from main memory, meaning we have nearby
values for future accesses. When a request comes in, we use some bits from that
address to index the line array. Just like in the last module on branch prediction,
this leads to the problem of aliasing again, since selecting only some of the bits to
index into the array is like a hash. This means that many addresses map to the same
line in the cache, but we can only store one of their lines of data. We need to note
down which addresses' line is currently stored in the data array, in what we call the
"tag array". There is one tag for each cache line. When we access the cache, we access
the tag array with the same index to see if the data we need is present. This design
is called a "direct-mapped cache". Direct-mapped caches work fairly well, but for some
programs, we can be unlucky with the program accessing to aliasing lines frequently.
We can do something about this by duplicating both the arrays, so that each line of
data can now be stored in one of two places in the cache. When we access the cache, we
look at both arrays and only get a miss if neither of the tags match. This is called a
"2-way set-associative cache", because each line has a set of two places or "ways" it
could reside. The "associativity" of this cache is therefore two. Set-associative
caches introduce a further complication: when we want to add data, where do we put it?
In a 2-way set-associative cache, there are two choices. How we decide which cache
line to evict is called the "replacement policy". There are many different types of
replacement policy. A simple one is just to make a pseudo-random choice from all the
possible cache lines. Another option is to keep track of when each cache line was last
accessed and to evict the one last used furthest in the past. This is called a "least
recently used policy", and takes advantage of temporal locality. It does, however,
means storing extra information in the tags to track usage. Many other ideas are
possible too. [music]

Module 4, Video 4
[music] Now that we've seen how caches work, let's see how they affect the performance
of a processor. Recall the processor performance equation, where the processing time
is proportional to the average cycles per instruction. Without a data cache, if 20
percent of instructions are loads, and main memory takes 20 cycles to access, our CPI
figure must be at least 5. However, if we provide a cache that holds the required data
80 percent of the time... ...and only takes 2 cycles to access, our CPI reduces to
2.2, which is a significant improvement! We can isolate the memory terms in this
equation to get the average memory access time —abbreviated to AMAT— which allows us
to compare different cache configurations more easily. Changing the cache
configuration will impact the AMAT. There are many different cache parameters we can
change, such as the size, replacement policy, associativity, whether we put data in
the cache for stores or just for loads, and so on. For example, reducing the size of
the cache will improve the access time for a hit, but will also increase the miss
rate. Let's say that we can halve the access time to 1 with a corresponding halving of
the hit rate. This alters the AMAT to 13, which in this case is worse for performance
overall. It's also useful to look at why an address might miss in the cache. Broadly
speaking, we can divide cache misses into three different categories. Compulsory
misses occur when we attempt to access an address that we have never seen before and
so never had the opportunity to cache it. Capacity misses occur when there is more
data being accessed than the cache could hold, even if we had complete freedom in
where to put each cache block. Conflict misses occur in caches where there are more
addresses hashing to the same index than arrays to hold the data. We can alter our
cache configurations to lower these misses, but as always, there are trade-offs
involved. Compulsory misses can be reduced by increasing the cache block size, to take
advantage of spatial locality. But for a fixed cache size, this reduces the number of
different addresses or cache lines that can be stored. A technique called "pre-
fetching" can also be used to predict the addresses that will soon be accessed, and
bring their data into the cache early. But this increases energy consumption, and may
make the cache perform worse if the predictions are not highly accurate. Capacity
misses can be reduced through increasing the size of the cache. Although, as we saw
before, this impacts the number of cycles taken to determine a hit. Conflict misses
can be reduced through increasing the number of cache blocks in each set, with an
increase in energy consumption as a side effect of this. [music]

Module 4, Lab
[music] In this exercise, we're going to be using a cache memory simulator to explore
the effects of cache memories on processor performance. The simulator accurately
simulates the behavior of the cache memory system, but it's using a model of the rest
of the processor to quickly allow us to simulate the effects of the cache on the
processor without needing to simulate the full processor. We can configure a number of
parameters about our cache, for example, the number of levels in our cache, whether or
not the cache separates instructions and data, or is unified, keeping them both in the
same cache. We can also configure the size, the line size, and the associativity, and
changing these numbers will affect design parameters, such as the access times in the
case of an L1 level one cache hit or a cache miss, and also the design size. Once
we're happy we've found design we'd like to investigate, we can press "run", at which
point the simulator will run through the program. We can see the hit rates for
instructions in the level one cache, and also data in the level one cache are
displayed. And we can also see the average memory access time, the results from this.
And then below everything, we can see a table of past simulations so that we can
quickly refer back to our previous experiments when we do new ones. So, for example,
let's say we were curious about the effects of increasing the size of the cache. If we
change the parameter and then press "run", we can immediately see that the design size
has substantially increased, which sort of makes sense because we've doubled the
contents of the cache, the size of the contents of the cache, and therefore we'd
expect roughly double the area. And we can also see that the hit rates have improved.
So there's about a 1% improvement to the L1 instruction cache hits and a 1%
improvement to the L1 data cache hits, which has reduced the overall average memory
access time. And so we can compare these two designs to see which of them we think is
better. It's a trade-off of course, though. The larger design has got better
performance, but it is larger, and so depending on the context, we may need to pick
the smaller design or the bigger design depending on our performance goals. In the
exercises, you'll be invited to come up with a series of designs for caches that meet
certain performance goals. For example, you'll have a constraint on the area of the
design or the execution time of the program, and you need to optimize the cache to
meet those goals. Best of luck. [music]

Module 5, Video 1
[music] In this module, we'll look at how to further improve performance by exploiting
"instruction-level parallelism." In Module 2, we explored how pipelining can improve
the performance of our processor. This reduced our clock period, and allowed execution
of instructions to be overlapped, improving throughput. One way to boost performance
further would be to create a much deeper pipeline. At some point, this would mean even
the ALU in our Execute stage will be pipelined. Consider the simple program shown in
the slide. Some instructions are dependent on a result from the previous instruction.
Remember in our 5-stage pipeline that these dependent instructions could be executed
in consecutive clock cycles with the aid of data forwarding. If execution takes place
over two pipeline stages within the pipeline, we need to stall if adjacent instruction
share a dependency. This allows time for the result to be computed. The programmer may
be able to rewrite their program to get the same result with fewer stalls, by placing
an independent instruction between our pair of dependent instructions. In this case,
we can move the third and fifth instructions earlier to optimize performance. The
performance of programs that run on our "super-pipelined" processor would, to some
degree, be determined by the availability of independent instructions that could be
executed in parallel in the pipeline. This is "instruction-level parallelism"—or ILP.
Very deep pipelines are problematic as they would require: a very high-frequency clock
to be distributed across the chip very precisely; careful balancing of logic between
many, very short, pipeline stages; the pipelining of logic that is difficult to divide
further into stages; and the division of logic at points requiring many pipelining
registers to be inserted. A different approach to exploit ILP is to make our pipeline
wider rather than deeper. In this design, the processor will fetch, decode, and
potentially execute multiple instructions each cycle. Such a design avoids the
problems of a super-pipelined processor, although as we'll see in the next video, it
does introduce some new complications. [music]

Module 5, Video 2
[music] In this video, we are going to explore "superscalar" processors, which can
process multiple instructions in each pipeline stage. In our simple 5-stage pipeline,
there is at most one instruction per pipeline stage. At best, we can complete one
instruction per cycle. We call such a design a "scalar" processor. In a 2-way
superscalar version of this processor, we would extend this design so it is able to
fetch, decode, execute and writeback up to two instructions at a time. In general,
superscalar processors may vary the number of instructions that can be processed
together in each stage. Let's step through the design. Our instruction cache will need
to supply two instructions per cycle. Typical superscalar processors only ever fetch
adjacent instructions on a given cycle. This can lower performance if, for example,
the first instruction fetched is a taken branch, because then the second would not be
required. Note that now every cycle lost due to control hazards will cost us two
instructions rather than one, so accurate branch prediction matters even more in
superscalar designs. The Decode stage must now decode and read the registers for two
instructions simultaneously. Fortunately, we are able to extend the register file
design to read many register values at the same time. The Decode stage also needs to
check whether the two instructions are independent. If so, and if the functional units
they both need are available, it can "issue" them for execution in parallel on the
next clock cycle. Otherwise, in this simple design, it will only issue the first, and
keep the second back. A simple design such as this—where two instructions are fetched,
decoded and issued— is called a "2-way" or "dual-issue" processor. In other designs,
the width may vary at different stages of the pipeline. To support the execution of
multiple instructions at the same time, the Execute stage is expanded and contains two
execution pipelines. It's common for these to have slightly different capabilities to
save area. For example, the top pipeline can execute both ALU and memory instructions,
while the second pipeline only executes ALU instructions. To ensure that dependent
instructions can execute on consecutive clock cycles, we must add data forwarding
paths. These data forwarding paths must allow results stored in either execution
pipeline to be forwarded to the input of either ALU. During writeback, we need to
store both results to the register file. This means the register file must be
redesigned to allow two writes per clock cycle. Overall, these changes typically
require 25 percent more logic circuitry in our processor, compared with a scalar
processor. But we'd expect an improvement in execution time of between 25 and 30
percent for real world programs. [music]

Module 5, Video 3
[music] We've seen that instruction-level parallelism can be used on superscalar
processors, to run them faster than would be possible on a scalar processor. But how
much of a speedup is this in practice? Ultimately, this depends on how much
instruction-level parallelism is possible in a typical program. How might we measure
this? We can do this initially without considering any constraints that will be
imposed by the processor it will run on. Let's consider the instructions executed by
the program. Let's assume that we can predict all the branches in the program
perfectly. Then we can ignore branch instructions, as they don't need to flow down our
pipeline. Now let's imagine we can execute any instruction as soon as the data it
needs is ready. That is, we are only restricted by the presence of true data
dependencies. Note that some dependencies are carried through writes and reads to
memory. Rather than considering program order, we can now just look at the order the
dependencies impose on instructions. This is referred to as "data-flow analysis."
Assuming each instruction takes exactly one cycle to execute, the fastest possible
execution time of the whole program in cycles is given by the longest path in the
data-flow graph. The instruction-level parallelism of this program is the number of
instructions divided by this duration, as this gives the average number of
instructions we would need to be able to execute each cycle to achieve this duration.
In real programs, this can be anywhere from around five, to hundreds or even
thousands. An active area of research and innovation for computer architects is to
imagine processor designs that can expose and exploit as much of this parallelism as
possible. One insight architects have had is that superscaler processors need to have
a fast supply of instructions to be able to analyze dependencies effectively. This
often means that the front end of our processor pipeline is much wider than the rest
of the pipeline, so that it can "run ahead" and see what behavior the program will
have next. Fast and accurate branch prediction is vital, as we often have to predict
multiple branches ahead accurately, to achieve good performance. Another key insight
is that we don't have to wait to execute the instructions in program order. If all the
dependencies of an instruction are satisfied, the instruction can proceed down the
pipeline even if previous instructions are yet to execute. This can reduce program
execution time by taking advantage of more instruction-level parallelism. In practice
though, this creates extra complications, as we will see in the next module. [music]

Module 5, Lab
[music] In this exercise, we will be using a simulator to explore superscalar
microprocessor design. The simulator has a number of parameters that we can configure,
such as the number of pipeline stages, the width of the Fetch stage, the width of the
issue, and the number of ALUs in the design. We can see a diagram of the processor
that we've created and we can also see a number of parameters about that design, for
example the clock frequency and the overall area of the design. When we press step,
the simulator will advance one clock cycle and so for example here we can see that the
fetch stage has fetched the first 4 instructions. However, immediately we see one of
the problems with designs such as this, which is the three of the four instructions
that have been fetched are in fact useless because the first instruction was an
unconditional taken branch and therefore the remaining three instructions will not be
executed by the program and so these will immediately be discarded on the next cycle.
Pressing the run button allows us to simulate and we can use the fast-forward option
to simulate much quicker in order to get to the end of the long program execution. In
this case we can see that our design achieved an average cycles per instruction less
than one, which is to say that we on average executed more than one instruction per
cycle, which means that our superscaler design has fundamentally worked, and we can
see that, for example, the overall execution time of the program is 1.6 milliseconds.
In a table below, we can also see a record of our previous simulation runs. So let's
say for example, we were curious about the effect of increasing the issue width by
one. We can make that change and then press run again in order to run our new design,
and when it finishes we can scroll down to take a look and we can see that the program
execution time has indeed improved down to 1.51 milliseconds at a cost of only 1%
area. So it looks like this was a very good improvement to our design and it's almost
surely going to be a beneficial trade-off in practice. In the exercise you will be
invited to configure a number of different superscalar processor designs with various
targets in terms of clock frequency, design area and execution time. Once you're happy
that you've configured the process that you think completes the exercise, you can
scroll all the way down to the bottom, where you'll see the submit button that you can
press to have your answer checked. Good luck. [music]

Module 6, Video 1
Introduction:

So hi, my name is Peter Greenhalgh. I'm Senior Vice President of Technology and an Arm
fellow. I'm responsible for the Central Technology Group at Arm. We're about 250
people. We work on everything from machine learning to CPU, GPU, system IP, and the
solutions that we create as well. And we basically path-find future technology at the
product level that goes into all of our products and the IP that we produce. Arm is
known for the power efficiency of its microprocessors. How have you managed to keep a
focus on power when building processors with very complex and power-hungry features?
We've got some really great design teams. In fact, we churn out I think more CPUs than
pretty much anyone else does on the planet. I think we're producing something like
four or five CPUs per year. So we've got a lot of experience in designing for power
efficiency and performance, and in fact we can leverage the understanding that we have
all the way down to microcontrollers through to the smaller A-class processors, all
the way up to the high performance. There's a lot of sharing between the teams in
terms of strong knowledge and capability, and insight into how to design for both
performance and power efficiency. More specifically, I mean, ultimately, you have a
performance goal that you need to achieve, and then as part of that you have to figure
out how to get the best possible power out of the design when you're achieving that
performance goal. And to do that, there's kind of some different ways of looking at
it. There's the really detailed orientated work that you need to do around things,
like clock gating, data gating, all the things to try and stop unnecessary power use
deep within the microarchitecture when the instructions are flowing through the
pipeline or data's moving through the pipeline. And then there's essentially the
structure of the design that you've created. And that then dictates fundamentally what
the power of the design is going to be. You can't fix a design that's got bad
structure with improved clock gating, data gating, and just good low-level design. You
have to marry the two together. And that high-level work that you do is around making
sure that the pipeline is well balanced, that you aren't opening up the pipeline,
going too wide too soon; you're extracting data, you're extracting information as late
as you possibly can and just when you need it, and not just pipelining it down through
the design for the sake of it; and then, fundamentally, good microarchitecture around
branch prediction, which stops you putting things down through the pipeline that
you're just ultimately going to flush; good pre-fetching on the data side so that you
make sure you get the data in the design when you need it, and you're not sitting
around waiting for it. So you have to marry that altogether, and we've got a lot of
great techniques in order to achieve that, which fundamentally, I say, you need to
achieve the performance target, and then everything else comes together to achieve
that performance target in the best possible energy efficiency. How did Moore's Law
affect computer architectures of the past, and what will its influence be on future
designs? Gordon Moore's influence on the industry has been massive, and the tenets
behind the law still continue today, albeit in a slightly different form. I mean, I
started designing at .18 Micron, essentially 180 nanometers, and here we are today
working on 3 nanometers. So it's a vast difference now compared to when I started 22
years ago. And there's no way we could have got to where we are today without the
process scaling from all of the foundries out there and all the companies that provide
the foundry technology. So it's a little bit like magic, all of the work that they do.
I can't say I understand it in detail, but it's incredible technology, and that allows
us... If it hadn't happened, we'd still be stuck in all the designs which were fairly
simple. There's no way that we'd have got to the sort of designs that we have today of
massively out of order, deep, deep amount of instruction and depth, very, very wide
designs. All of that has been made possible by the steady improvement, a predictable
improvement of the foundries. And that's kind of one of the key points which really
was captured by Moore's law or is captured by Moore's law of that kind of predictable
knowledge that you will get an improvement in the process. Is it 10%, is it 15? Is it
20% on, say, power, for example? It kind of doesn't matter in a way because you can
work with what you eventually get. You can do things like voltage scaling to be able
to make use of the power that's available to you. Is it 5%? Is it 10% on frequency?
Again, it kind of doesn't matter in a way. But what matters is when we start designing
a processor today and we finish it in 18 months time, and then two years after that it
arrives in the product in the shops that consumers can buy. We know that over that
period, the process improvements have happened, which allows us to liberate
essentially more performance, more energy efficiency from the design. And we don't
mind too much if it takes another three months or six months to get to the process. We
don't mind too much if the performance or power is not exactly where it was predicted
at the start. But, ultimately, we know we'll get an improvement, and we know there'll
be an improvement in two years, and three years, and four years, and Moore's Law may
have slowed, but it's certainly not stopped.

[music] As we saw in the last module, instruction level parallelism can be used to
improve program execution time in our microprocessor designs. To enable this, the
compiler creates an optimized instruction schedule when the program is converted into
machine code. Unfortunately, the compiler cannot know precisely what will happen at
run-time, so this design is constrained by the order of instructions in the program.
The compiler won't know what the program's input data will be, whether branches will
be mispredicted, or whether memory accesses hit or miss in our data cache. In
contrast, a superscalar processor with "out-of-order" execution can produce an
instruction schedule at run-time, only constrained by true data dependencies and its
hardware limits. This schedule is produced on demand and so can even change each time
the code runs. To do this, we introduce an "issue window" or "issue queue" after the
Decode stage. This holds instructions until they can be executed, not necessarily in
the order they arrived in. Within this window, instructions can be issued whenever
their dependencies are available, and when a functional unit is available to process
it. To be able to detect when an instruction is ready to be issued, we must know
whether the instruction's dependencies are ready when it enters the issue window. We
must then update this status as new results are produced. To implement this, the names
of result registers of executed instructions are broadcast to the issue window. The
instructions waiting there compare the register names to the registers they require.
However, this scheme has a problem: A register will be written multiple times in the
program, and since the instructions are executed out-of-order, the register name alone
is not sufficient to record dependencies. It also means that instructions would have
to wait until all previous reads of a register had finished before executing. These
are called "false dependencies." These problems can be resolved by "renaming" register
names at run-time so that each "in-flight" instruction writes to a unique destination
register. We use a "physical register file" that is large enough to ensure we don't
run out. We keep a "register mapping table" to store the mapping between
architectural, compiler-assigned registers, and physical registers. Register reads to
the same architectural register are renamed consistently, so that dependencies can be
tracked correctly with physical register names. Physical registers are reused only
when they are no longer used by any instruction currently in-flight or any entry in
the register mapping table. The other big issue with out-of-order execution is memory
dependencies. Load and store instructions can have memory dependencies because they
access the same memory location. To detect this, we need to compare the computed
memory addresses that the instructions access. We thus split memory operations into
two steps: address calculator and memory access. We issue their address calculation
step as soon as the dependencies are available. Then, the memory access step is placed
in a special load-store queue to be sent to our data cache as soon as possible. We
carefully ensure that operations that access the same address are kept properly
ordered, but independent accesses can be reordered if beneficial. No access can occur
until the addresses of all previous accesses are known. Since memory writes are
irreversible, store instructions must also wait until we are certain that they will
execute. [music]

Module 6, Video 2
[music] In the previous video, we outlined the concepts of out-of-order execution, and
register renaming. The issue window will be filled with instructions fetched along the
path that our branch predictor believes the program will take. While we hope our
branch predictor will be correct in most cases, it will sometimes be wrong. How do we
handle such cases? A simple approach is to start by recording the original program
order of the instructions, and then to monitor their progress. We call the structure
that stores the instructions the "reorder buffer." As each instruction executes and
produces a result, we can mark it as done. When the oldest instruction has completed,
we can remove it from the end of the reorder buffer, and the instruction is said to
have "committed." This stream of committed instructions represents how our program
would be executed on a simple in-order pipeline or by an unpipelined processor. It
usefully also provides a point at which we can process exceptions. For example, if the
program divides by zero or attempts to access memory that does not exist. We also
check branch instructions as they complete in order. If they have been mispredicted,
we flush the reorder buffer, our instruction window and any currently executing
instructions and start fetching down the correct path. To preserve correctness, we
must also restore our registers and the register map table to the values they had when
we mispredicted the branch. This can be done with the aid of a second register map
table, updated only when instructions commit in program order. This can simply be
copied to the map table used by our renaming hardware to "rewind time" for the
processor. All the register values we need will be present, as we don't recycle
registers before we know they will not be needed again. In reality, handling branches
in this way is too slow. Processors instead take many copies of the register map
tables and can handle branches as soon as they are resolved, and we discover they have
been mispredicted. They can also selectively neutralize the in-flight instructions in
the datapath that are on the wrong path, rather than flushing all of these
instructions away. [music]
Module 6, Video 3
[music] We can now bring everything together and look at what a typical pipeline for
an out-of-order superscalar processor might look like. The Fetch stage is aided by an
accurate branch predictor as we met in Module 3. It will fetch a group of instructions
on every clock cycle. This group of instructions will be requested from the
instruction cache, and will be from consecutive memory locations. Branches may reduce
the number of useful instructions that can, in practice, be fetched in on each cycle.
The Decode stage decodes multiple instructions in parallel. At this point, modern
high-performance processors may also split complex instructions into simpler
operations or "macro-ops." In some cases, there may also be opportunities to combine
simple instructions into a single operation. The next step on an instruction's journey
is renaming to receive a unique destination register. As we saw in the last video,
this increases opportunities for out-of-order execution. Remember, there are several
times more physical registers in our processor than those available to the compiler.
Instructions are placed in the reorder buffer, and are also "dispatched" to the Issue
stage. They will wait in the window as necessary, and are ready to be issued once all
their operands are available. In the most complex of today's superscalar processors,
there may be hundreds of instructions buffered in the issue window at the same time.
Instructions finally commit in program order. At this point, any physical registers
that are no longer needed can be added back to the pool of free registers. These are
then assigned during the register renaming step. Once an instruction is issued, it
reads its operands from the physical register file. The Execute stage consists of many
functional units operating in parallel. These may each support different operations
and take different numbers of cycles to execute. A network of forwarding paths is also
provided to ensure we can execute any dependent instruction on the next clock cycle
after the generation of the result. This requires being able to quickly communicate—or
"forward"— a result from the output of any functional unit, to the input of any other.
Some instructions will need access to memory. After computing their addresses, they
are placed in the processor's load-store queues. "Stores" are sent to memory in
program order, but "loads" can often be sent out of order, and ahead of other older
stores or loads that are not yet ready to be issued to memory. The memory system
reduces the average memory access time by providing numerous levels of cache memory.
After generating results, we write them back to the register file. This overview is
representative of the fastest modern microprocessors found today in laptops,
smartphones and servers. Whilst much extra innovation goes into real designs, they
generally follow the ideas discussed in the course. [music]

Module 6, Video 4
[music] One question computer architects always ask themselves is: "how much can we
scale up our design?" Let's take a look at some further potential optimizations to our
out-of-order superscalar processor. We could try to make it wider. For example, by
doubling the number of parallel instructions, we can fetch, decode and execute more
instructions per cycle. Would this double our performance? Sadly, no, things are not
that simple! In practice, some components quickly become very complex, and performance
gains may be hard to extract. For example, today's largest machines fetch at most ten
instructions per cycle from their instruction caches. Fetching more instructions than
this offers minimal performance gain, despite a large hardware cost. If we increase
the number of registers, the size of our issue window, the size of our load-store
queues, or perhaps use a larger and more accurate branch predictor, our processor's
performance will only improve slightly despite a significant increase in the size of
these structures. After a point, the increase in performance is no longer worth the
cost of the extra transistors. It's also possible that performance might reduce
overall as we may need to lower our clock frequency as the structures get larger.
Finally, we could introduce more pipeline stages, but we know this doesn't necessarily
lead to higher performance, as mispredictions may become more costly. The combination
of these issues means that extracting performance using instruction-level parallelism
alone becomes more expensive as more performance is sought. This graph shows how the
energy cost of executing an instruction grows quickly as we try to build higher
performance processors. Let's look at some example designs. Suppose we have a core,
which requires a certain area. If we double its area, its performance improves,
although there is a small rise in energy per instruction. If we quadruple its area
instead, its performance has now doubled compared to our original core, while energy
has increased by 50 percent. Going further, if we increase our processor's area by a
factor of 10, performance is only 2 point 5 times our original core, but energy per
instruction is now 3 times higher. Its performance does not improve as fast as the
cost of running it! Of course, engineers are clever and determined, and are constantly
developing new techniques to bypass many of these issues. This means the performance
of processors—even ones running a single thread or program— still improves by around
10 to 25 percent each year. Nevertheless, ultimately we often need more performance
than can be provided by instruction-level parallelism alone. A modern solution is to
employ multiple processor cores on the same chip—called a "multicore" processor. This
changes the task for programmers; they may need to redesign their programs to take
advantage of such parallelism, but if they can, it can give vast performance benefits.
As we've learned throughout the course, every decision involves trade-offs and
compromise. We are faced with a fascinating but often highly-constrained design
problem. We've seen how performance bottlenecks, that at first seem impassable, can be
overcome with innovative designs. What might the future hold for microprocessors? Can
you think of ideas? What would you design? [music]

Module 6, Lab
[music] In this exercise, we'll be using a simulator to explore an out-of-order
superscalar processor design. The simulator allows us to configure a number of aspects
of our processor, for example, the number of pipeline stages, the width of the Fetch
stage, the size of the issue window, the number of ALUs, and the size of our re-order
buffer. The changes will be reflected in the pipeline diagram, which we can see below,
and also in the statistics below that, with key design metrics, such as the clock
frequency, clock period, and design size, visible below. When we press "step," the
simulation will advance by 1 clock cycle, and so we can see, for example, on the first
clock cycle the 1st 4 instructions are loaded. Although three of them are unusable
because they follow a taken branch and therefore these will not be executed and will
be discarded on the next clock cycle. We can press "run" to watch our design in action
and in order to quickly get to the end, we can use the "fast forward" feature to
simulate the millions of instructions in this particular program. After the simulation
is complete, we can check below to see a number of statistics about our pipeline,
which are useful for understanding why the performance is as it is, and in particular
we can see the program execution time. 1.18 milliseconds gives us the overall
execution time of the program that our design achieved, and notably our average
instructions per cycle gives us the number of instructions that we were able to
complete on each clock cycle, in this case well over 1, indicating that we are taking
advantage of instruction level parallelism in this simulation. Below all that, we can
see our completed simulations and we can check back on these as we explore new
designs. So let's say, for example, we were curious about the effect of increasing the
re-order buffer size. We could change that and then press "run" to quickly run our
next experiment. And then if we scroll down to our table, we can compare the results
and see that actually the program execution time was substantially increased by
increasing the size of the re-order buffer. Although, admittedly, this did come at a
not insignificant increase in the area of our design, and so whether or not this
represents a good trade-off in practice would very much depend on the problem we're
trying to solve. In the exercises, you'll be given a number of scenarios for
processors, which generally revolve around certain constraints on the design size, or
the clock frequency, or the target program execution time. And once you think you've
found a design that meets the goals required, configure it up using the settings and
then scroll down to the "submit" button at the bottom, and click that to have your
answer checked. Best of luck. [music]