Computer Architecture Essesntials by ARM

Module 1, Intro
[music] I'm Richard Grisenthwaite, Chief Architect at Arm. What does a Chief
Architect do? The Chief Architect is responsible for the evolution of the Arm
architecture. One of the things about an architecture is, you want your software to
continue running on future products. So it's a very long term commitment, and the
role of the chief architect is essentially to curate the architecture over time,
adding new features as needed by the partner. Why are microprocessors so important
and ubiquitous? I think the fundamental reason is that they are a general purpose
device. They describe a basic instruction set that does the stuff that you can
construct pretty much any program out of. Indeed, they're Turing complete, which
means, by definition, you can create any computable problem, solving a computable
problem with the processor. They're not customized to any particular usage, but they
do two things really rather well, which are data processing and decision making. In
other words, having, for example, added two numbers and compared them, you're then
taking a branch and making a decision based off that. Now in reality, an awful lot of
general purpose problems are ones that involve: work out some value or some criteria,
compare against that, and then make a decision on it. And essentially that allows you
to solve a huge number of different problems and that's why the microprocessor has
become so ubiquitous. What is the difference between architecture and
microarchitecture? Architecture is what it has to do, and microarchitecture is how it
does it. So for example, we will define in the architecture a set of instructions
that do "add" or "load" or "branch", but it says nothing about whether you've got a
ten-stage pipeline or a three-stage pipeline. It says nothing about branch
prediction. All of those sort of features which actually determine the performance of
the device; all of those are part of the microarchitecture. The key point is that
software that is written to be compliant on the architecture will actually run on
lots of different implementations, lots of different microarchitectures that
essentially implement that architecture in different ways, choosing a trade-off
between the power, the performance and the area of the design. What does it take to
build a commercially successful processor today? If you actually start from scratch
with an established architecture, but we want to create a new microarchitecture, we
reckon, for a high end processor, you're talking about three-to-four hundred person-
years' worth of work in order to create a properly competitive multi-issue out-of-
order machine compared with the state-of-the-art that you can get from Arm. In
addition to that — and that's just to come up with the RTL — in addition to that,
you've then got to do the implementation. If you're going to build that on a three-
nanometer process, the leading edge to get the best possible performance, you're
talking about tens of millions of dollars for the mask sets. There's a whole bunch of
software you've got to go and build on top of that and do all of that work. When
we've looked at companies that are interested in actually starting up, taking an Arm
architecture license — say I want to go and build my own business — we reckon that
you need to be prepared to spend getting on for half a billion dollars before you're
actually going to be successful because it takes time. Your first product is not
necessarily going to be fully competitive because it would be slightly surprising if
the first thing that you built was as good as what people have been building for many
years. It takes time to build up that expertise and skills. And so you're going to
see a couple of iterations before even the best teams end up really being
competitive. And so as I say, I think if you went from nothing and wanted to
essentially create something, and that's using the Arm architecture with all of its
existing software and so on, you're talking the best part of half a billion. What is
the Arm business model? The Arm business model fundamentally is the licensing of
semiconductor IP to as wide a range of companies as possible and in as many ways as
possible, in order to maximize the uptake of the IP. When we talk about IP, we're
talking about essentially designs for processors and we license that both as an
architecture license, where you effectively gain the right to build your own
processor to the specification of the Arm architecture, or an implementation license
where we are licensing, actually, an implementation that is compliant with the Arm
architecture in the form of some richer transfer-level RTL. What makes Arm different
from its competitors? So if you go back to where we were when Arms started out in the
early 90s, there were many, many different architectures available and they were all
kind of doing the same thing but slightly differently. They would have different
instructions, different instruction sets, and so software needed to be ported. A huge
part of Arm's success actually comes from the fact that we created a business model
of licensing the IP to make it very easy for people to build processors, to
incorporate the designs into their SoCs, into their systems. And that then made it
very straightforward for people to be able to use the Arm architecture. Now what this
then meant was, people said: I will port more software to this because it's more
widely available and you get this positive feed-forward effect whereby more
availability of hardware encourages more software, encourages more hardware, and so
on. And essentially that meant that a lot of people said: there's no point in me
having a different architecture. I'm not getting a valuable difference from doing
that. All I've got is, kind of, a needless change to the software that I need to
make. So actually the whole model, we went on at Arm, which was: let's license our IP
to a great number of different players to come up with different solutions to meet
the form factor of a camera or a PDA, or whatever it was back in the day. Those
things made it much more straightforward for people to incorporate our technology.
[music]

Module 1, Video 1
Computer architecture is the study of tools and techniques that help us to design
computers. More precisely, it helps us understand how to meet the needs of particular
markets and applications, using the technology and components that are available. For
example, we might need to produce the chip at the heart of a smartphone using 10
billion transistors and a power of less than two Watts. How do we achieve the
performance a customer wants? The challenge is a fascinating one, and one that
requires a broad understanding. For example, what target market are we designing for?
What are the characteristics of the applications to be run? How will programming
languages and compilers interact with the microprocessor? How best to craft the
narrow interface between the hardware and software? How to organize the components of
our microprocessor? And how to design a circuit, given the characteristics of
individual transistors and wires? Like many design problems, computer architecture
requires many trade-offs to be made and evaluated. Each design decision will impact
trade-offs between size, performance, power, security, complexity, and cost. Trade-
offs must be re-evaluated regularly, due to advances in fabrication technology,
applications, and computer architecture. Computer architecture must be grounded in
quantitative techniques and experimentation, but the endless number of possible
designs means that the field depends on a high degree of human ingenuity and art.
Perhaps surprisingly, the earliest computers and today's most advanced machines have
much in common. They both execute a stored program constructed from machine
instructions. These instructions perform simple operations such as adding two
numbers. Nevertheless, greater numbers of faster transistors, and the application of
a relatively small number of computer architecture concepts, have enabled us to
construct machines that can perform billions of instructions per second, and shrink
these machines to fit in hand-held battery-powered devices. It is this rapid progress
that has supported breakthroughs in machine learning, drug discovery, climate
modeling, and supports our modern world where computation and storage are almost
free. The task of designing a microprocessor is split into different levels of
abstraction: "Architecture;" "Microarchitecture;" and "Implementation."
"Architecture" focuses on the contract between programmers and hardware. It allows
compatible families of microprocessor products to be built. The ARMv8-A architecture
is an example of this. Architecture includes the "Instruction Set Architecture," or
ISA, which defines what instructions exist. It also defines precisely the behavior of
memory and other features needed to build a complete processor. "Microarchitecture"
focuses on the organization and structure of the major components of a
microprocessor. It has to match the rules set by the architecture. The
microarchitecture still has flexibility though; and so the implementation specifies
the circuit detail precisely. This culminates in the exact circuit design for
manufacture. Each of these levels is vital, and each comes with its own challenges
and opportunities.

Module 1, Video 2
At the beginning of the 20th century, "computers" were people employed to perform
calculations. These computers used mechanical calculators to help them perform
arithmetic. They followed instructions to decide what calculation to perform next.
These instructions defined the algorithm or program they were executing. They would
consult paper records to find the inputs to their calculations, and would store their
intermediate results on paper so they could refer back to them later. A modern
electronic computer is organized in a similar way. We will look in more detail at
these components of a microprocessor in the next video, but for now let's look at how
it operates. Microprocessors also follow instructions one by one, and then perform
relevant calculations. This idea of fetching instructions from memory and executing
them is called the "Fetch-Execute Cycle." In the "Fetch" stage, the computer reads
the next instruction from the program. This instruction is encoded in binary as ones
and zeroes, so it must be decoded to understand what it means. This is done in the
"Decode" stage. Once it is clear what to do, we move to the "Execute" phase. This can
involve different tasks such as reading memory, performing a calculation, and storing
a result. Once done, the computer is then ready to begin the cycle again, by fetching
the next instruction. Instructions are normally fetched sequentially in order, but
some special instructions called "branches" can change which instruction will be
executed next. For branches, a calculation determines the next instruction. This can
mean evaluating a condition, or reading a register to determine the next
instruction's location. Branches allow computers to make decisions, and to re-use
instruction sequences for common tasks. A modern computer program, like a web
browser, contains millions of instructions, and computers execute billions of
instructions per second, but they all conceptually follow this "Fetch-Execute Cycle."

Module 1, Video 3
Modern microprocessors are circuits built using anywhere from 1,000 to 100 billion
tiny transistors. The key to designing circuits with such huge numbers of parts is to
build them from re-usable blocks. Let's take a look at some of these. Each transistor
is an electrically-controlled switch. When there is too little voltage at the gate,
the switch is off, so the electrical signal cannot propagate from the drain to the
source. However, when there is sufficient voltage at the gate, the switch is on, so
the signal does propagate. When designing processors, we use digital electronics. The
only voltage or current values we consider represent zero and one, enabling us to
build robust circuits from imprecise components, even in the event of electrical
noise or manufacturing imperfections. This means our circuits represent binary
numbers, since they only have two states: zero and one. We use transistors to build
increasingly complex circuits. We can design circuits that can remember binary
values, or select between multiple inputs, or even do arithmetic, like addition. We
can then use those circuits as building blocks in even larger designs. When designing
a digital system, we must keep everything synchronized to control when its behavior
occurs. To do this, we use a "Clock Signal," which is a wire in the circuit whose
signal cycles between zero and one. We measure the rate in Hertz. For example, if you
hear a processor has a clock speed of two Gigahertz, it means 2 billion ones and
zeroes per second. The maximum speed of the clock signal is determined by the
longest, and therefore slowest, path in the circuit between 2 clocked flip-flops.
This is referred to as the "Critical Path." The signal must have time to propagate
all the way along the critical path before the clock cycle completes. A
microprocessor needs to do many types of arithmetic. For this, we build an
"Arithmetic Logic Unit" or "ALU." This circuit receives 2 numbers as input, as well
as an indication of what operation to perform, for example, addition or subtraction.
In addition to logic, a microprocessor needs memory. Memory is organized as arrays of
memory cells that are able to store many "Words" of data. A specific word, commonly
32 bits in length, can be accessed by specifying its "Address." Each address is a
number that indicates the location in the memory that should be read or written.
Memory cells range from hundreds of bits to millions of bits in size, but larger ones
are slower to access, as signals and their long internal wires take longer to
propagate. For that reason, almost all microprocessors include at least two types for
storing data: a big slow "Data Memory," and a small fast memory called a "Register
File." In reality, the "Data Memory" may be implemented using many different sizes of
memory, as we'll see in Module 4. As well as storing data in memory, we also use some
memory to store the instructions. We need a way to keep track of which instruction we
will fetch next, so we have a "Program Counter." This stores the address in
Instruction Memory of the next instruction to be accessed. Since instructions are
encoded in binary, we also have "Instruction Decode Logic" that converts that binary
to the various signals needed to control the microprocessor.

Module 1, Video 4
We've already seen how a microprocessor is controlled by instructions. But what are
they really? An instruction is a simple command that the microprocessor hardware can
perform directly. We write them as text like this to make them easier to read. But
for the microprocessor we encode them in binary. We use a program called an assembler
to translate between the human text and the binary. In this video we'll be looking
specifically at an Arm instruction set called A64 but other instruction sets follow
similar principles. Arithmetic and logic instructions are the simplest type of
instruction. The first word tells us what operation will be performed such as
addition or multiplication. The values after this tell the processor where to put the
result and where to get the inputs. Values starting with X are addresses in the
register file. Arithmetic instructions read one or two registers and then put the
result into a third register. Branch instructions are used to make decisions and to
repeat instructions. Normally the microprocessor executes instructions in sequential
order but branches change that, and explicitly tell the microprocessor the address of
the instruction to run next. This is done by giving the address of the next
instruction in the instruction memory. Some branches are unconditional, meaning they
always occur and always affect the next instruction address. Other branches are
conditional, meaning the processor will perform a calculation to decide whether to
follow the branch or to continue executing instructions sequentially following the
branch. These are preceded by comparison instruction to calculate the condition.
Loads and stores are the instructions for accessing the data memory. Loads copy
values from memory to the register file. Stores do the opposite. In both cases, the
instruction needs to know the address in the data memory and the location in the
register file to copy between. For data memory, loads and stores read an address from
a base register. They can also optionally add to this base address by reading another
register, or by simply specifying a number in the instruction itself. Using sequences
of instructions we can build programs. Here is an example of a small program that
implements Euclid's greatest common divisor algorithm. Let's take a look at it
working, one instruction at a time. To start with, the input stored in X1 and X2 are
compared. If they're equal, because we have found the greatest common divisor a
conditional branch instruction moves to instruction 7. If they're not equal, another
conditional branch instruction can be used to determine whether X1 is smaller.
Finally, we use an arithmetic instruction to subtract either X1 or X2 depending on
which was larger and then unconditionally branch back to the start. Here we can see
an example of the program running. One instruction executes at a time. As we step
through the program, the values in the registers X1 and X2 are shown. And we see
these are modified each time we reach instruction 3 or 5. The instructions repeat in
a loop, as is common for many different types of program. Each time round the loop
instruction 0 and 1 are checking whether or not to continue the loop. When they
detect X1 and X2 have become equal, the program finishes and the processor moves on
to the next task.

Module 1, Lab
In this exercise, we're going to be using ASim. ASim is a behavioral simulator for a
subset of the Arm AArch64 instruction set. Behavioral simulator means that it's not
simulating real circuit level details of a microprocessor it's just simulating the
behavior of each instruction as it runs. Nevertheless, behavioral simulators are
vital tools for computer architects. We use them to check the designs we've built,
match with the behavior we intended and we can also use them to explore new ideas,
for example adding new instructions to the instruction set. We're just going to be
using it though to get familiar with the basics of Arm AArch64. So we can create a
new file which will allow us to type in some Arm AArch64 instructions and when we
press assemble that will cause ASim to begin the simulation of the instructions. If
we scroll down we can see that below we've put a little cheat sheet of Arm AArch64
instructions that you can use as a reference. Do note that these are just examples
though. So for example this first one is mov X1 X2 and the descriptive text says that
this copies the value of register X2 to register X1. But equally that would imply
that if we use mov X3, X4 instead that would copy the value of X4 to the register X3.
So we don't have to just use these exact instructions, we can tweak them as we need
to, to achieve our goals. So going back to the ASim, let's say that we wanted to add
the number two to the number three. In our cheat sheet we could see that there is an
'add' instruction but it requires that the inputs for the addition are stored in
registers. So first of all, we're going to need to load up two registers with the
numbers that we want to add in this case two and three. So looking at our cheat sheet
again we can see that there's an instruction mov that allows us to do that. So if I
write mov X2, #3 this would cause the register X2 to be loaded with the value 3 and
similarly I could write mov X1,#2 and this would cause the register X2 X1 to be
loaded with the value two. Last but not least, we could then do the addition we
wanted to do, such as add X3 X1 X2. This will cause X1 to be added to X2 and the
results stored in X3. If I press assemble now, we will see that the UI changes
slightly. Here we can now see the simulated state of the processor. ASim is only
going to, because it's a behavioral simulator it's only going to show the state
before and after each instruction. And so right now we are before executing the first
instruction. That's the yellow highlighting there and we can also see it in the
simulated memory. If I press step we can now see that that instruction has completed,
and it's before executing the mov X1 X2. And notably we can see that X2 has been
loaded with the number 3 which is exactly what we would expect. If I press step
again, we can see that mov X1 X2 has now occurred. MOV X1 #2, sorry has now occurred
which has loaded the value 2 into the register X one. And last but not least, if we
step again we see that X3 has become equal to five which is exactly what we would
expect if registry X1 was added to registry X2. So this allows us to get a feel for
what it's like for a machine to execute these instructions. We can reset back to the
beginning if we want to watch it go through again. If we want to do an edit for
example adding a new instruction we can just do that. Press assemble and that will
add the new instruction to our program, and we can see its effects by simulating. Now
in the exercise you will be invited to take an existing program and add 1 new
instruction to it at the indicated position. Do note that when you assemble the
exercise program, you'll be taken to a gray bit of code, which is our testing code.
But as you step, you'll see that the simulator flicks between the testing code and
the code that we wrote the exercise one code. You can also click between them using
this. Feel free to try to work out what our testing code does but you can just ignore
it if you want to. The point of the exercise is just to add the instruction in the
indicated position. When you're comfortable you've got the right instruction you can
press run to get all the way to the end. And if you really think that's correct if
you scroll down to the bottom of the page you'll see the submit button which you can
use to send in your answer. Best of luck.

Module 2, Intro
[music] Hello, my name is Martin Weidmann. I'm an engineer and product manager with
Arm's Architecture and Technology Group. I look after the A-profile architecture and
I maintain Arm's Interrupt Controller specifications. Computer architecture is
sometimes called a science of trade-offs. Why is everything a trade-off when it comes
to designing a processor? Let's take an example of where we've had to make trade-offs
when developing process architecture. So the Arm architecture has a feature called
Pointer Authentication. This is often abbreviated to PAC for Pointer Authentication
Code. What this feature is trying to do is protect against a form of attack called
ROP and JOP. These are Return Orientated and Jump Orientated Programming, and it's
where an attacker tries to subvert things like the call stack to run legitimate code,
but in ways that weren't expected by the programmer or the compiler. PAC or Pointer
Authentication tries to defend against this kind of attack by using part of an
address to provide an encrypted signature. So we can check the signature and the
address match and if they don't, we can spot an attack in progress. So why is this a
trade-off? Well, because to add security, we want that signature to be as big as
possible. The bigger the signature, the more bits we use for that, the stronger
cryptographically that signature is. The trade-off is: the more bits we use for the
signature, the fewer bits we have available for other things, such as the address. So
you can have a big signature with a small address, but if you want the address to get
bigger, then you get a smaller signature, and that's then cryptographically weaker.
So the trade-off we have to make when designing a technology like that is: What's the
right amount of bits for the signature? What's the strength of cryptography we need
from that signature in order to get the design goal, which is to defeat these attacks
and give us more robust computing? What sort of guiding principles do you use when
designing a processor? So when you're designing a processor, the key thing you have
to bear in mind is: What's it going to be used for? What you don't want to end up
with is a very expensive paperweight. So we need to understand the requirements that
the processor has to meet. We have to understand the design trade-offs we're making
and how they work into meeting those requirements. We also have to consider not just
the processor itself, but how we're going to show that that processor is correct.
We'll put as much time, if not more, into testing and validating the design as we do
into designing it. How do you design a new microprocessor? If you wanted to create a
new processor from scratch, the first thing you're going to have to do is understand
the market that that processor is going to address and to then build a team to design
that processor. There isn't such a thing as one processor for every possible market.
The requirements for something like an embedded microcontroller are going to be very
different to what you want from the processor in your mobile phone, your laptop, or
your server. So you need to understand those requirements as the first step into
building a new processor. What determines the best design for a microprocessor? So
when you're designing a processor, you need to work out what the best design for a
given market or application is going to be. There's no magic formula for this. It's
going to depend a lot on what you're trying to achieve with that processor. You need
to understand things like the power requirements, the performance requirements. Is it
going to work in a highly noisy electrical environment? There's a big difference
between the reliability requirements you need from something like a watch versus a
satellite. So you would take those requirements and you'd work out what the best set
of trade-offs is going to be, and that's an art more than it is a science. How do the
underlying technologies contribute to this best design? A lot of technologies go into
our processor. There's the design of the microarchitecture, the implementation of the
processor. There's the silicon process you're going to use, how you're going to
integrate that processor into an SoC or ASIC. Is it a single die, or is it going to
be using chiplets or multiple sockets? All of those different technologies are going
to be factors into how you design the process or what trade-offs you make, and what
performance and power you get out of the design once you're done. In reality there
may be many different 'best' designs, so how do you pick one? So when you're
designing a processor, what you want is the best design. But often there isn't "a"
best design, there's just different trade-offs. You have to decide what the best set
of trade-offs is for the particular use case you're going for. And that's also going
to depend on: Is this a device that will be off the shelf, used for lots of different
applications—a general purpose processor? Or is this being designed for one specific
use case? Again, there isn't really a magic bullet or single answer for this type of
question. You need to understand how the processor is going to be used and then use
your experience to judge the trade-offs, and what will give you the best mix of
power, performance, area, cost, reliability for your target use case.

Module 2, Video 1
[music] In this module, we're going to explore how to improve the simple
microprocessor design from Module 1 in order to allow it to execute programs more
efficiently. First, let's find out how long a program takes to execute. The time
taken to perform the average instruction is equal to the number of clock cycles taken
to perform an instruction multiplied by the duration of one clock cycle. The time
taken to run our program is found by multiplying the average time to perform an
instruction by the number of instructions in our program. How could we make this
faster? One thing we could try is to reduce the number of instructions in a program.
We might be able to optimize the code removing unnecessary and repeated work and
selecting instructions to minimize code size and maximize performance. We could give
our microprocessor the ability to perform more operations in order to help
programmers or compilers further reduce the number of instructions in their program.
For example, allowing the loading of two data values at the same time might allow
fewer instructions to be used in the program. The downside to this approach, is that
adding more instructions will require extra circuitry in the processor and therefore
we likely increase the clock period. If the extra instructions are rarely used this
could even mean an overall decrease in performance. We see this theme often in
computer architecture trade-offs that we have to carefully balance. Another approach
is to use faster transistors perhaps constructed from a more recent fabrication
technology. This would reduce the clock period but may increase costs. The rest of
this module focuses on an optimization to reduce the clock period called pipelining.
This is the most important optimization we use when designing processors. It uses a
similar concept to an assembly line in a factory where work can start on the next
item before the previous one finishes. Let's take a closer look. Imagine that each
instruction has to go through four circuits in a processor. If we attempt to do all
of these in one clock cycle this means our clock period is the latency of all four
circuits added together. If we were to pipeline this, we would add a pipeline
register in the middle. This divides the circuit into two sections called stages.
Notice that although each instruction takes a similar amount of time to travel down
the whole pipeline, the pipeline design can execute nearly twice as many instructions
per second. The throughput has doubled. This is because we can set the clock period
much shorter. It's now the maximum latency of the two stages. We can pipeline into
many stages and this allows for much faster execution of programs. Unfortunately,
though, pipelining a real microprocessor design is not quite as simple because the
processor has various feedback signals and loops in the circuit. In the next video,
we'll take a look at the challenges of pipelining in practice. [music]

Module 2, Video 2
[music] In this video, we're going to look at applying the pipeline optimization to a
realistic microprocessor design. In the first module, we met the components of a
microprocessor, so let's look at how these are really connected. This diagram shows
all the connections needed for a real, unpipelined microprocessor. Each clock cycle,
the processor starts by fetching an instruction from the instruction memory. Once the
instruction reaches the decode logic, it is decoded to produce the control signals
necessary to execute it. The exact control signals vary depending on the type of
instruction. For example, arithmetic instructions access the register file and
interact with the ALU. Ultimately, no matter how the instruction was executed, the
last step of each clock cycle is to update the program counter. This is done by the
branch unit. For non-branch instructions, this just means incrementing the program
counter. However, for branch instructions, the branch unit has to do some
calculations. When we apply our pipeline optimization to this design, we face some
challenges. The design has several loops because instructions have dependencies. How
can we break these cycles? The key observation is that not every instruction is the
same. In real programs, branch instructions usually make up less than 20 percent of
the program. For non-branches, the branch unit doesn't actually need to wait for the
ALU before calculating the result. Let's look at how we can use this fact to pipeline
the processor. Once the first instruction shown in yellow reaches the pipeline
register, we're ready to begin fetching the next instruction, shown in blue. The
yellow instruction can be in the execute stage whilst the blue instruction is being
fetched. Once the yellow instruction is finished, the blue instruction is ready to
enter the execute stage and a new green instruction enters the fetch stage. What
about the branches though? Let's imagine this next yellow instruction is a branch.
The fetch stage works normally until the branch unit, but the branch unit cannot
proceed. Consequently, the pipeline stalls. The fetch stage spends a cycle waiting
whilst the execute stage executes the branch. Finally, once the ALU is done, the
branch unit can proceed and the next instruction, in this case blue, can be fetched.
Overall, this means that the processor wasted one cycle stalling due to the branch.
Since only 20 percent of instructions are branches, this means that each instruction
would require on average 1.2 cycles. The same idea of stalling the pipeline can be
used to create even longer pipeline designs. This diagram shows a typical five-stage
processor pipeline. In the next video, we'll look at how we can manage or prevent
some of the stalls in a design like this. [music]

Module 2, Video 3
[music] Instructions within a program may be dependent on each other. That is, one
instruction may produce a value that a subsequent instruction consumes. Data values
may be communicated through registers or memory. The simple program shown has a
number of so-called true data dependencies. This means we must take care to execute
these instructions in order, and make sure results are correctly communicated.
Additionally, the outcomes of branch instructions may affect the path taken through
the program, and consequently, this affects whether an instruction is actually
executed. This sort of dependency is known as a control dependency. In the previous
video, we met a realistic processor pipeline with five stages. Circumstances that
prevent an instruction making progress in our pipeline are known as pipeline hazards.
Let's take a look at how dependencies cause hazards. This program has a true data
dependency. The first instruction writes to register one, which is then read by the
second instruction. If we send this down our pipeline, we see that the second
instruction must stall, waiting for register one to be written, before it can read
and proceed. This is a data hazard. Unfortunately, dependent instructions are common
and stalling in this way would significantly increase the average cycles per
instruction. Let's take a closer look at the hazard though. The ADD instruction is in
the execute stage, meaning its result is being computed. The SUB instruction needs
that result to proceed. Rather than waiting for the ADD to reach the writeback stage,
we could add an extra path into our pipeline to carry the output of one stage to a
later instruction, making the result available straight away. We call this a
forwarding path. In this case, the ALU result is forwarded to the SUB instruction to
be used as X1. This small piece of extra circuitry allows this data hazard to be
eliminated completely. Unfortunately, even if we add forwarding paths everywhere,
it's not possible to eliminate all data hazards. For example, this program has a data
hazard due to the load instruction. There are other types of hazard too. This program
contains a control hazard. We cannot be sure which instruction to fetch until after
the branch instruction executes. Consequently, this program has two stall cycles. We
will look in more detail at how control hazards can be mitigated in the next module.
Another class of hazards, called "structural hazards" occur when two instructions
require the same resources simultaneously. For example, if instructions and data were
stored in the same memory, and this could only be accessed once per cycle, we would
have to very frequently stall our pipeline to let these stages access memory one-by-
one. [music]

Module 2, Video 4
[music] In the previous videos, we explored how pipelining could improve performance
by reducing our clock period and by overlapping the execution of different
instructions. We also saw that it was sometimes necessary to stall our pipeline to
ensure that instructions were executed correctly. Ideally, our average cycles per
instruction, or CPI, will remain at 1.0. If we must stall, however, this will
increase. For example, if 20 percent of our instructions were loads and each of these
caused one stall cycle, our CPI would be 1.2. If a further 20 percent of instructions
were branches, and each of these caused two stall cycles, our CPI would be 1.6. The
longer we make our pipeline, the more stall cycles there will be, and eventually the
cost of stalls may outweigh the benefit of the faster clock period. For example,
let's imagine we added a stage to our five-stage pipeline from the previous video.
Now the number of stalls after a branch instruction increases to three, hurting our
CPI. On the other hand, our clock period would improve. So whether or not this helps
speed program execution would depend on the exact details. It may eventually become
more difficult to reduce our clock period by adding further pipelining stages. This
is because it becomes harder to perfectly balance the logic between stages and
because of the constant delays associated with clocking and our pipeline registers.
To mitigate these issues, we will need to invest in more transistors and our design
will require more area and power. The deeper our pipeline gets, the greater the
investment we need to make in terms of area and power for the same incremental
improvement. Commercial processes today have anywhere from two to twenty pipeline
stages. The faster, more expensive and power-hungry processors tend to have longer
pipelines than the smaller, cheaper processes in embedded devices. As with many
techniques in computer architecture, eventually it becomes more profitable to invest
our time and resources in an alternative way of improving performance. In later
modules, we'll explore how we can reduce the CPI, even in heavily pipelined
processes. [music]

Module 2, Lab
[music] In this exercise, we're going to be using a model of a processor pipeline to
explore the effect of the pipelining optimization. Computer architects use models
like this to make high level decisions early on about what parameters they will use
for a processor and using a model such as this saves the burden of actually building
the processor to find out its performance. The model is not simulating accurately the
performance of the processor but rather it's giving us an idea for what performance
we might expect. So what can we do with this model? Well, we can configure the number
of pipeline stages, which we can see affects the diagram. And we can also turn on or
off the forwarding optimization. As we change these numbers notice that the design
parameters change down here. So for example, the clock frequency is improved by
increasing the number of pipeline stages but the design area will get bigger. And so
this may be a consideration depending on the problem. We can also choose which of two
programs we're going to put down our pipeline. When we press the step forward button
the pipeline advances to the next clock cycle and we can see the instructions have
started to flow down our pipeline and interesting events such as forwarding will be
noted in the simulation. Sometimes the simulation will detect that there will be a
stall for example, in this case, we can see that there is a data hazard because the
instruction in the red memory stage writes to register X13 which is read by the
instruction in the yellow decode stage and therefore a stall cycle is necessary in
order to allow the result to be generated. If we press the play button, the
simulation will proceed automatically and we can see various stall events happening
as the simulation proceeds. But notice that the the program we're simulating is
nearly 1,000,000 cycles long so watching it play out at this speed is going to take
quite a while. So we can use the fast forward slider to simulate much, much faster.
Notice that the statistics down the bottom have updated depending on the results of
the simulation, and at this point we can see that the program is finished and the
simulation of the program, the simulated program took 3.98 milliseconds to execute.
We can also see that just below, the results of past simulations are stored in little
tables so we can easily refer back to them when we're doing later experiments. So as
an experiment, let's imagine what would happen if we disabled the forwarding
optimization but change nothing else and we'll just run this program through. What we
can see immediately is the design side is slightly better which is what we would
expect. It's got 1% better in fact in this case because of the lack of the forwarding
wires. But now that the program is finished, we can see that the program execution
time is a lot worse. 6.34 milliseconds is about 50% worse. So again, looking in our
table, we can compare the execution times in the area and we can see that in most
cases the forwarding optimization would be a big optimization here because at the
cost of an increase in the area of about 1%, we've had a improvement in execution
time of about 50% which is likely to be a good trade off, but not always. It would
depend on the exact scenario. Perhaps that 1% area is more important than the
performance of this program. In the exercise, you'll be invited to design a or
suggest a design using the number of pipeline stages and whether forwarding is
enabled that will meet certain criteria. You can play about and do as many
simulations as you wish to figure out what the best program might be. Once you've got
it set up select the processor that you're happy with at the top and then scroll down
to the submit button and press that. Good luck. [music]

Module 3, Intro
[music] Hi, I'm Nigel Stevens. I'm Lead Instruction Set Architect for the Arm A-
Profile architecture. I've been at Arm for about 14 years and I have responsibility
for the Arm V8-A instruction set including recent developments such as the Scalable
Vector Extension and Scalable Matrix Extension. What do we mean by the instruction
set architecture? The instructions in architecture, primarily, most people think of I
guess as the OP codes, the encodings of instructions that are executed by an Arm-
based processor. But it also includes other aspects as well such as the exception
model, system programming features, memory management and suchlike. The architecture
for Arm is rather distinct from what other companies may call an architecture. For
Arm, architecture is a legal contract, if you will, between hardware and software. If
software uses only those instructional codes and features of the ISA that are
described by the Arm architecture to perform its work, and the hardware that it's
running on implements all of those op codes and features exactly as defined by the
architecture, then any architecturally compliant software will run on any
architecturally compliant hardware that implements that Arm architecture. And that
doesn't mean just processors from Arm itself, but also processors that are designed
by our partners and which we have validated are conformant with our architecture. How
do you decide which instructions to include in an ISA? When we are looking at
requests from partners or from internal research to add a new instruction, we go
through quite a long process of trying to justify that instruction, or, quite
commonly, a set of instructions rather than a single instruction. We have to show
that it gives us some real benefit in performance, the performance of your code
running on that CPU. Or maybe not performance. Maybe it's security you're trying to
achieve. But it has to give you some really concrete benefit that is worth the cost
of adding all of the software, the validation software, the implementation costs for
all of the different implementations, compiler support, so on and so forth. It has to
meet the... It has to answer that cost benefit analysis. What is the difference
between an ISA and a microarchitecture? The difference between an instruction set
architecture, or ISA, and the microarchitecture is that the ISA is an abstract
concept. It defines a set of instruction encodings which software can use, and which
hardware has to recognize and implement. How that is implemented is a choice for the
microarchitecture. So the instruction set architecture is fixed, it's defined by Arm.
The microarchitecture is defined by whatever team of people is designing that CPU.
And there are many different approaches to implementing the Arm architecture, from
very small, efficient cores with in-order pipelines up to very high-performance,
state-of-the-art, out-of-order execution, and everywhere in between. So the
microarchitecture is implementation-specific, the architecture is generic, and
software written for the architecture should run on any microarchitecture. Why does
Arm produce processors with different instruction sets? Arm supports multiple
instruction sets. Some of that is to do with legacy: you can't abandon your legacy
software, your legacy ecosystem. So as the architecture has advanced and we've
introduced major new instruction sets, we still have to continue to support old
software. It takes years, maybe 10 years to move the software ecosystem to a major
new ISA. So for example, AArch64, which is the 64-bit architecture that we introduced
with Arm V8, also supported the AArch, what we called AArch32, the old 32-bit
architecture that was implemented in the Arm V7 architecture, and prior to that
including the Arm and the Thumb instruction sets. And we needed to do that because,
while some software might start to migrate to the 64-bit architecture, there's still
a lot of software on the planet which is going to continue using the 32-bit
architecture, and that has to survive. So that's part of the reason: it's about
legacy. You can't just obsolete the whole world when you introduce a new
architecture, a new instruction set architecture in particular. There are other
reasons as well, which is there are certain instruction sets that are different for
reasons of the ecosystem that they're working with. So if you were to compare, for
example, the A-profile architecture that's designed for application processors that
run rich operating systems with virtual memory supporting SMP, symmetric multi
processing operating systems running large applications, whatever it may be: web
browsers on your phone or something, or a web server in a server farm somewhere. You
have your R-profile architecture, which is designed for high-performance, real-time
embedded systems. The constraints there are somewhat different. The instruction set
is actually fairly similar to the A-profile, but some of the underpinnings of the
architecture, the system side of the architecture, are simplified in order to give
more consistent and predictable real-time response to things like interrupts or
memory translation and suchlike for real-time systems. And then at the other extreme
you have the M-profile architecture which is designed to be capable of being built in
a very simple, ultra-low power implementation with low code size and again, similar
to the R profile, very predictable real-time performance. So the answer is there are
different instruction sets for reasons of the market that they're trying to address,
and then there are different instruction sets because, well, we have history. [music]

Module 3, Video 1
[music] In the previous module, we explored how pipelining can be used to improve
performance. We also saw how it is sometimes necessary to stall our pipeline to
ensure our program is executed correctly. In a simple pipeline, it will be necessary
to stall the pipeline whenever we encounter a branch instruction. This is because we
must wait until our branch is executed before we can be sure which instruction to
fetch next. As a recap, branches are instructions that change which instruction in
the program will be executed next. There are two types of branches: conditional
branches and unconditional branches. Unconditional branches always change which
instruction executes next, whereas conditional ones may or may not, depending on the
computations in the program. In real programs, between approximately one fifth and
one quarter of all instructions are branches, and the majority of these are
conditional. Executing a branch involves calculating the new address to load into our
program counter. This is the branch's "target address." However, conditional branches
have an extra task: we must first determine whether the branch is taken. If the
branch is not taken, we can effectively ignore the branch and fetch the next
instruction as normal. Recall the processor performance equation from an earlier
video. Since we have to wait for branches to complete before fetching the next
instruction, we generate stall cycles. These increase the average number of "cycles
per instruction," which reduces our microprocessor's performance. The longer our
pipeline gets, the longer it is before each branch is resolved, and the more costly
branches become. Can you think of a way to avoid some of this stalling? One idea is
to evaluate branches earlier in the pipeline, for example in the Decode stage instead
of in the Execute stage. This can indeed help to reduce the number of stalls, but we
may still need to stall if the branch depends on other instructions that haven't been
executed yet. Another idea is to continue fetching instructions in program order,
effectively assuming that each branch is not taken. The number of stalls in the
pipeline for a not-taken branch is zero in this design. On the other hand, if the
branch is in fact taken, the subsequent instructions that we fetched will be
incorrect. So, we must remove all instructions that have been fetched on this
incorrect path from our pipeline. This is called "flushing" the pipeline.
Unfortunately, in real programs, branches are taken much more than not taken. Could
we then simply assume instead that all branches will be taken? Sadly not, no, because
then we would also need to know the specific "target address" immediately, in order
to know which instruction to fetch next. It may at first seem impossible to know this
before the instruction is decoded. However, computer architects have found a way to
do exactly this. The next video will look at "dynamic branch prediction:" the idea of
predicting the behavior of the branch instruction before it has even been fetched.
[music]

Module 3, Video 2
[music] In this video, we'll explore how to predict the behavior of a branch
instruction. This can sometimes eliminate the cost of branches altogether.
Fundamentally, a branch involves changing the value of the program counter, which is
the address of the next instruction in the program. If we could predict what this
change will be, quickly and accurately, we would have no need to stall. Precisely, we
need to predict that we are fetching a branch, predict it as taken or not taken, and
predict what its target address is. How could we ever make such predictions? What
would we base them on? Well, since instructions are often executed multiple times, we
can accurately make these predictions based on just the instruction address. If we've
previously seen that a particular address contains a branch, and we see that address
again, we can predict whether that branch will be taken, and its target address,
based on its behavior last time. Amazingly, for real programs, simply predicting
repeating behavior is typically around 90 percent accurate. This means we could
eliminate stalls caused by branch instructions 90 percent of the time. Let's apply
these insights to try to build a branch predictor. We will add two extra blocks to
the Fetch stage of the pipeline we met in Module 2. The first will remember
information about recently executed branches. This will include the program counter
values of branches and their target addresses. This memory is called the "Branch
Target Buffer" or BTB. The second block will make predictions about whether an
address containing a branch is taken or not. We simply call this the "branch
predictor." In the next video, we will look at these in more detail. Combining these
two gives us all the information we need to predict the next value of the program
counter based solely on the current value of the program counter. We don't even need
to decode the instruction to make this prediction. Here we can see how a running
branch predictor behaves for a sample program. Each program counter is checked in the
BTB to see if it's predicted to be a branch and to identify its predicted target. We
also simultaneously check the branch predictor to see if the branch is predicted to
be taken. Based on these predictions, the branch unit computes the predicted next
program counter. Many cycles after the prediction, feedback will be given by the rest
of the pipeline as to whether or not the prediction was correct. Whenever the
prediction is wrong, we have to flush the pipeline and update the BTB and branch
predictor. The pipeline will then resume fetching from the correct program counter as
computed by the pipeline. The majority of instructions are not branches, so most of
the time the branch unit just adds 1 to the program counter. The BTB contains the
instruction address and target address of some recently executed branches. The exact
number of entries in the BTB varies considerably. BTBs in large modern processors
contain many thousands of branches. In operation, the BTB checks the supply program
counter against its memory to see whether it has a match, and if so, it returns the
target address. Otherwise, it predicts that the instruction is not a branch. After
each branch executes, the BTB is updated with the true target address. BTB cannot be
arbitrarily large, so it may have to forget an existing branch to remember a new one.
A simple BTB design like this is typically around 90 percent accurate at predicting
target addresses in real programs. [music]

Module 3, Video 3
[music] In the previous video, we met the two major components of dynamic branch
prediction, the BTB and the branch predictor. In this video, we'll take a deeper look
at the branch predictor. It predicts whether or not a branch will be taken, based on
the program counter. A simple branch predictor would try to remember what the branch
did last time and predict that the same behavior will repeat. Let's see how such a
predictor might be organized. Remembering a branch prediction for every possible
instruction address would take up far too much memory, so we reuse the same memory
for different branches via a process called "hashing." We hash the address of the
branch to a smaller number. This does unfortunately lead to a problem called
"aliasing," where two different branches can hash to the same value, but this is rare
in practice. Let's see what happens now, when we execute a simple loop. We see a
misprediction when it encounters our branches for the first time, and another when we
exit the loop. The first case will be dependent on the value in our predictor's
memory, and it may be that we are able to predict the branch correctly the first time
we see it. The second case is hard to avoid, although some more sophisticated branch
predictors will learn how many iterations a loop will make. A common improvement to
this simple scheme is to avoid instantly flipping our prediction just because the
branch does something unexpected once. This can be achieved with a saturating
counter, which instead remembers how many times the branch has been taken recently,
versus not taken. The counter increments when the branch is taken and decrements when
not taken. It predicts "taken" if the majority of recent executions of this branch
were taken. When building higher-performance processors, we often have to discard
many instructions every time we mispredict a branch, so accurate branch prediction is
very important. Therefore, branch prediction is still an active area of research. One
of the key ideas used is correlation between branches. In real programs, a common
pattern is for example a pair of branches that always have opposite behavior. A
branch predictor can take advantage of this by remembering a history of whether or
not recent branches were taken or not taken and incorporating this in the hash.
Another idea is to combine multiple different types of predictor in a "tournament
predictor." We use a "meta predictor" to predict which of two branch predictors will
do a better job. As you can see, branch prediction can get quite complicated. [music]

Module 3, Video 4
[music] In this module, we've met the concept of branch prediction. Even using
relatively small simple circuits, we can accurately predict real branch behavior more
than 95 percent of the time. Could we do better, and do we really need to? In small,
simple processors, these prediction accuracies are fine, because each misprediction
causes only a few stall cycles. Increasing the accuracy is not that impactful.
However, in complex processors with very long pipelines, the difference between 98
percent and 99 percent prediction accuracy can be significant for performance as a
misprediction could incur dozens of stall cycles. Accurate prediction really does
matter. One of the problems we face in making accurate predictors is that they need
to be small enough and fast enough to fit in a microprocessor. We can imagine all
sorts of ways to do accurate branch prediction, but if their circuit would be slower
than simply executing the branch, they would not be useful. Modern high-performance
processors will often have multiple branch predictors, for example a small fast one
and a slower complex one. The slower one can override the prediction of the fast one
if it thinks it got it wrong, which does incur some stall cycles, but fewer than a
total misprediction. Another problem we face is that some branches are just very hard
to predict. No matter what technique is used, there are some branches in real
programs that are effectively "random". For example, when compressing or
decompressing data, the combination of the underlying algorithm and the input data
may provide no clear patterns to a prediction. No matter how hard we try, some
branches will never be predicted correctly 100 percent of the time. A final problem
is that since the predictors work based on observing the program, there will always
be a period of time when the predictors train on a program to learn its behavior.
[music]

Module 3, Lab
[music] In this exercise, we're going to be using a branch predictor simulator to
explore dynamic branch prediction. This simulator will accurately simulate the
details of a branch predictor but it uses a trace of a real program executing on a
real machine to avoid the need to simulate the rest of the processor. Computer
architects use techniques such as this to quickly explore one area of process design
understanding that perhaps the accuracy may not be completely correct given that
we're not simulating the full process of design. The interface allows us to configure
details of our branch predictor for example the maximum of the saturating counters
used in the branch predictors table or the hash function. And we can see the impact
of changes on the delay of the branch predictor and also the design size which are
two key metrics when designing a branch predictor. Once we've happily configured the
design we want we can press run to simulate a program and the results of that
program's execution will be displayed in the rest of the stats. So for example, we
can see here that the predictor predicted 95.24% of the branches correctly. Which is
fairly good and this resulted in overall execution time of the program of 5.335
milliseconds. Just below we can see a table that records previous simulation runs as
well so we can do multiple experiments and see which produces the best results. For
example, if we were curious about the effects of using a saturating counter with a
maximum of three rather than one we could change that and notice that the design size
has substantially increased and when we press run, we noticed that the predictor
accuracy, however, has also increased. It's gone up to 96.31% and consequently the
execution time the program has fallen slightly and so we can compare these two
designs and see whether or not this represents a good trade-off for our processor.
Perhaps the area cost is justified or perhaps it's too much. It would depend on the
exact processor we're trying to design In the problems you'll be invited in to come
up with designs that are suitable for particular constraints. For example, particular
constraints on the runtime of the program or the design size. Once you've configured
the branch predictor that you think meets the objectives you can scroll down to the
submit button and click that and you'll be told whether the answer is great or not.
Good luck. [music]

Module 4, Video 1
[music] So far, we've looked at the microprocessor's "datapath"— meaning its
execution units, registers, and control circuitry. We have given less attention to
its memory. We usually implement memory using a different type of chip than the
microprocessor, using a technology called DRAM. It is very dense, allowing us to
store lots of data in a small area. However, one issue with DRAM is its speed. Since
the 1980s, processor performance has increased very rapidly at roughly 55 percent per
year, so CPUs of today are many orders of magnitude faster than those of 40 years
ago. In contrast, memory performance has grown much more modestly. Whilst memories
are also much faster than they were in previous decades, their performance has not
kept pace with processors. This leads to a processor-memory performance gap, with it
becoming more costly to access memory as time goes on. One of the issues that makes
these memories slow is their size. We can make a memory as fast as our
microprocessor, if it is very small. However, this is the opposite of what
programmers want. Programmers can use extra memory to solve more complex problems, or
to solve existing problems faster. This complicates microprocessor design because
although we want a large memory to hold all our data, large memories are slow and
this slows down our processor. How do we overcome this? What if we could get the
speed benefit of a small memory alongside the size benefit of a large memory? One
solution is to have a large, slow memory and a small, fast memory in our system. For
this to be useful, we need the small memory to hold the data that we use most often,
so that we often get the speed benefits of accessing it, and only rarely have to
access the slow memory. Whilst there are different arrangements of these two
memories, the configuration that is most often used is an "on-chip cache memory." The
small memory sits inside the processor between the pipeline and the large memory. We
keep all data in the large main memory, and put copies of often-used data in the
small memory, which we call a cache. But we are not limited to only one cache! Our
pipeline reads memory in two places: when fetching the instructions; and when
accessing the data. It makes sense to have two caches here, each optimized for
different purposes. The instruction cache is optimized for fast reading of
instructions at Fetch. The data cache is optimized for reading and writing data from
the memory stage. We will often put a larger "level 2" cache between these two caches
and the main memory. The L2 cache is a "medium-sized" memory: faster and smaller than
main memory, but slower and larger than the two L1 caches. Using a hierarchy of
caches reduces the bandwidth requirements to main memory and the energy cost of
moving data around. [music]

Module 4, Video 2
[music] We previously looked at the need for a cache, which can be used to store
often-used data for fast access by the processor. But which data is used often enough
that it is worth including in the cache? Programs are mostly made of loops. Here's a
simple one that sums values in memory. It displays the two characteristics that we
can exploit to decide what data to put into a cache. Let's look briefly at how it
works. Each time round the loop, there are two loads and one store. The first load is
to load the data we're summing. The other load, and the store, are to update the
running sum. Notice two things here. First, we access the running sum over and over
again; each time round the loop. Second, when we access part of the data in one loop
iteration, we've already accessed its predecessor in the previous iteration, and will
access its successor in the next. Caches exploit two types of "locality" in programs
in order to be effective. The first is temporal locality: if a piece of data is
accessed, it is likely that it will be accessed again in the near future. The running
sum has temporal locality. The second type of locality is spatial locality: If a
piece of data is accessed, then its close neighbors are quite likely to be accessed
in the near future. By close neighbors, we mean data whose memory addresses are not
far from each other. The data accesses have spatial locality. It turns out that most
programs exhibit lots of temporal and spatial locality. We can exploit this to
determine what to put in the cache. Exploiting temporal locality is fairly easy: we
simply see which values have been accessed and place them in the cache. Exploiting
spatial locality is also quite simple: when a piece of data is accessed, place its
neighbors in the cache. We'll see in the next module how this is actually achieved.
We use locality to guess what values will be accessed next and store them in the
cache. If we are correct, we get the benefit of fast memory, but if we are wrong, we
must perform a slow access to main memory— just as we would if the cache were not
present. In real programs, we see hit rates of 90 percent or more in microprocessor
caches, resulting in vast performance improvements. However, it's worth noting that
some programs have less locality, and for those programs, caches offer little
performance benefit. [music]

Module 4, Video 3
[music] We've looked at the reasons why we build caches, but how do they actually
work? To the outside world, a cache simply takes an address as input, and either
provides the data that is stored at that location at output, or returns a signal to
say that it doesn't have it. If the data is found, this is called a "cache hit". If
the data is not in the cache, this is called a "cache miss", and it means we must
look for the data in main memory instead. After each miss, we update the contents of
the cache. The fundamental building block of a cache is called the "cache line". It's
a number of data bytes from consecutive addresses in main memory. Cache lines are
typically 32 or 64 bytes long, and a cache typically has an array of hundreds to many
thousands of lines. The line captures spatial locality, because it is larger than the
data read by a single load instruction. When a cache miss occurs, the whole line
containing that data is copied to the cache from main memory, meaning we have nearby
values for future accesses. When a request comes in, we use some bits from that
address to index the line array. Just like in the last module on branch prediction,
this leads to the problem of aliasing again, since selecting only some of the bits to
index into the array is like a hash. This means that many addresses map to the same
line in the cache, but we can only store one of their lines of data. We need to note
down which addresses' line is currently stored in the data array, in what we call the
"tag array". There is one tag for each cache line. When we access the cache, we
access the tag array with the same index to see if the data we need is present. This
design is called a "direct-mapped cache". Direct-mapped caches work fairly well, but
for some programs, we can be unlucky with the program accessing to aliasing lines
frequently. We can do something about this by duplicating both the arrays, so that
each line of data can now be stored in one of two places in the cache. When we access
the cache, we look at both arrays and only get a miss if neither of the tags match.
This is called a "2-way set-associative cache", because each line has a set of two
places or "ways" it could reside. The "associativity" of this cache is therefore two.
Set-associative caches introduce a further complication: when we want to add data,
where do we put it? In a 2-way set-associative cache, there are two choices. How we
decide which cache line to evict is called the "replacement policy". There are many
different types of replacement policy. A simple one is just to make a pseudo-random
choice from all the possible cache lines. Another option is to keep track of when
each cache line was last accessed and to evict the one last used furthest in the
past. This is called a "least recently used policy", and takes advantage of temporal
locality. It does, however, means storing extra information in the tags to track
usage. Many other ideas are possible too. [music]

Module 4, Video 4
[music] Now that we've seen how caches work, let's see how they affect the
performance of a processor. Recall the processor performance equation, where the
processing time is proportional to the average cycles per instruction. Without a data
cache, if 20 percent of instructions are loads, and main memory takes 20 cycles to
access, our CPI figure must be at least 5. However, if we provide a cache that holds
the required data 80 percent of the time... ...and only takes 2 cycles to access, our
CPI reduces to 2.2, which is a significant improvement! We can isolate the memory
terms in this equation to get the average memory access time —abbreviated to AMAT—
which allows us to compare different cache configurations more easily. Changing the
cache configuration will impact the AMAT. There are many different cache parameters
we can change, such as the size, replacement policy, associativity, whether we put
data in the cache for stores or just for loads, and so on. For example, reducing the
size of the cache will improve the access time for a hit, but will also increase the
miss rate. Let's say that we can halve the access time to 1 with a corresponding
halving of the hit rate. This alters the AMAT to 13, which in this case is worse for
performance overall. It's also useful to look at why an address might miss in the
cache. Broadly speaking, we can divide cache misses into three different categories.
Compulsory misses occur when we attempt to access an address that we have never seen
before and so never had the opportunity to cache it. Capacity misses occur when there
is more data being accessed than the cache could hold, even if we had complete
freedom in where to put each cache block. Conflict misses occur in caches where there
are more addresses hashing to the same index than arrays to hold the data. We can
alter our cache configurations to lower these misses, but as always, there are trade-
offs involved. Compulsory misses can be reduced by increasing the cache block size,
to take advantage of spatial locality. But for a fixed cache size, this reduces the
number of different addresses or cache lines that can be stored. A technique called
"pre-fetching" can also be used to predict the addresses that will soon be accessed,
and bring their data into the cache early. But this increases energy consumption, and
may make the cache perform worse if the predictions are not highly accurate. Capacity
misses can be reduced through increasing the size of the cache. Although, as we saw
before, this impacts the number of cycles taken to determine a hit. Conflict misses
can be reduced through increasing the number of cache blocks in each set, with an
increase in energy consumption as a side effect of this. [music]

Module 4, Lab
[music] In this exercise, we're going to be using a cache memory simulator to explore
the effects of cache memories on processor performance. The simulator accurately
simulates the behavior of the cache memory system, but it's using a model of the rest
of the processor to quickly allow us to simulate the effects of the cache on the
processor without needing to simulate the full processor. We can configure a number
of parameters about our cache, for example, the number of levels in our cache,
whether or not the cache separates instructions and data, or is unified, keeping them
both in the same cache. We can also configure the size, the line size, and the
associativity, and changing these numbers will affect design parameters, such as the
access times in the case of an L1 level one cache hit or a cache miss, and also the
design size. Once we're happy we've found design we'd like to investigate, we can
press "run", at which point the simulator will run through the program. We can see
the hit rates for instructions in the level one cache, and also data in the level one
cache are displayed. And we can also see the average memory access time, the results
from this. And then below everything, we can see a table of past simulations so that
we can quickly refer back to our previous experiments when we do new ones. So, for
example, let's say we were curious about the effects of increasing the size of the
cache. If we change the parameter and then press "run", we can immediately see that
the design size has substantially increased, which sort of makes sense because we've
doubled the contents of the cache, the size of the contents of the cache, and
therefore we'd expect roughly double the area. And we can also see that the hit rates
have improved. So there's about a 1% improvement to the L1 instruction cache hits and
a 1% improvement to the L1 data cache hits, which has reduced the overall average
memory access time. And so we can compare these two designs to see which of them we
think is better. It's a trade-off of course, though. The larger design has got better
performance, but it is larger, and so depending on the context, we may need to pick
the smaller design or the bigger design depending on our performance goals. In the
exercises, you'll be invited to come up with a series of designs for caches that meet
certain performance goals. For example, you'll have a constraint on the area of the
design or the execution time of the program, and you need to optimize the cache to
meet those goals. Best of luck. [music]

Module 5, Video 1
[music] In this module, we'll look at how to further improve performance by
exploiting "instruction-level parallelism." In Module 2, we explored how pipelining
can improve the performance of our processor. This reduced our clock period, and
allowed execution of instructions to be overlapped, improving throughput. One way to
boost performance further would be to create a much deeper pipeline. At some point,
this would mean even the ALU in our Execute stage will be pipelined. Consider the
simple program shown in the slide. Some instructions are dependent on a result from
the previous instruction. Remember in our 5-stage pipeline that these dependent
instructions could be executed in consecutive clock cycles with the aid of data
forwarding. If execution takes place over two pipeline stages within the pipeline, we
need to stall if adjacent instruction share a dependency. This allows time for the
result to be computed. The programmer may be able to rewrite their program to get the
same result with fewer stalls, by placing an independent instruction between our pair
of dependent instructions. In this case, we can move the third and fifth instructions
earlier to optimize performance. The performance of programs that run on our "super-
pipelined" processor would, to some degree, be determined by the availability of
independent instructions that could be executed in parallel in the pipeline. This is
"instruction-level parallelism"—or ILP. Very deep pipelines are problematic as they
would require: a very high-frequency clock to be distributed across the chip very
precisely; careful balancing of logic between many, very short, pipeline stages; the
pipelining of logic that is difficult to divide further into stages; and the division
of logic at points requiring many pipelining registers to be inserted. A different
approach to exploit ILP is to make our pipeline wider rather than deeper. In this
design, the processor will fetch, decode, and potentially execute multiple
instructions each cycle. Such a design avoids the problems of a super-pipelined
processor, although as we'll see in the next video, it does introduce some new
complications. [music]

Module 5, Video 2
[music] In this video, we are going to explore "superscalar" processors, which can
process multiple instructions in each pipeline stage. In our simple 5-stage pipeline,
there is at most one instruction per pipeline stage. At best, we can complete one
instruction per cycle. We call such a design a "scalar" processor. In a 2-way
superscalar version of this processor, we would extend this design so it is able to
fetch, decode, execute and writeback up to two instructions at a time. In general,
superscalar processors may vary the number of instructions that can be processed
together in each stage. Let's step through the design. Our instruction cache will
need to supply two instructions per cycle. Typical superscalar processors only ever
fetch adjacent instructions on a given cycle. This can lower performance if, for
example, the first instruction fetched is a taken branch, because then the second
would not be required. Note that now every cycle lost due to control hazards will
cost us two instructions rather than one, so accurate branch prediction matters even
more in superscalar designs. The Decode stage must now decode and read the registers
for two instructions simultaneously. Fortunately, we are able to extend the register
file design to read many register values at the same time. The Decode stage also
needs to check whether the two instructions are independent. If so, and if the
functional units they both need are available, it can "issue" them for execution in
parallel on the next clock cycle. Otherwise, in this simple design, it will only
issue the first, and keep the second back. A simple design such as this—where two
instructions are fetched, decoded and issued— is called a "2-way" or "dual-issue"
processor. In other designs, the width may vary at different stages of the pipeline.
To support the execution of multiple instructions at the same time, the Execute stage
is expanded and contains two execution pipelines. It's common for these to have
slightly different capabilities to save area. For example, the top pipeline can
execute both ALU and memory instructions, while the second pipeline only executes ALU
instructions. To ensure that dependent instructions can execute on consecutive clock
cycles, we must add data forwarding paths. These data forwarding paths must allow
results stored in either execution pipeline to be forwarded to the input of either
ALU. During writeback, we need to store both results to the register file. This means
the register file must be redesigned to allow two writes per clock cycle. Overall,
these changes typically require 25 percent more logic circuitry in our processor,
compared with a scalar processor. But we'd expect an improvement in execution time of
between 25 and 30 percent for real world programs. [music]

Module 5, Video 3
[music] We've seen that instruction-level parallelism can be used on superscalar
processors, to run them faster than would be possible on a scalar processor. But how
much of a speedup is this in practice? Ultimately, this depends on how much
instruction-level parallelism is possible in a typical program. How might we measure
this? We can do this initially without considering any constraints that will be
imposed by the processor it will run on. Let's consider the instructions executed by
the program. Let's assume that we can predict all the branches in the program
perfectly. Then we can ignore branch instructions, as they don't need to flow down
our pipeline. Now let's imagine we can execute any instruction as soon as the data it
needs is ready. That is, we are only restricted by the presence of true data
dependencies. Note that some dependencies are carried through writes and reads to
memory. Rather than considering program order, we can now just look at the order the
dependencies impose on instructions. This is referred to as "data-flow analysis."
Assuming each instruction takes exactly one cycle to execute, the fastest possible
execution time of the whole program in cycles is given by the longest path in the
data-flow graph. The instruction-level parallelism of this program is the number of
instructions divided by this duration, as this gives the average number of
instructions we would need to be able to execute each cycle to achieve this duration.
In real programs, this can be anywhere from around five, to hundreds or even
thousands. An active area of research and innovation for computer architects is to
imagine processor designs that can expose and exploit as much of this parallelism as
possible. One insight architects have had is that superscaler processors need to have
a fast supply of instructions to be able to analyze dependencies effectively. This
often means that the front end of our processor pipeline is much wider than the rest
of the pipeline, so that it can "run ahead" and see what behavior the program will
have next. Fast and accurate branch prediction is vital, as we often have to predict
multiple branches ahead accurately, to achieve good performance. Another key insight
is that we don't have to wait to execute the instructions in program order. If all
the dependencies of an instruction are satisfied, the instruction can proceed down
the pipeline even if previous instructions are yet to execute. This can reduce
program execution time by taking advantage of more instruction-level parallelism. In
practice though, this creates extra complications, as we will see in the next module.
[music]

Module 5, Lab
[music] In this exercise, we will be using a simulator to explore superscalar
microprocessor design. The simulator has a number of parameters that we can
configure, such as the number of pipeline stages, the width of the Fetch stage, the
width of the issue, and the number of ALUs in the design. We can see a diagram of the
processor that we've created and we can also see a number of parameters about that
design, for example the clock frequency and the overall area of the design. When we
press step, the simulator will advance one clock cycle and so for example here we can
see that the fetch stage has fetched the first 4 instructions. However, immediately
we see one of the problems with designs such as this, which is the three of the four
instructions that have been fetched are in fact useless because the first instruction
was an unconditional taken branch and therefore the remaining three instructions will
not be executed by the program and so these will immediately be discarded on the next
cycle. Pressing the run button allows us to simulate and we can use the fast-forward
option to simulate much quicker in order to get to the end of the long program
execution. In this case we can see that our design achieved an average cycles per
instruction less than one, which is to say that we on average executed more than one
instruction per cycle, which means that our superscaler design has fundamentally
worked, and we can see that, for example, the overall execution time of the program
is 1.6 milliseconds. In a table below, we can also see a record of our previous
simulation runs. So let's say for example, we were curious about the effect of
increasing the issue width by one. We can make that change and then press run again
in order to run our new design, and when it finishes we can scroll down to take a
look and we can see that the program execution time has indeed improved down to 1.51
milliseconds at a cost of only 1% area. So it looks like this was a very good
improvement to our design and it's almost surely going to be a beneficial trade-off
in practice. In the exercise you will be invited to configure a number of different
superscalar processor designs with various targets in terms of clock frequency,
design area and execution time. Once you're happy that you've configured the process
that you think completes the exercise, you can scroll all the way down to the bottom,
where you'll see the submit button that you can press to have your answer checked.
Good luck. [music]

Module 6, Video 1
Introduction:

So hi, my name is Peter Greenhalgh. I'm Senior Vice President of Technology and an
Arm fellow. I'm responsible for the Central Technology Group at Arm. We're about 250
people. We work on everything from machine learning to CPU, GPU, system IP, and the
solutions that we create as well. And we basically path-find future technology at the
product level that goes into all of our products and the IP that we produce. Arm is
known for the power efficiency of its microprocessors. How have you managed to keep a
focus on power when building processors with very complex and power-hungry features?
We've got some really great design teams. In fact, we churn out I think more CPUs
than pretty much anyone else does on the planet. I think we're producing something
like four or five CPUs per year. So we've got a lot of experience in designing for
power efficiency and performance, and in fact we can leverage the understanding that
we have all the way down to microcontrollers through to the smaller A-class
processors, all the way up to the high performance. There's a lot of sharing between
the teams in terms of strong knowledge and capability, and insight into how to design
for both performance and power efficiency. More specifically, I mean, ultimately, you
have a performance goal that you need to achieve, and then as part of that you have
to figure out how to get the best possible power out of the design when you're
achieving that performance goal. And to do that, there's kind of some different ways
of looking at it. There's the really detailed orientated work that you need to do
around things, like clock gating, data gating, all the things to try and stop
unnecessary power use deep within the microarchitecture when the instructions are
flowing through the pipeline or data's moving through the pipeline. And then there's
essentially the structure of the design that you've created. And that then dictates
fundamentally what the power of the design is going to be. You can't fix a design
that's got bad structure with improved clock gating, data gating, and just good low-
level design. You have to marry the two together. And that high-level work that you
do is around making sure that the pipeline is well balanced, that you aren't opening
up the pipeline, going too wide too soon; you're extracting data, you're extracting
information as late as you possibly can and just when you need it, and not just
pipelining it down through the design for the sake of it; and then, fundamentally,
good microarchitecture around branch prediction, which stops you putting things down
through the pipeline that you're just ultimately going to flush; good pre-fetching on
the data side so that you make sure you get the data in the design when you need it,
and you're not sitting around waiting for it. So you have to marry that altogether,
and we've got a lot of great techniques in order to achieve that, which
fundamentally, I say, you need to achieve the performance target, and then everything
else comes together to achieve that performance target in the best possible energy
efficiency. How did Moore's Law affect computer architectures of the past, and what
will its influence be on future designs? Gordon Moore's influence on the industry has
been massive, and the tenets behind the law still continue today, albeit in a
slightly different form. I mean, I started designing at .18 Micron, essentially 180
nanometers, and here we are today working on 3 nanometers. So it's a vast difference
now compared to when I started 22 years ago. And there's no way we could have got to
where we are today without the process scaling from all of the foundries out there
and all the companies that provide the foundry technology. So it's a little bit like
magic, all of the work that they do. I can't say I understand it in detail, but it's
incredible technology, and that allows us... If it hadn't happened, we'd still be
stuck in all the designs which were fairly simple. There's no way that we'd have got
to the sort of designs that we have today of massively out of order, deep, deep
amount of instruction and depth, very, very wide designs. All of that has been made
possible by the steady improvement, a predictable improvement of the foundries. And
that's kind of one of the key points which really was captured by Moore's law or is
captured by Moore's law of that kind of predictable knowledge that you will get an
improvement in the process. Is it 10%, is it 15? Is it 20% on, say, power, for
example? It kind of doesn't matter in a way because you can work with what you
eventually get. You can do things like voltage scaling to be able to make use of the
power that's available to you. Is it 5%? Is it 10% on frequency? Again, it kind of
doesn't matter in a way. But what matters is when we start designing a processor
today and we finish it in 18 months time, and then two years after that it arrives in
the product in the shops that consumers can buy. We know that over that period, the
process improvements have happened, which allows us to liberate essentially more
performance, more energy efficiency from the design. And we don't mind too much if it
takes another three months or six months to get to the process. We don't mind too
much if the performance or power is not exactly where it was predicted at the start.
But, ultimately, we know we'll get an improvement, and we know there'll be an
improvement in two years, and three years, and four years, and Moore's Law may have
slowed, but it's certainly not stopped.

[music] As we saw in the last module, instruction level parallelism can be used to
improve program execution time in our microprocessor designs. To enable this, the
compiler creates an optimized instruction schedule when the program is converted into
machine code. Unfortunately, the compiler cannot know precisely what will happen at
run-time, so this design is constrained by the order of instructions in the program.
The compiler won't know what the program's input data will be, whether branches will
be mispredicted, or whether memory accesses hit or miss in our data cache. In
contrast, a superscalar processor with "out-of-order" execution can produce an
instruction schedule at run-time, only constrained by true data dependencies and its
hardware limits. This schedule is produced on demand and so can even change each time
the code runs. To do this, we introduce an "issue window" or "issue queue" after the
Decode stage. This holds instructions until they can be executed, not necessarily in
the order they arrived in. Within this window, instructions can be issued whenever
their dependencies are available, and when a functional unit is available to process
it. To be able to detect when an instruction is ready to be issued, we must know
whether the instruction's dependencies are ready when it enters the issue window. We
must then update this status as new results are produced. To implement this, the
names of result registers of executed instructions are broadcast to the issue window.
The instructions waiting there compare the register names to the registers they
require. However, this scheme has a problem: A register will be written multiple
times in the program, and since the instructions are executed out-of-order, the
register name alone is not sufficient to record dependencies. It also means that
instructions would have to wait until all previous reads of a register had finished
before executing. These are called "false dependencies." These problems can be
resolved by "renaming" register names at run-time so that each "in-flight"
instruction writes to a unique destination register. We use a "physical register
file" that is large enough to ensure we don't run out. We keep a "register mapping
table" to store the mapping between architectural, compiler-assigned registers, and
physical registers. Register reads to the same architectural register are renamed
consistently, so that dependencies can be tracked correctly with physical register
names. Physical registers are reused only when they are no longer used by any
instruction currently in-flight or any entry in the register mapping table. The other
big issue with out-of-order execution is memory dependencies. Load and store
instructions can have memory dependencies because they access the same memory
location. To detect this, we need to compare the computed memory addresses that the
instructions access. We thus split memory operations into two steps: address
calculator and memory access. We issue their address calculation step as soon as the
dependencies are available. Then, the memory access step is placed in a special load-
store queue to be sent to our data cache as soon as possible. We carefully ensure
that operations that access the same address are kept properly ordered, but
independent accesses can be reordered if beneficial. No access can occur until the
addresses of all previous accesses are known. Since memory writes are irreversible,
store instructions must also wait until we are certain that they will execute.
[music]

Module 6, Video 2
[music] In the previous video, we outlined the concepts of out-of-order execution,
and register renaming. The issue window will be filled with instructions fetched
along the path that our branch predictor believes the program will take. While we
hope our branch predictor will be correct in most cases, it will sometimes be wrong.
How do we handle such cases? A simple approach is to start by recording the original
program order of the instructions, and then to monitor their progress. We call the
structure that stores the instructions the "reorder buffer." As each instruction
executes and produces a result, we can mark it as done. When the oldest instruction
has completed, we can remove it from the end of the reorder buffer, and the
instruction is said to have "committed." This stream of committed instructions
represents how our program would be executed on a simple in-order pipeline or by an
unpipelined processor. It usefully also provides a point at which we can process
exceptions. For example, if the program divides by zero or attempts to access memory
that does not exist. We also check branch instructions as they complete in order. If
they have been mispredicted, we flush the reorder buffer, our instruction window and
any currently executing instructions and start fetching down the correct path. To
preserve correctness, we must also restore our registers and the register map table
to the values they had when we mispredicted the branch. This can be done with the aid
of a second register map table, updated only when instructions commit in program
order. This can simply be copied to the map table used by our renaming hardware to
"rewind time" for the processor. All the register values we need will be present, as
we don't recycle registers before we know they will not be needed again. In reality,
handling branches in this way is too slow. Processors instead take many copies of the
register map tables and can handle branches as soon as they are resolved, and we
discover they have been mispredicted. They can also selectively neutralize the in-
flight instructions in the datapath that are on the wrong path, rather than flushing
all of these instructions away. [music]

Module 6, Video 3
[music] We can now bring everything together and look at what a typical pipeline for
an out-of-order superscalar processor might look like. The Fetch stage is aided by an
accurate branch predictor as we met in Module 3. It will fetch a group of
instructions on every clock cycle. This group of instructions will be requested from
the instruction cache, and will be from consecutive memory locations. Branches may
reduce the number of useful instructions that can, in practice, be fetched in on each
cycle. The Decode stage decodes multiple instructions in parallel. At this point,
modern high-performance processors may also split complex instructions into simpler
operations or "macro-ops." In some cases, there may also be opportunities to combine
simple instructions into a single operation. The next step on an instruction's
journey is renaming to receive a unique destination register. As we saw in the last
video, this increases opportunities for out-of-order execution. Remember, there are
several times more physical registers in our processor than those available to the
compiler. Instructions are placed in the reorder buffer, and are also "dispatched" to
the Issue stage. They will wait in the window as necessary, and are ready to be
issued once all their operands are available. In the most complex of today's
superscalar processors, there may be hundreds of instructions buffered in the issue
window at the same time. Instructions finally commit in program order. At this point,
any physical registers that are no longer needed can be added back to the pool of
free registers. These are then assigned during the register renaming step. Once an
instruction is issued, it reads its operands from the physical register file. The
Execute stage consists of many functional units operating in parallel. These may each
support different operations and take different numbers of cycles to execute. A
network of forwarding paths is also provided to ensure we can execute any dependent
instruction on the next clock cycle after the generation of the result. This requires
being able to quickly communicate—or "forward"— a result from the output of any
functional unit, to the input of any other. Some instructions will need access to
memory. After computing their addresses, they are placed in the processor's load-
store queues. "Stores" are sent to memory in program order, but "loads" can often be
sent out of order, and ahead of other older stores or loads that are not yet ready to
be issued to memory. The memory system reduces the average memory access time by
providing numerous levels of cache memory. After generating results, we write them
back to the register file. This overview is representative of the fastest modern
microprocessors found today in laptops, smartphones and servers. Whilst much extra
innovation goes into real designs, they generally follow the ideas discussed in the
course. [music]

Module 6, Video 4
[music] One question computer architects always ask themselves is: "how much can we
scale up our design?" Let's take a look at some further potential optimizations to
our out-of-order superscalar processor. We could try to make it wider. For example,
by doubling the number of parallel instructions, we can fetch, decode and execute
more instructions per cycle. Would this double our performance? Sadly, no, things are
not that simple! In practice, some components quickly become very complex, and
performance gains may be hard to extract. For example, today's largest machines fetch
at most ten instructions per cycle from their instruction caches. Fetching more
instructions than this offers minimal performance gain, despite a large hardware
cost. If we increase the number of registers, the size of our issue window, the size
of our load-store queues, or perhaps use a larger and more accurate branch predictor,
our processor's performance will only improve slightly despite a significant increase
in the size of these structures. After a point, the increase in performance is no
longer worth the cost of the extra transistors. It's also possible that performance
might reduce overall as we may need to lower our clock frequency as the structures
get larger. Finally, we could introduce more pipeline stages, but we know this
doesn't necessarily lead to higher performance, as mispredictions may become more
costly. The combination of these issues means that extracting performance using
instruction-level parallelism alone becomes more expensive as more performance is
sought. This graph shows how the energy cost of executing an instruction grows
quickly as we try to build higher performance processors. Let's look at some example
designs. Suppose we have a core, which requires a certain area. If we double its
area, its performance improves, although there is a small rise in energy per
instruction. If we quadruple its area instead, its performance has now doubled
compared to our original core, while energy has increased by 50 percent. Going
further, if we increase our processor's area by a factor of 10, performance is only 2
point 5 times our original core, but energy per instruction is now 3 times higher.
Its performance does not improve as fast as the cost of running it! Of course,
engineers are clever and determined, and are constantly developing new techniques to
bypass many of these issues. This means the performance of processors—even ones
running a single thread or program— still improves by around 10 to 25 percent each
year. Nevertheless, ultimately we often need more performance than can be provided by
instruction-level parallelism alone. A modern solution is to employ multiple
processor cores on the same chip—called a "multicore" processor. This changes the
task for programmers; they may need to redesign their programs to take advantage of
such parallelism, but if they can, it can give vast performance benefits. As we've
learned throughout the course, every decision involves trade-offs and compromise. We
are faced with a fascinating but often highly-constrained design problem. We've seen
how performance bottlenecks, that at first seem impassable, can be overcome with
innovative designs. What might the future hold for microprocessors? Can you think of
ideas? What would you design? [music]
Module 6, Lab
[music] In this exercise, we'll be using a simulator to explore an out-of-order
superscalar processor design. The simulator allows us to configure a number of
aspects of our processor, for example, the number of pipeline stages, the width of
the Fetch stage, the size of the issue window, the number of ALUs, and the size of
our re-order buffer. The changes will be reflected in the pipeline diagram, which we
can see below, and also in the statistics below that, with key design metrics, such
as the clock frequency, clock period, and design size, visible below. When we press
"step," the simulation will advance by 1 clock cycle, and so we can see, for example,
on the first clock cycle the 1st 4 instructions are loaded. Although three of them
are unusable because they follow a taken branch and therefore these will not be
executed and will be discarded on the next clock cycle. We can press "run" to watch
our design in action and in order to quickly get to the end, we can use the "fast
forward" feature to simulate the millions of instructions in this particular program.
After the simulation is complete, we can check below to see a number of statistics
about our pipeline, which are useful for understanding why the performance is as it
is, and in particular we can see the program execution time. 1.18 milliseconds gives
us the overall execution time of the program that our design achieved, and notably
our average instructions per cycle gives us the number of instructions that we were
able to complete on each clock cycle, in this case well over 1, indicating that we
are taking advantage of instruction level parallelism in this simulation. Below all
that, we can see our completed simulations and we can check back on these as we
explore new designs. So let's say, for example, we were curious about the effect of
increasing the re-order buffer size. We could change that and then press "run" to
quickly run our next experiment. And then if we scroll down to our table, we can
compare the results and see that actually the program execution time was
substantially increased by increasing the size of the re-order buffer. Although,
admittedly, this did come at a not insignificant increase in the area of our design,
and so whether or not this represents a good trade-off in practice would very much
depend on the problem we're trying to solve. In the exercises, you'll be given a
number of scenarios for processors, which generally revolve around certain
constraints on the design size, or the clock frequency, or the target program
execution time. And once you think you've found a design that meets the goals
required, configure it up using the settings and then scroll down to the "submit"
button at the bottom, and click that to have your answer checked. Best of luck.
[music]