ia ..

er ly
at on
l..
m ts
se en
ur ud
co St
ig EM
NOT FOR PUBLIC RELEASE
yr O
By RCOEM at 15:03:47, 25-11-2023
ht
op C
C rR

Memory System Design

Fo

Suresh Balpande
Overview

ia ..
er ly
at on
Basic memory circuits

l..
⚫

m ts
⚫ Organization of the main memory

se en
ur ud
⚫ Cache memory concept

co St
⚫ Virtual memory mechanism
ig EM
Secondary storage
yr O
⚫
ht
op C
C rR
Fo
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
An Example Memory Hierarchy
Memory Hierarchy
Main Memory I/O Processor

ia ..
CPU

er ly
at on
l..
m ts
se en
ur ud
co St
Cache
ig EM
yr O
ht
op C
C rR
Fo

Magnetic
Disks Magnetic Tapes
4 / 19
 Memory Arrays

ia ..
er ly
Memory Arrays

at on
l..
m ts
Content Addressable Memory

se en
Random Access Memory Serial Access Memory
(CAM)

ur ud
Read/Write Memory Read Only Memory

co St
Shift Registers Queues
(RAM) (ROM)
(Volatile)
ig EM
(Nonvolatile)
yr O
Serial In Parallel In First In Last In
ht
Static RAM Dynamic RAM Parallel Out Serial Out First Out First Out
op C

(SRAM) (DRAM) (SIPO) (PISO) (FIFO) (LIFO)

C rR
Fo

Mask ROM Programmable Erasable Electrically Flash ROM

ROM Programmable Erasable
(PROM) ROM Programmable
(EPROM) ROM
(EEPROM)
Access Modes
Four types

ia ..
er ly
⚫ RAM (Random Access Mode)

at on
l..
m ts
⚫ SAM (Serial Access Mode)

se en
⚫ Semi Random Access Mode

ur ud
co St
⚫ Associative Access Mode
ig EM
yr O
ht
op C

Memory retention
C rR
Fo

PROM, ROM, RAM,

Sequential Access Method

ia ..
er ly
at on
Start at the beginning and read through in

l..
⚫

m ts
order

se en
Access time depends on location of data and

ur ud
⚫

co St
previous location
ig EM
⚫ Example: tape
yr O
ht
op C
C rR
Fo
Direct Access Method

ia ..
er ly
at on
Individual blocks have unique address

l..
⚫

m ts
⚫ Access is by jumping to vicinity then

se en
performing a sequential search

ur ud
co St
⚫ Access time depends on location of data
ig EM
within "block" and previous location
yr O
ht
op C

⚫ Example: hard disk

C rR
Fo
Random Access Method

ia ..
er ly
at on
Individual addresses identify locations exactly

l..
⚫

m ts
⚫ Access time is consistent across all locations

se en
and is independent previous access

ur ud
co St
⚫ Example: RAM ig EM
yr O
ht
op C
C rR
Fo
Associative Access Method

ia ..
er ly
⚫ Addressing information must be stored with

at on
l..
data in a general data location

m ts
se en
⚫ A specific data element is located by a

ur ud
comparing desired address with address

co St
portion of stored elements
ig EM
yr O
⚫ Access time is independent of location or
ht
op C
C rR

previous access
Fo

⚫ Example: cache
Performance and cost:

ia ..
er ly
at on
C=Memory storage + access circuitry

l..
⚫

m ts
⚫ S=Bits of storage capacity

se en
ur ud
co St
⚫ Cost c of memory= C/S
ig EM
yr O
ht
op C
C rR
Fo
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Semiconductor RAM
Memories
Random-Access Memory
(RAM)

ia ..
er ly
⚫ Static RAM (SRAM)

at on
l..
⚫ Each cell stores bit with a six-transistor circuit.

m ts
⚫ Retains value indefinitely, as long as it is kept powered.

se en
⚫ Relatively insensitive to disturbances such as electrical noise.

ur ud
⚫ Faster (8-16 times faster) and more expensive (8-16 times more

co St
expensice as well) than DRAM.
ig EM
yr O
⚫ Dynamic RAM (DRAM)
ht
op C

⚫ Each cell stores bit with a capacitor and transistor.

C rR

⚫ Value must be refreshed every 10-100 ms.

Fo

⚫ Sensitive to disturbances.
⚫ Slower and cheaper than SRAM.
 SRAM vs DRAM Summary

ia ..
er ly
at on
l..
m ts
Tran. Access

se en
per bit time Sensitive? Cost Applications

ur ud
co St
SRAM 6 1X No 100x cache memories

DRAM 1 10X
ig EM
Yes 1X Main memories,
yr O
frame buffers
ht
op C
C rR

⚫ Virtually all desktop or server computers since 1975

Fo

used DRAMs for main memory and SRAMs for cache

 Static Memories
⚫ The circuits are capable of retaining their state as long as power
is applied.

ia ..
er ly
at on
l..
m ts
se en
ur ud
co St
ig EM
yr O
ht
op C

To write information the data is imposed on the bit line and the
C rR

inverse data on the inverse bit line.

Fo

Then the access transistors are turned on by setting the word

line to high. As soon as the information is stored in the
inverters, the access transistors can be turned off and the
information in the inverter is preserved.
Static Memories
⚫ CMOS cell: Low power consumption

ia ..
er ly
b Vsupply b

at on
l..
m ts
se en
T T
3 4

ur ud
T1 T2

co St
X Y
ig EM
T T6
yr O
5
ht
op C
C rR
Fo

Word line

Bit lines

An example of a CMOS memory cell.

 DRAMs

ia ..
er ly
⚫ Static RAMs are fast, but they cost more area and are more expensive.

at on
l..
⚫ Dynamic RAMs (DRAMs) are cheap and area efficient, but they can not
retain their state indefinitely – need to be periodically refreshed.

m ts
se en
Bit line

ur ud
co St
Word line
ig EM
yr O
ht
op C
C rR

T
C
Fo

Figure 5.6. A single-transistor dynamic memory cell

DDR SDRAM (Double-Data-Rate SDRAM)

ia ..
er ly
Standard SDRAM performs all actions on the rising

at on
⚫

l..
edge of the clock signal.

m ts
se en
⚫ DDR SDRAM accesses the cell array in the same

ur ud
way, but transfers the data on both edges of the

co St
clock. ig EM
⚫ The cell array is organized in two banks. Each can
yr O
ht
be accessed separately.
op C
C rR

⚫ DDR SDRAMs and standard SDRAMs are most

Fo

efficiently used in applications where block

transfers are prevalent.
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Read-Only Memories
 Read-Only-Memory

ia ..
er ly
at on
Volatile / non-volatile

l..
⚫
memory

m ts
Bit line

se en
⚫ ROM

ur ud
⚫ PROM: Word line
programmable ROM

co St
⚫ EPROM: erasable, ig EM
reprogrammable Not connected to store a 1
yr O
T
ROM Connected to store a 0
ht
op C
C rR

⚫ EEPROM: can be P
programmed and
Fo

erased electrically

Figure 5.12. A ROM cell.

Flash Memory

ia ..
er ly
⚫ Similar to EEPROM

at on
l..
m ts
⚫ Difference: only possible to write an entire block

se en
of cells instead of a single cell

ur ud
co St
⚫ Low power
ig EM
⚫ Use in portable equipment
yr O
ht
op C

⚫ Implementation of such modules

C rR

⚫ Flash cards
Fo

⚫ Flash drives i.e. Memory cards and pen drives

 Main Memory

MEMORY ADDRESS MAP

Example: 512 bytes RAM using 128 Bytes and
512 bytes ROM

ia ..
er ly
at on
l..
Hexa Address bus
Component

m ts
address 10 9 8 7 6 5 4 3 2 1

se en
RAM 1 0000 - 007F 0 0 0 x x x x x x x

ur ud
RAM 2 0080 - 00FF 0 0 1 x x x x x x x
RAM 3 0100 - 017F 0 1 0 x x x x x x x

co St
RAM 4 0180 - 01FF 0 1 1 x x x x x x x
ROM ig EM
0200 - 03FF 1 x x x x x x x x x
yr O
Address space assignment to each memory chip
ht
op C
C rR

Memory Connection to CPU

Fo

- RAM and ROM chips are connected to a CPU through the data and address
buses

- The low-order lines in the address bus select the byte within the chips and
other lines in the
address bus select a particular chip through its chip select inputs
 MAIN MEMORY RAM and ROM Chips

Typical RAM chip

ia ..
Chip select 1 CS1

er ly
Chip select 2 CS2

at on
l..
128 x 8
Read RD 8-bit data bus
RAM
Write WR

m ts
7-bit address AD 7

se en
ur ud
co St
CS1 CS2 RD WR Memory function State of data bus

ig EM
0
0
0
1
x
x
x
x
Inhibit
Inhibit
High-impedence
High-impedence
yr O
1 0 0 0 Inhibit High-impedence
ht
op C

1 0 0 1 Write Input data to RAM

C rR

1 0 1 x Read Output data from RAM

1 1 x x Inhibit High-impedence
Fo

Typical ROM chip

Chip select 1 CS1
Chip select 2 CS2
512 x 8 8-bit data bus
Read RD ROM
9-bit address AD 9
 Main Memory
CONNECTION OF MEMORY TO CPU
CPU
Address bus
16-11 10 9 8 7-1 RD WR Data bus

ia ..
er ly
Decoder

at on
l..
3 2 1 0
CS1 AD7 means AD7 to AD1
CS2

Data
m ts
RD 128 x 8
RAM 1

se en
WR
AD7

ur ud
CS1
CS2

Data
co St
RD 128 x 8
WR RAM 2
ig EM AD7

CS1
CS2
yr O

Data
RD 128 x8
ht
RAM 3
op C

Why WR
AD7
C rR

CS1
Fo

CS2
RD 128 x 8 Data
WR RAM 4
AD7
CS1
CS2
Data

1- 7 512 x 8
8
9 } AD9 ROM
 ia ..
er ly
at on
l..
m ts
se en
ur ud
co St
ig EM
yr O
ht
op C
C rR
Fo

Cache Memories
M. V. Wilkes, “Slave Memories and Dynamic Storage Allocation,”
IEEE Transactions on Electronic Computers, vol. EC-14, no. 2, pp. 270-
271, April 1965.
 Cache memory

ia ..
er ly
⚫ If the active portions of the program and data are

at on
l..
placed in a fast small memory, the average

m ts
se en
memory access time can be reduced,

ur ud
⚫ Thus reducing the total execution time of the

co St
program ig EM
⚫ Such a fast small memory is referred to as cache
yr O
ht
op C

memory
C rR

⚫ The cache is the fastest component in the memory

Fo

hierarchy and approaches the speed of CPU

component
 Cache memory

ia ..
When CPU needs to access memory, the

er ly
⚫

at on
l..
cache is examined

m ts
se en
ur ud
co St
⚫ If the word is found in the cache, it is read
ig EM
from the fast memory
yr O
ht
op C
C rR

⚫ If the word addressed by the CPU is not found

Fo

in the cache, the main memory is accessed to

read the word
Cache memory

ia ..
er ly
at on
l..
⚫ When the CPU refers to memory and finds the

m ts
word in cache, it is said to produce a hit

se en
ur ud
⚫ Otherwise, it is a miss

co St
ig EM
⚫ The performance of cache memory is frequently
yr O
ht
op C

measured in terms of a quantity called hit ratio

C rR

⚫ Hit ratio = hit / (hit+miss)

Fo

Miss Ratio = miss / (hit + miss) = no. of miss/total accesses

= 1 - hit ratio(H)
 Cache Memory
⚫ High speed (towards CPU speed)

ia ..
er ly
⚫ Small size (power & cost)

at on
l..
m ts
se en
Miss

ur ud
Main

co St
CPU Memory
ig EM
Cache (Slow)
yr O

Mem
ht
op C

(Fast)
C rR

Hit Cache
Fo

95% hit ratio

Access = 0.95 Cache + 0.05 Mem 29 / 19

Cache/Main Memory Structure

ia ..
er ly
at on
Tag- unique

l..
identifier for a

m ts
group of data

se en
ur ud
co St
ig EM
yr O
ht
op C
C rR
Fo

Main
Processor Cache memory

Figure . Use of a cache memory.

 Definitions

ia ..
er ly
at on
l..
⚫ Cache block - The basic unit for cache storage. May

m ts
contain multiple bytes/words of data.

se en
⚫ Cache line - Same as cache block

ur ud
co St
⚫ Cache set - A “row” in the cache. The number of blocks
ig EM
per set is determined by the layout of the cache (e.g.
yr O
direct mapped, set-associative, or fully associative).
ht
op C
C rR

⚫ Tag - A unique identifier for a group of data. Because

different regions of memory may be mapped into a
Fo

block, the tag is used to differentiate between them.

 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
Cache write Operation

l..
 Locality of Reference

Cache memory is based on the principle of locality of reference.

ia ..
● Locality of reference

er ly
at on
l..
Locality of reference refers to a phenomenon in which a computer

m ts
program tends to access same set of memory locations for a

se en
particular time period.

ur ud
● Temporal locality

co St
Temporal locality means current data or instruction that is being
ig EM
fetched may be needed soon.
yr O
● Spatial locality
ht
op C

Spatial locality means instruction or data near to the current memory

C rR

location that is being fetched, may be needed soon in the near future.
Fo

If the active portions of the program and data are placed in a fast
small memory, the total execution 34 time of the program can be
reduced.
 Writing to Memory

ia ..
⚫ Cache and memory become inconsistent when data is

er ly
at on
l..
written into cache, but not to memory – the cache

m ts
coherence problem.

se en
ur ud
⚫ Strategies to handle inconsistent data:

co St
⚫ Write-through ig EM
Write to memory and cache simultaneously always.
yr O
⚫
ht
op C

⚫ Write to memory is ~100 times slower than to cache.

C rR

⚫ Write-back
Fo

⚫ Write to cache and mark block as “dirty”.

⚫ The dirty block is thrown out of the cache to make room for
another block, which is done later.
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Write-through vs Write-Back

36
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Write-through vs Write-Back
 Address Mapping

ia ..
00000000 Main

er ly
00000001

at on
Memory

l..
•
Cache •

m ts
00000

se en
00001 •
• •

ur ud
• •

co St
• •
• ig EM •
FFFFF •
yr O
•
ht
op C

•
C rR

3FFFFFFF
Fo

Address Mapping !!!

39 / 19
 Address Mapping

ia ..
The memory system has to quickly determine if a

er ly
at on
l..
given address is in the cache. There are three

m ts
popular methods of mapping addresses to cache

se en
ur ud
locations.

co St
ig EM
⚫ Direct-Each address has a specific place in the cache.
yr O
ht
op C

⚫ Fully Associative– Search the entire cache for an

C rR

address.
Fo

⚫ Set Associative– Each address can be in any of a

small set of cache locations.
 1. Direct Mapping
The simplest technique, known as direct mapping,

ia ..
maps each block of main memory into only one

er ly
at on
l..
possible cache line. or In Direct mapping, assign each

m ts
memory block to a specific line in the cache.

se en
ur ud
i = j modulo m

co St
where
i = cache line number ig EM
j = main memory block number
yr O
m = number of lines in the cache
ht
op C
C rR

Example : say number

Fo

of lines m=4
0 mod 4 => 0
1 mod 4=> 1
…..
4 mod 4=> 0
1. Direct Mapping
The simplest technique, known as direct
mapping, maps each block of main memory
Each block contains 32 words

ia ..
into only one possible cache line.

er ly
Or

at on
l..
In Direct mapping, assign each memory

m ts
block to a specific line in the cache.

se en
ur ud
co St
ig EM
yr O
ht
op C
C rR

select one of the 32 words in a block

Fo

7 bit cache block field

determiners the Cache position

16 Pages
 2. Fully Associative Mapping

ia ..
⚫ A main memory block can load into any line of cache

er ly
at on
l..
⚫ Memory address is interpreted as:

m ts
⚫ Least significant w bits = Block offset

se en
⚫ Most significant s bits = Tag used to identify which block is stored in a

ur ud
particular line of cache

co St
⚫ Every line's tag must be examined for a match
ig EM
⚫ Cache searching gets expensive and slower
yr O
ht
op C
C rR

Tag – Block Number Block offset

Fo

(5 in example) (2 in ex.)

As there is no fix block, the memory address has only two fields:
word and tag.
 Line size=Block Size

Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Fully Associative Mapping
Block Size
 Associative Mapping

Address

ia ..
er ly
00012000

at on
l..
m ts
se en
Can have Cache
any number

ur ud
of locations 00012000 01A6

co St
Data
ig EM
15000000 0005 01A6
yr O
ht
op C

08000000 47CC
C rR
Fo

Tag Word
12 4 Main memory address

30 Bits 16 Bits
(Key) (Data) 111011111111,1100
45 / 19
 Associative Memory

Cache Location

ia ..
00000000 Main

er ly
00000001

at on
Memory

l..
•
00000 Cache •

m ts
se en
00001 00012000
• •

ur ud
00012000
• •

co St
• 08000000
15000000
• ig EM •
FFFFF
08000000 •
yr O
15000000
ht
op C

•
C rR

3FFFFFFF
Fo

Address (Key) Data

46 / 19
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Fully Associative Cache Organization
 Cntd..
Advantage

ia ..
⚫ Any empty block in cache can be use

er ly
at on
l..
⚫ Flexible arrangement

m ts
se en
⚫ Must check all tags to check for a hit,

ur ud
co St
⚫ expensive
ig EM
What is the next technique?
yr O
ht
op C

Something between direct mapping and

C rR

associative mapping
Fo
 3. Set Associative Mapping

ia ..
er ly
at on
l..
⚫ Set associative mapping combines direct mapping with

m ts
fully associative mapping by arrangement lines of a

se en
cache into sets.

ur ud
co St
⚫ Set-associative mapping allows each word that is
ig EM
present in the cache can have two or more words in the
yr O
main memory for the same index address.
ht
op C
C rR

⚫ Set associative cache mapping combines the best of

Fo

direct and associative cache mapping techniques

 Set Associative Mapping

ia ..
er ly
at on
l..
m ts
se en
ur ud
No. of Blocks=

co St
No. of lines ig EM
In cache =
yr O
ht
op C
C rR

Assuming
Fo

No. of set=2 i.e. S0 and S1

 Set Associative Mapping

ia ..
er ly
at on
l..
Instead of
lines, have

m ts
to consider

se en
set here

ur ud
co St
ig EM
yr O
ht
op C
C rR

Block
Fo

i = k modulo n
where
i = set number
i.e. Set 1 k = main memory block number
i.e. Set 0 n = Total number of set
 Set Associative Mapping

ia ..
er ly
at on
l..
m ts
se en
ur ud
co St
ig EM
yr O
ht
op C
C rR
Fo

The 5-bits word field selects one of the 32 words in a block. The set field needs 6-bits to
determine the desired block from 64 sets. However, there are now 31 pages. To identify
which of the 32 blocks (pages) that are mapped into the particular set of cache, five tag
bits are required.
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
K-Way Set Associative Cache Organization
 Average access time Tav at a level

ia ..
Tav = (Thit * Phit) + (Tmiss * Pmiss)

er ly
at on
l..
m ts
se en
⚫ T hit = The time taken to resolve requests that hit in the

ur ud
level,

co St
⚫ P hit = The hit rate of the level (expressed as a
ig EM
probability)
yr O
ht
op C

⚫ Tmiss = The average access time of the levels below this

C rR

one in the hierarchy, and

Fo

⚫ Pmiss = The miss rate of the level

 Example-1
Assume that hit rate of 75 % at a level of the memory hierarchy. The
memory requests take 12 ns to complete if they hit in the level .

ia ..
er ly
Memory requests that miss takes 100 ns to complete.

at on
l..
Using the formula, the average access time

m ts
se en
= (l2 ns * 0. 75) + (100 ns * 0.25) = 34 ns

ur ud
Example 2: Assume─ a memory system contains the cache, main memory, and

co St
virtual memory • Assume─ the access time of the cache = 5 ns • Cache hit rate = 80
percent • The access time of the main memory = 100 ns • Main memory hit rate =
ig EM
99.5 percent • The access time of the virtual memory = 10 milliseconds (ms) . Start
yr O
at the bottom of the hierarchy and work up • Hit rate of the virtual memory =100
ht
op C

percent
C rR

The average access time for requests that reach the main memory = (100 ns *
Fo

0.995) + (10 ms * 0.005) = 50,099.5 ns

Given this, the average access time for requests that reach the cache (which
is all requests) = (5 ns*0.80) + (50,099.5 ns * 0.20) =10,024 ns
Example 3
The average memory access time for a machine with a cache

ia ..
er ly
hit rate of 90% where the cache access time is 10 ns and the

at on
l..
memory access time is 100 ns is

m ts
se en
⚫ Sol:

ur ud
Average memory access time =

co St
Hit Ratio x Cache access time + Miss Ratio x Memory
ig EM
access time
yr O
ht
op C
C rR
Fo

= 0.90 x 10 ns + 0.10 x 100 ns

= 9 ns + 10 ns
= 19 ns
Exercises
1. Calculate the average time experienced by a processor if a cache

ia ..
er ly
hit rate is 0.88, miss penalty is 0.012 milliseconds and cache access

at on
l..
time is 10 microseconds

m ts
2. In a certain system the main memory access time is 100 ns. The

se en
ur ud
cache is 10 time faster than the main memory and uses the write

co St
though protocol. If the hit ratio for read request is 0.92 and 85% of
ig EM
the memory requests generated by the CPU are for read, the
remaining being for write; then the average time consideration both
yr O
ht
op C

read and write requests is

C rR

Hint: Memory access time = 100 ns , cache access time would be =

Fo

10 ns (10 time faster)

Example 4
A block-set associative cache memory consists of 128 blocks divided into four block
sets . The main memory consists of 16,384 blocks and each block contains 256 eight
bit words.

ia ..
er ly
1. How many bits are required for addressing the main memory?

at on
l..
2. How many bits are needed to represent the TAG, SET and WORD fields?

m ts
Number of Bits in Block Offset-

se en
Given-
•Number of blocks in cache memory = 128

ur ud
We have-Block size= 256 bytes= 28 bytes Thus, Number of
•Number of blocks in each set of cache = 4 bits in block offset or word = 8 bits

co St
•Main memory size = 16384 blocks
•Block size = 256 bytes
•1 word = 8 bits = 1 byte
ig EM Number of Bits in Set Number-
yr O
Main Memory Size- Number of sets in cache= Number of lines in cache / Set size
ht
op C

We have-Size of main memory = 128 blocks / 4 blocks= 32 sets= 25 sets

C rR

= 16384 blocks Thus, Number of bits in set number = 5 bits

Fo

= 16384 x 256 bytes=4MB

= 222 bytes Number of Bits in Tag Number-
Thus, Number of bits required to address
main memory = 22 bits Number of bits in tag= Number of bits in physical address –
block offset
(Number of bits in set number + Number of bits in word)= 22
bits – (5 bits + 8 bits)
= 22 bits – 13 bits= 9 bits Thus, Number of bits in tag = 9 bits
 ia ..
er ly
at on
l..
Memory Interleaving

m ts
se en
ur ud
Processor

co St
ig EM
words
yr O
Cache
ht
op C

Small, fast
C rR

memory
Fo

blocks

Memory Memory Memory Memory

bank 0 bank 1 bank 2 bank 3

Main memory
Why do we use Memory Interleaving?
⚫ When the processor requests data from the main memory, a block

ia ..
(chunk) of data is transferred to the cache and then to processor.

er ly
at on
l..
⚫ So whenever a cache miss occurs, the data is to be fetched from the

m ts
main memory. But main memory is relatively slower than the cache.

se en
So to improve the access time of the main memory, interleaving is

ur ud
used.

co St
⚫ For example, we can access all four modules at the same time, thus
ig EM
achieving parallelism. The data can be acquired from the module
yr O
using the higher bits. This method uses memory effectively.
ht
op C
C rR

⚫ Benefits of Interleaved Memory

Fo

⚫ An instruction pipeline may require instruction and operands both at the

same time from main memory, which is not possible in the traditional
method of memory access. Similarly, an arithmetic pipeline requires two
operands to be fetched simultaneously from the main memory. So, to
overcome this problem, memory interleaving comes to resolve this.
Interleaved Memory Processor
Memory Interleaving is an abstraction technique words
which divides memory into a number of modules
Cache

ia ..
such that successive words in the address space are

er ly
Small,

at on
placed in the different module. fast

l..
memory

m ts
blocks

se en
ur ud
Memory Memory Memory Memory
bank 0 bank 1 bank 2 bank 3

co St
Main memory
ig EM
yr O
Suppose a 64 MB memory made up of the 4 MB chips as shown above.
ht
op C

We organize the memory into 4 MB banks, each having eight of the 4 MB chips. The memory
C rR

thus has 16 banks, each of 4 MB.

64 MB memory = 2^26, so 26 bits are used for addressing.
Fo

16 = 2^4, so 4 bits of address select the bank (L) , and 4 MB = 2^22 (M) so 22 bits of address
to each chip.
In general, an N-bit address, with N = L + M, is broken into two parts:
1.L-bit bank select, used to activate one of the 2^L banks of memory, and
2.M-bit address that is sent to each of the memory banks.
Memory Interleaving?
Memory Bank Memory Bank

ia ..
er ly
at on
l..
m ts
se en
ur ud
co St
Memory Bank Memory Bank
ig EM
yr O
ht
op C
C rR
Fo
Types of Interleaved Memory

ia ..
er ly
at on
⚫ High Order interleaving

l..
Based on addressing

m ts
scheme
⚫ Lower Order interleaving

se en
ur ud
co St
⚫ Block Level Interleaving
ig EM Based on Data
scheme
yr O
⚫ Byte Level Interleaving:
ht
op C
C rR
Fo
 Types of Interleaved Memory
⚫ 1. High order interleaving: In high order memory

ia ..
er ly
interleaving, the most significant bits of the memory

at on
l..
address decides memory banks where a particular

m ts
se en
location resides.

ur ud
co St
ig EM
yr O
ht
op C
C rR
Fo
 Low order interleaving:
⚫ The least significant bits select the memory bank

ia ..
er ly
(module) in low-order interleaving. In this, consecutive

at on
l..
memory addresses are in different memory modules,

m ts
se en
allowing memory access faster than the cycle time.

ur ud
co St
ig EM
yr O
ht
op C
C rR
Fo
 Types of Interleaved Memory
Block Level Interleaving: Byte Level Interleaving:
• Byte level interleaving, on the other

ia ..
⚫ Block level interleaving

er ly
hand, distributes individual bytes of

at on
l..
involves organizing
data across multiple memory

m ts
memory into blocks or modules.

se en
chunks, and each block is • This is especially useful for

ur ud
stored in a different scenarios where data is accessed in a

co St
memory module. more scattered or non-contiguous
ig EM
⚫ When a request is made for fashion.
yr O
a particular block of data, • By interleaving at the byte level, the
ht
op C

memory system can effectively

C rR

the memory controller can

handle requests that involve
access multiple modules
Fo

accessing data from different

simultaneously, improving locations within a memory word.
overall memory bandwidth • Byte level interleaving is well-suited
and reducing access for applications that require frequent
latency. access to dispersed data elements.
Summary:

ia ..
er ly
⚫ Memory Interleaving: Memory interleaving is a technique used to

at on
l..
increase memory bandwidth.

m ts
⚫ It involves dividing the memory into multiple banks, and each

se en
bank can be accessed simultaneously.

ur ud
co St
⚫ When a processor requests data, the memory controller can access
ig EM
multiple banks simultaneously,
⚫ increasing the amount of data that can be transferred in a single
yr O
ht
op C

cycle. This reduces the cycle time, and thus increases the memory
C rR

bandwidth.
Fo

⚫ Interleaving can be done at different levels, such as byte-level

interleaving, word-level interleaving, or block-level interleaving
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Virtual Memories
 Overview
Techniques that automatically move program and data blocks
into the physical main memory when they are required for

ia ..
er ly
execution are called virtual-memory techniques.

at on
l..
m ts
⚫ Physical main memory is not as large as the address space spanned by an

se en
address issued by the processor.

ur ud
232 = 4 GB, 264 = …

co St
⚫ ig EM
When a program does not completely fit into the main memory, the parts of
it not currently being executed are stored on secondary storage devices.
yr O
⚫ Virtual addresses will be translated into physical addresses.
ht
op C
C rR

⚫ Virtual memory uses both hardware and software to enable a computer to

compensate for physical memory shortages, temporarily transferring data
Fo

from random access memory (RAM) to disk storage.

⚫ Mapping chunks of memory to disk files enables a computer to treat
secondary memory as though it were main memory.
 Virtual Memory
⚫ Only part of the program needs to be in memory

ia ..
er ly
for execution

at on
l..
m ts
⚫ Logical address space can therefore be much

se en
larger than physical address space

ur ud
co St
⚫ Allows for more efficient process creation
ig EM
yr O
ht
op C
C rR
Fo
What are the benefits of using virtual memory?
•It can handle twice as many addresses as main

ia ..
er ly
at on
l..
memory.

m ts
•It enables more applications to be used at once.

se en
•It has increased speed when only a segment of a

ur ud
co St
program is needed for execution.
ig EM
•It enables multiple larger applications to run
yr O
ht
op C

simultaneously.
C rR

•Allocating memory is relatively inexpensive.

Fo

•It does not need external fragmentation.

•Data can be moved automatically.
 Fo
C rR
op C
yr O
ig EM
ht
co St
ur ud
se en
m ts
at on
er ly
ia ..
l..
Thanks