CHAPTER 4

MEMORY SYSTEM
DESIGN
CHARACTERISTICS OF A MEMORY
SYSTEM
 Location
 Processor
 Internal(Main)
 External (Secondary)

 Capacity
 WordSize
 Number of Words

 Unit of Transfer
 Word
 Block
2
CHARACTERISTICS OF A MEMORY
SYSTEM
 Access Method
Sequential (Tape)
 Start at the beginning and read through in order
 Access time depends on location of data and previous

location
Direct (Disk)
 Individual blocks have unique address
 Access is by jumping to vicinity plus sequential search

 Access time depends on location of data and previous

location
3
CHARACTERISTICS OF A MEMORY
SYSTEM
 Access Method (Contd.)
Random (RAM/ROM)
 Individual addresses identify locations exactly
 Access time is independent of location or previous access

Associative (Cache)
 Based on content
 Data is located by a comparison with contents of a portion

of the store
 Access time is independent of location or previous access

4
CHARACTERISTICS OF A MEMORY
SYSTEM
 Performance
 Access Time
 Time between presenting the address and getting the valid
data
 Cycle Time
 Time may be required for the memory to “recover” before
next access
 Cycle time is access + recovery

 Transfer Rate
 Rate at which data can be moved

5
CHARACTERISTICS OF A MEMORY
SYSTEM
 Physical Type
Semiconductor
 RAM / ROM
Magnetic
 Disk & Tape
Optical
 CD & DVD
Magneto-Optical
 CD-RW 6
CHARACTERISTICS OF A MEMORY
SYSTEM
 Physical Characteristics
Volatile/ Non-Volatile
Erasable / Non-Erasable
Power Consumption

 Organization
Physical arrangement of bits into words
 Not always obvious
7
MEMORY HIERARCHY
 Memory design is governed by three questions:
 How large?
 How fast?
 How much?
 Three rules:
 Faster access time, greater cost per bit.
 Greater capacity, smaller cost per bit.
 Greater capacity, slower access time.

To solve this dilemma, designers use a hierarchy of memory

systems.
8
MEMORY HIERARCHY

Register Inboard Memory

Cache
< $ / bit Main Memory

> Capacity Magnetic Disk

CD ROM Outboard Storage
> Access Time CD-RW
DVD
< Frequency DVD-RW
of access
Magnetic Tape Off-Line
WORM Storage

9
LOCALITY OF REFERENCE
 The memory hierarchy presented works because of a natural
phenomena known as “locality of reference”.
 Locality of reference- is the term for phenomena in which the same
values, or related storage locations, are frequently accessed,
depending on memory access pattern.
 During the execution of a program, memory references for
instructions and data tend to cluster.
 Keeping the current cluster in the faster memory level allows
faster memory access.
10
MAIN MEMORY
 Relatively large and fast.
 Used to store programs and data during the computer
operation.
 The principle technology is based on semiconductor ICs.
 Usually referred to as Random Access Memory (RAM).
 The more accurate name would be Read / Write Memory (R / WM)

11
RAM
 Allows both read and write operations.
 Both operations are performed electrically.

 Volatile.
 Used for temporary storage only.
 If the power is disconnected, the contents become invalid.

 Two main varieties.

 Static.
 Dynamic.

12
DYNAMIC RAM (DRAM)
 Usually used for Main Memory in most computer
systems.
 Inexpensive.

 Uses only one transistor per bit.

 Data is stored as charge in capacitors.
 Destructive read.
 Charge on capacitor
is drained during a read.
 Data must be re-written

after a read.

13
DRAM – (CONTD.)
 Charge on a capacitor decays naturally.
 Therefore, DRAM needs refreshing even when powered to
maintain the data.
 Refreshing is done by reading and re-writing each word every few
milliseconds.
 Refresh Rate.
 During “suspended” operation, notebook computers use power
mainly for DRAM refresh.

14
STATIC RAM (SRAM)
 Consists of internal flip flop like structures that store the
binary information.
 No charges to leak.
 No refreshing is needed.
 Non-destructive
read.
 More complex construction.
 Larger cell, Less dense.
 More expensive.
 Faster.

 Usually used for Cache Memory.

15
SRAM VS. DRAM
 Storage cells in DRAM are simpler and smaller.
+ DRAM is more dense.
 More bits per square area.
+ DRAM is less expensive.
+ DRAM uses less power.

_ DRAM requires extra circuitry to implement refresh

mechanism.
_ DRAM is slower.

16
SRAM CHIP ORGANIZATION

17
READ ONLY MEMORY (ROM)
 Read but cannot write.
 Non volatile.
 Used for:
 Microprogramming.
 System programs.
 Whole programs in embedded systems.
 Library subroutines and function tables.
 Constants.

 Manufactured with the data wired into the chip.

 No room for mistakes.
18
ROM STRUCTURE

19
PROGRAMMABLE ROM (PROM)
 Non volatile.
 Can be programmed - written into - only once.

 Programming is done electrically and can be done after

manufacturing.
 Special equipment is needed for the programming
process.
 Uses fuses instead of diodes.
 Fuses that need to be removed are “vaporized” during the
programming process using a high voltage pulase (10 – 30 V).

 CAN NOT BE ERASED. 20

ERASABLE PROM (EPROM)
 Uses floating-gate MOS transistors with insulating material
that changes behavior when exposed to ultraviolet light.
 Programmed electrically and erased optically.
 Erasing can be repeated a relatively large but limited number of
times (~100,000 times).
 Erasing time ~20 minutes.

 Electrically read and written.

 Before writing, ALL cells must be erased by exposure to
ultraviolet light.

 Non volatile.
 More expensive than PROM. 21
ELECTRICALLY ERASABLE PROM
(EEPROM)
 Uses the same floating-gate transistors, except that the
insulating material is much thinner.
 Its operation can be inverted using voltage.
 Can be written to any time without erasing the previous
contents.
 Only the bytes addressed are modified.
 Write takes a relatively long time (~100sec/byte).
 Can be erased only about 10,000 times.
 Non volatile.
 Updatable in place.
 More expensive and less dense than EPROM.

22
FLASH MEMORY
 Called flash due to the speed of re-programming.
 Uses electrical erasure technology.
 An entire chip can be erased in 1-2 sec.
 Possible to erase only blocks of data.
 Does not provide byte level erasure.
 Uses one transistor per bit.
 Very high density.

 Cost is between EPROM and EEPROM.

 Non Volatile.

23
ORGANIZATION OF A MEMORY CHIP

 The basic element of a semiconductor memory

is the memory cell.
 There are different types, but they all share some
common properties:
 Two states, 1 and 0.
 It is possible to write into the cell. (At least once).
 They can be read to sense the state.
24
ORGANIZATION OF A MEMORY CHIP
 How to organize a-16 Mbit chip?
1 Mega words of 16 bits each.
 Tall and narrow organization.
 Chips like to be square.
 Typical organization is:
 2048 x 2048 x 4bit array.
 Organized internally as a square structure with decoders for

row and column.

 Simplifies decoding logic.

 Reduces number of address pins.

 Row and column address bits are multiplexed.

25
ORGANIZATION OF THE MEMORY CHIP

26
MEMORY MODULE ORGANIZATION
 Most high capacity RAM chips contain only a single bit
per location.
 Tobuild a multi-bit per location module, we will need
multiple chips.

 Design a 256K Byte memory system using 8 256K X 1

chips.
 256K requires 18 address wires
 We will apply 9 wires to the row selectors and 9 to the column
selectors
 The outputs of the chips are combined together to form the 8
bit output of the system. 27
ORGANIZATIO
N OF THE 256 K
BYTE SYSTEM

 Each chip receives all 18 bits

of the address.
 Each chip produces/receives
a single bit of the data.

28
MEMORY MODULE ORGANIZATION
 What if the size of the system is not the same as the
chips?

 Design a 1 MByte system using 256K X 1 chips.

 We will have to arrange the chips themselves into columns
and rows.
 There will be 4 columns of chips.
 Number of columns = system’s address space / chip’s address
space.
 There will be 8 rows of chips.

 Number of rows = system’s word size / chip’s word size.

 Some of the address wires will have to be used for selecting

different rows of chips.
29
ORGANIZATION OF THE 1 M BYTE SYSTEM

30
31
CACHE MEMORY
 Cache Memory is intended to give:
 Memory speed approaching that of the fastest memories available.
 Large memory size at the price of less expensive types of
semiconductor memories.

 Small amount of fast memory.

 Sits between normal main memory and CPU.
 May be located on CPU chip or module.

32
CONCEPTUAL OPERATION
 Relatively large and slow main memory together with faster,
smaller cache.
 Cache contains a copy of portions of main memory.
 When processor attempts to read a word from memory, a
check is made to determine if the word exists in cache.
 If it is, the word is delivered to the processor.
 If not, a block of main memory is read into the cache, then the word is
delivered to the processor.

Word Block
Transfer Transfer

CPU Cache
Memory
Main
Memory 33
HIT RATIO
 A measure of the efficiency of the cache structure.
 When the CPU refers to memory and the word is found in the
cache, this called a hit.
 When the word is not found in cache, this is called a miss.

 Hit ratio is the total number of hits divided by the total

number of access attempts (hits + misses).
 It has been shown practically that hit rations higher than 0.9
are possible.

34
CACHE VS. MAIN MEMORY STRUCTURE0
1
Tag Block 2 Block
(K words)
0
1
2
3

.
.
C-1 .

Block Length
(K Words)

Cache
Main
2n - 1
Memory

Word Length
35
MAIN MEMORY AND CACHE MEMORY
 Main Memory consists of 2n addressable words.
 Each word has a unique n-bit address.
 We can consider that main memory is made up of blocks of K
words each.
 Usually, K is about 16

 Cache consists of C lines of K words each.

 A block of main memory is copied into a line of Cache.

 The “tag” field of the line identifies which main memory block each
cache line represents

36
ELEMENTS OF CACHE DESIGN
 Size
 Mapping function

 Replacement algorithm

 Write policy

 Line size

 Number of caches

37
MAPPING FUNCTION
 There are fewer cache lines than memory blocks.
 How do we map a memory block to a cache line?

 Assume the following:

 Cache can hold 64 Kbytes.
 Data is transferred in blocks of 4 bytes.
 Cache is 16K lines of 4 bytes each.
 Main memory is 16 Mbytes.
 Memory is 4M blocks of 4 bytes each.

 How do we map the 4M blocks into the 16K lines?

38
DIRECT MAPPING
 Map each block of memory into only one possible cache
line.
 A block of main memory can only be brought into the same
line of cache every time.
Cache Line Main memory blocks assigned
0 0, C, 2C, 3C, …
1 1, C+1, 2C+1, 3C+1, …

… …

C–1 C-1, 2C-1, 3C-1, 4C-1, …

39
DIRECT MAPPING
 A main memory address is considered to be made up of
two pieces:
 Block address
 Upper bits of the address
 Word address within a block
 Lower bits of the address

 The block address section is further considered to be

made of two items:
 Cache line number
 Lower bits
 Tag 40
DIRECT MAPPING ADDRESS STRUCTURE
 16 Mbytes of memory.
 24 bits in address.

 4 byte blocks.
2 bits.
 16 K lines in cache.
 14 bits.
 Rest is used to identify the block mapped to the line.

8 14 2
Tag Line or Slot Word
41
READING FROM A DIRECT MAPPED
SYSTEM
 The processor produces a 24 bit address.
 The cache uses the middle 14 bits to identify one of its
16 K lines.
 The upper 8 bits of the address are matched to the tag
field of the cache entry.
 If they match, then the lowest order two bits of the address
are used to access the word in the cache line.
 If not, address is used to fetch the block containing the
specified word from main memory to the cache.

42
DIRECT MAPPING CACHE ORGANIZATION

43
DIRECT MAPPING
 Advantages.
 Simple.
 Inexpensive to implement.

 Disadvantages.
 There is a fixed location for each block in the cache.
 If a program addresses words from two blocks mapped to the same
line, the blocks have to be swapped in and out of cache repeatedly.

44
ASSOCIATIVE MAPPING
 To improve the hit ratio of the cache, another mapping
techniques is often utilized, “associative mapping”.

 A block of main memory may be mapped into ANY line

of the cache.
 A block of memory is no longer restricted to a single line of
cache.

45
ASSOCIATIVE MAPPING
 A main memory address is considered to be made up of
two pieces:
 Tag
 Upper bits of the address
 Word address within a block
 Lower 2 bits of the address

46
ASSOCIATIVE MAPPING ADDRESS
STRUCTURE
 16 Mbytes of memory.
 24 bits in address.

 4 byte blocks.
2 bits.
 Rest is used to identify the block mapped to the line.

22 2
Tag Word

47
READING FROM AN ASSOCIATIVE MAPPED
SYSTEM
 The processor produces a 24 bit address.
 The upper 22 bits of the address are matched to the tag
field of EACH cache entry.
 This matching must be done simultaneously to each of the
entries.
 i.e. Associative memory.

48
ASSOCIATIVE MAPPING CACHE
ORGANIZATION

49
ASSOCIATIVE MAPPING
 Advantages.
 Improves hit ratio for certain situations.

 Disadvantages.
 Requires very complicated matching hardware for matching
the tag and the entries for each line.
 Expensive.

50
SET ASSOCIATIVE MAPPING
 Set Associative Mapping helps reduce the complexity of
the matching hardware for an associative mapped cache.

 Cache is divided into a number of sets.

 Each set contains a number of lines.
 A 2-way set associative cache has 2 lines per set.

 A block of memory is restricted to a SPECIFIC set of

lines.
 A block of main memory may map to ANY line in the given
set.

51
SET ASSOCIATIVE MAPPING
 A main memory address is considered to be made up of
three pieces:
 Tag.
 Upper bits of the address.
 Set number.
 Middle bits of the address.
 Word address within a block.
 Lower 2 bits of the address.

52
SET ASSOCIATIVE MAPPING ADDRESS
STRUCTURE
 16 Mbytes of memory.
 24 bits in address.

 4 byte blocks.
 Lowest order 2 bits.
 8K sets in a 2-way associative cache.
 13 bits.
 Rest is used to identify the block mapped to the line.

9 13 2
Tag Set Word
53
READING FROM A SET ASSOCIATIVE
MAPPED SYSTEM
 The processor produces a 24 bit address.
 The cache uses the middle 13 bits to identify one of its 8
K sets.
 The upper 9 bits of the address are matched to the tag
field of the cache entries that make up the set.
 The number of lines to match to is very limited.
 Therefore, the matching hardware is much simpler.

54
SET ASSOCIATIVE MAPPING CACHE
ORGANIZATION

55
SET ASSOCIATIVE MAPPING
 Advantages.
 Combines advantages of direct and associative mapping
techniques.

 Disadvantages.
 Increasing the size of the set does not always improve the
hit ratio.
 2-way set associative has a much higher hit ratio than direct
mapping.
 Increasing it to 4-way improves the hit ratio slightly more.

 Beyond that no significant improvement has been seen.

56
REPLACEMENT ALGORITHMS
 What happens if there is a “miss” and the cache is
already full?
 One of the items in the cache needs to be “replaced” with the
new item.
 Which one??
 Depends on the mapping technique used.

 Direct mapping.
 Nochoice.
 Memory blocks map into certain cache lines.
 The entry occupying that line must be swapped out.
57
REPLACEMENT ALGORITHMS
 Associative & Set Associative:
 Random.
 First-inFirst-out (FIFO).
 Least Recently Used (LRU).
 Least Frequently Used (LFU).
 The last three require additional bits for each entry to keep track of
order, time or number of times used.

 Usually, these algorithms are implemented in hardware for

speed.

58
WRITING INTO CACHE
 Cache entries are supposed to be exact “copies” of what
is in main memory.
 Whathappens when the CPU wants to write into memory??
 Which memory does it write too???

 Two techniques are possible.

 Write-through.
 Write-back.

59
WRITE-THROUGH
 The simplest and most commonly used technique is to
update both the cache and main memory at the same
time.

 Advantage.
 Memory and cache are always in sync.

 Disadvantage.
 Memory write becomes slow.

60
WRITE-BACK
 The update is done ONLY to the word in the cache and
the block containing the word is marked.
 When the block is to be swapped out of cache, the word
is written back to main memory.

 Advantage.
 Reduces memory traffic because a word may be updated
several times while in cache.
 Disadvantage.
 Cache and memory will be out of sync for a while.
 What about DMA??

61
NUMBER OF CACHES
 When a cache miss occurs, the system suffers through a
large delay while the block is read from main memory into
the cache.

 Two possible solutions.

 Speed up the transfer of information.
 The transfer rate is limited by issues that may not be under our control.

 Speed up the source of the information.

 Main memory is between 7X and 10X slower than cache.

 We can insert an intermediate level of memory between cache and

main memory.
62
CACHE LEVELS
 In most of today’s designs, cache sits on the same chip as
the CPU. “On-chip cache”
 Data travels a very short distance
 No need to use the very slow bus
 This is known as L1 cache
 Intel calls this level L0

 To reduce the penalty of a cache miss, a second level of

cache is inserted between main memory and the on-chip
cache.
 L2 cache
63
CACHE LEVELS
Memory System
Data
On Bus Bus
Off-Chip
CPU Chip
Bus Cache Cache Main
Memory
MPU Chip

Pentium Pentium Pro

64
“L2” CACHE
 A very fast, SRAM based, cache is placed off-chip.
 Slower than the on-chip cache.
 Larger than the on-chip cache.

 On-Module Cache.
 CPU uses a dedicated, internal, fast, memory bus to access cache.
 On-Mother-Board Cache.
 The CPU has to use the system bus to get to it.
 Still much faster than DRAM based main memory.

65
CACHE STRATEGY
 On-Chip Cache is optimized to increase “hit rate”.
 Blocksize about 4 words
 Many blocks

 Off-Chip Cache is optimized to reduce “miss penalty”.

 Largerblock size
 Smaller number of blocks.

66
SECONDARY MEMORY
TYPES OF EXTERNAL MEMORY
 Magnetic Disk
 RAID
 Removable

 Optical
 CD-ROM
 CD-Recordable (CD-R)
 CD-R/W
 DVD

 Magnetic Tape

68
MAGNETIC DISK
 Disk substrate coated with magnetizable material
(iron oxide…rust)
 Substrate used to be aluminium

 Now glass
 Improved surface uniformity
 Increases reliability
 Reduction in surface defects
 Reduced read/write errors
 Lower flight heights (See later)
 Better stiffness
 Better shock/damage resistance
69
HOW HARD DRIVE WORKS?
 See video

70
READ AND WRITE MECHANISMS
 Recording & retrieval via conductive coil called a
head
 May be single read/write head or separate ones
 During read/write, head is stationary, platter rotates
 Write
 Current through coil produces magnetic field
 Pulses sent to head
 Magnetic pattern recorded on surface below

71
READ AND WRITE MECHANISMS CONT..
 Read (traditional)
 Magnetic field moving relative to coil produces current
 Coil is the same for read and write
 Read (contemporary)
 Separate read head, close to write head
 Partially shielded magneto resistive (MR) sensor
 Electrical resistance depends on direction of magnetic
field
 High frequency operation
 Higher storage density and speed
72
INDUCTIVE WRITE MR READ

73
DATA ORGANIZATION AND
FORMATTING
 Concentric rings or tracks
 Gaps between tracks
 Reduce gap to increase capacity
 Same number of bits per track (variable packing density)
 Constant angular velocity

 Tracks divided into sectors

 Minimum block size is one sector

 May have more than one sector per block

74
DISK DATA LAYOUT

75
DISK VELOCITY
 Bit near centre of rotating disk passes fixed point slower than
bit on outside of disk
 Increase spacing between bits in different tracks

 Rotate disk at constant angular velocity (CAV)

 Gives pie shaped sectors and concentric tracks
 Individual tracks and sectors addressable
 Move head to given track and wait for given sector
 Waste of space on outer tracks
 Lower data density
 Can use zones to increase capacity
 Each zone has fixed bits per track
 More complex circuitry 76
DISK LAYOUT METHODS DIAGRAM

77
FINDING SECTORS
 Must be able to identify start of track and sector
 Format disk
 Additionalinformation not available to user
 Marks tracks and sectors

78
WINCHESTER DISK FORMAT
SEAGATE ST506

79
CHARACTERISTICS
 Fixed (rare) or movable head
 Removable or fixed

 Single or double (usually) sided

 Single or multiple platter

 Head mechanism
 Contact (Floppy)
 Fixed gap
 Flying (Winchester)

80
FIXED/MOVABLE HEAD DISK
 Fixed head
 One read write head per track
 Heads mounted on fixed ridged arm

 Movable head
 Oneread write head per side
 Mounted on a movable arm

81
REMOVABLE OR NOT
 Removable disk
 Can be removed from drive and replaced with another disk
 Provides unlimited storage capacity
 Easy data transfer between systems

 Nonremovable disk
 Permanently mounted in the drive

82
MULTIPLE PLATTER
 One head per side
 Heads are joined and aligned

 Aligned tracks on each platter form cylinders

 Data is striped by cylinder

 reduces head movement
 Increases speed (transfer rate)

83
MULTIPLE PLATTERS

84
TRACKS AND CYLINDERS

85
FLOPPY DISK
 8”, 5.25”, 3.5”
 Small capacity
 Up to 1.44Mbyte (2.88M never popular)
 Slow
 Universal

 Cheap

 Obsolete?

86
ASSIGNMENT
 Design a bus system for Eight Registers.
 Sub. date
 Section 3: Dec. 23, 2019
 Section 2: Dec. 26, 2019

87