Learning Objectives
By the end of this chapter, you will be able to:
- Define the computer memory hierarchy.
- Describe the different levels of the memory hierarchy (registers, cache, RAM, storage).
- Explain the trade-off between speed, cost, and capacity at each level.
- Differentiate between volatile and non-volatile memory.
- Differentiate between RAM and ROM, and describe their types.
The Memory Hierarchy
Computers use a variety of different types of memory, each with its own characteristics. The memory hierarchy is a way of organizing these different types of memory based on their speed, capacity, and cost. The general principle is that the faster the memory, the more expensive it is per byte, and therefore the smaller its capacity.
The hierarchy is typically visualized as a pyramid:
- Top of the Pyramid: Fastest, most expensive, smallest capacity.
- Bottom of the Pyramid: Slowest, cheapest, largest capacity.
flowchart TB
R[Registers: fastest, smallest] --> C[Cache L1, L2, L3]
C --> M[Main memory RAM]
M --> S[Secondary storage: SSD, HDD]
S --> T[Tertiary and offline: tape, cloud archives]
Levels of the Memory Hierarchy
- Registers
- Location: Inside the CPU.
- Description: The fastest and smallest form of memory. Registers are used by the CPU to hold the data it is currently processing.
- Cache Memory
- Location: On or very close to the CPU.
- Description: A small, very fast type of memory that is used to store frequently accessed data and instructions from the main memory. By keeping this data close to the CPU, cache memory reduces the time the CPU has to wait for data from the slower RAM.
- Levels: L1 (smallest and fastest, inside each core), L2 (larger, slightly slower), and L3 (shared by all cores).
- Primary Memory (RAM)
- Location: On the motherboard.
- Description: Random Access Memory (RAM) is the computer’s main working memory. It is where the operating system, applications, and the data currently in use are kept so they can be quickly reached by the CPU. RAM is volatile, meaning its contents are lost when the computer is powered off.
- Secondary Storage
- Location: Separate devices.
- Description: Secondary storage is used for long-term, persistent storage of data and programs. It is much slower than RAM but has a much larger capacity and is non-volatile, meaning it retains its data even when the power is off.
- Examples: Hard Disk Drives (HDDs), Solid-State Drives (SSDs), and external USB drives.
Comparison of levels
| Level | Typical size | Relative speed | Cost per GB | Volatile? |
|---|---|---|---|---|
| Registers | Bytes | Fastest | Highest | Yes |
| Cache | KB to tens of MB | Very fast | Very high | Yes |
| RAM | 8–64 GB | Fast | Moderate | Yes |
| SSD | 256 GB–4 TB | Slower than RAM | Low | No |
| HDD | 1–20 TB | Slower still | Lowest | No |
Analogy: Think of studying for an exam. The fact you are thinking about right now is in your head (registers); the notes open on your desk are cache; the books on your shelf are RAM; and the library across town is secondary storage. You keep what you need most close at hand.
Primary Memory: RAM and ROM
| Aspect | RAM (Random Access Memory) | ROM (Read-Only Memory) |
|---|---|---|
| Volatility | Volatile — lost when power is off | Non-volatile — kept without power |
| Read/write | Read and write | Mainly read; written by the manufacturer |
| Purpose | Holds running programs and current data | Holds firmware such as BIOS/UEFI startup instructions |
| Size | Large (GBs) | Small |
| Speed | Fast | Slower than RAM |
Types of RAM
- DRAM (Dynamic RAM): Used for main memory; must be refreshed thousands of times per second; cheaper and denser. Modern PCs use DDR4 or DDR5 DRAM.
- SRAM (Static RAM): Used for cache; faster and does not need refreshing, but more expensive and larger per bit.
Types of ROM
- PROM: Programmable once by the user.
- EPROM: Erasable with ultraviolet light and reprogrammable.
- EEPROM / Flash: Erasable electrically; used for BIOS/UEFI chips and in USB drives and SSDs. This is why firmware can be updated.
Why the Hierarchy Works: Locality
Programs tend to reuse the same data and instructions repeatedly (temporal locality) and access data stored next to each other (spatial locality), such as items in a list. Because of this, keeping a small amount of recently used data in fast cache gives most of the speed of fast memory at a fraction of the cost.
Summary
The computer memory hierarchy organizes memory types based on a trade-off between speed, cost, and capacity. At the top are the fastest, smallest, and most expensive types of memory—CPU registers and multi-level cache—which hold the data the CPU is actively using. In the middle is primary memory (RAM), the computer’s main workspace, which is volatile, alongside non-volatile ROM that holds startup firmware. At the bottom is secondary storage (HDDs, SSDs), which is the slowest, cheapest, and largest, providing non-volatile, long-term storage for all of the computer’s files and applications. Locality of reference is what makes this layered design efficient.
Key Takeaways
- The memory hierarchy is a trade-off between speed, cost, and capacity.
- The levels are: Registers > Cache > RAM > Secondary Storage.
- RAM is volatile (loses data when powered off), while secondary storage and ROM are non-volatile.
- Cache memory is used to bridge the speed gap between the very fast CPU and the slower RAM.
- DRAM is used for main memory; SRAM is used for cache.
Discussion Questions
- Why don’t we just build computers with a very large amount of the fastest type of memory (registers)?
- What is the purpose of cache memory?
- What is the difference between primary memory (RAM) and secondary storage?
Exam-Style Questions
Short answer
- What is the memory hierarchy?
- Differentiate between RAM and ROM.
- Differentiate between SRAM and DRAM.
Long answer
- Explain the memory hierarchy with a diagram, describing each level.
- Explain the types of primary memory and their uses.
