CPU Cache is high-speed, small-capacity static RAM (SRAM) located directly on or very near the CPU die, designed to store frequently accessed data and instructions to minimize latency.
In modern computing, the CPU is significantly faster than the RAM. If the CPU had to wait for RAM every time it needed to read or write a variable, it would spend roughly 95% of its time doing nothing.
Caches are organized in levels (L1, L2, L3). As you move from L1 to L3, the storage capacity increases, but the speed decreases.
| Memory Level | Access Latency (Approx.) | Analogy (Time) |
|---|---|---|
| CPU Register | 1 cycle | 1 second |
| L1 Cache | ~4 cycles | 4 seconds |
| L2 Cache | ~12 cycles | 12 seconds |
| L3 Cache | ~40 cycles | 40 seconds |
| Main RAM | ~200 cycles | 3-4 minutes |
L1 Cache (Primary Cache)
- Location: Built inside each individual CPU core.
- Access Speed: Fastest (~1 nanosecond / 4 cycles).
- Size: Very small (typically 32KB to 96KB per core).
- Role: The immediate “workspace.” It is usually split into L1i (Instructions) and L1d (Data) so the CPU can fetch code and data simultaneously without interference.
L2 Cache (Secondary Cache)
- Location: Also usually private to each core.
- Access Speed: Fast, but slower than L1 (~3-5 nanoseconds / 12 cycles).
- Size: Moderate (typically 256KB to 1MB per core).
- Role: Acts as a feeder for the L1. If the CPU doesn’t find what it needs in L1 (a “cache miss”), it checks L2.
L3 Cache (Last Level Cache - LLC)
- Location: Shared across all cores on a single CPU chip.
- Access Speed: Significantly slower than L1/L2 (~15-20 nanoseconds / 40+ cycles).
- Size: Large (anywhere from 2MB to 768MB on high-end EPYC/Xeon chips).
- Role: Its primary job is to prevent the CPU from having to go all the way to the System RAM. Since it is shared, it also helps cores communicate with each other by sharing data in a common pool.