QuickStudy: Cache Memory

Computerworld - Computers store data using a similar hierarchy. When applications start, data and instructions are moved from the slow hard disk into main memory (dynamic RAM, or DRAM), where the CPU can get them more quickly. DRAM acts as a cache for the disk.

Levels Upon Levels

Although DRAM is faster than the disk, it's still pokey. So data that's needed more often is moved up to the next faster memory, called the Level 2 (L2) cache. This may be located on a separate, high-speed static RAM chip next to the CPU, but new CPUs usually incorporate the L2 cache directly on the processor chip.

Intel Corp.'s Pentium III processor has 32KB of L1 cache on the processor chip and either 256KB of L2 on-chip or 512KB of L2 off-chip. The L2 cache on the CPU chip can be accessed four times faster than if it were on a separate chip.

When the processor needs to execute an instruction, it looks first in its own data registers. If the needed data isn't there, it goes to the L1 cache and then to the L2 cache. If the data isn't in any cache, the CPU calls out to the main RAM. It might not even be there, in which case the system has to retrieve it from the disk.

When the CPU finds data in one of its cache locations, it's called a "hit"; failure to find it is a "miss." Every miss introduces a delay, or latency, as the processor tries a slower level. In a well-designed system with software algorithms that prefetch data before it's requested, the hit rate can reach 90%.

For high-end processors, it can take from one to three clock cycles to fetch information from L1, while the CPU waits and does nothing. It takes six to 12 cycles to get data from an L2 on the processor chip, and dozens or even hundreds of cycles for off-CPU L2.

Caches are more important in servers than in desktop PCs because servers have so much traffic between processor and memory generated by client transactions. Intel turned a 50-MHz, 80486-based PC into a server in 1991 by adding a 50-MHz cache to the processor chip. Although the bus connecting processor and memory ran only at 25 MHz, this cache let many programs run entirely within the 486 chip at 50 MHz.