On most modern computers, the CPU runs at a faster speed than the rest of the computer. This is mainly due to the amount of time it takes for an electrical signal to travel across the motherboard. The CPU is small, so signals take less time for signals to travel, so it can be clocked at a faster rate.
Because of this, each time the CPU needs to access RAM memory (which is located on the motherboard), it has to wait a few cycles until the data has been read or written. This wastes time and slows the processor down.
Cache memory is a small amount of very fast memory which is usually located on the CPU chip (or very close to it on the motherboard). The CPU can access cache memory more quickly than normal RAM, but there is much less of it.
Cache memory is used to store both instructions and data, so the can be accessed faster by the CPU.
The system tries to keep the most frequently used data in the cache. It uses an algorithm to decide which data should go in the cache. Algorithms can be quite complex, but here are some simple techniques:
- The most recently executed instructions might be held in the cache, in case the processor loops back and needs to execute them again.
- Data which has been accessed frequently might be held in the cache, as it is likely to be accessed again.
- Instruction and data might be pre-fetched. This means values are fetched from memory if the system thinks it might be used next, even if the CPU hasn't requested it yet.
Levels of cache
Often the cache memory is arranged in levels:
- the L1 cache contains a small amount of very fast memory
- the L2 cache contains a larger amount of slightly slower memory
- the L3 cache contains an even larger amount of even slower memory
Some systems have more levels. Even the slowest cache memory is faster than the system RAM, otherwise there would be no point using it!
Here is an example of a typical arrangement of cache memory, but it can vary a lot between systems:
In this case, each core has its own block of very fast L1 cache memory, on the CPU chip. The L2 cache is also on the CPU chips, but it is slightly slower and is shared by all the cores. The L3 cache is not on the CPU chip, it is on the motherboard but very close to the CPU - it is slower than he L1 and L2 cache memory, but faster than the normal RAM.