CPU cache

CPU cache is especially fast memory located on or near the central processing unit (CPU) of a computer that speeds up performance by storing important data to which the CPU needs access quickly and frequently. By providing a means of storing this vital data in an easy-to-reach place, the CPU cache minimizes processing errors and helps a computer run as rapidly and smoothly as possible. Originally, the CPU cache was developed in response to computing speed problems that arose as microprocessors' clock speeds began to significantly outpace memory access times as new computers were being manufactured in the 1980s. Today, many CPUs have several layers of cache storage. Typically, these levels are denoted as L1, L2, L3, and sometimes even L4. Moving forward, researchers continue to seek new ways to improve performance by experimenting with the CPU cache to make it as efficient as possible.

Technical Background

The basic operation cycle of a computer is referred to as an instruction cycle, or simply a cycle. Specifically, this cycle is the process by which a computer retrieves a program's instruction from memory, determines the action this instruction requires, and carries out that action. Whenever a computer is running, the CPU repeats the instruction cycle continuously.

In practice, the CPU can only successfully complete a cycle as quickly as it can be fed data. When the CPU is unable to access the data it needs to complete a calculation on a given cycle, that cycle is missed and the calculation will not be completed until the CPU is able to find the necessary data in the computer's memory in a subsequent cycle. Although an occasional missed cycle may have little or no perceivable effect on a computer's overall performance, repeated misses can dramatically reduce its speed.

For early computers, missed cycles were rarely a serious problem because while the speed of their CPUs was limited, their main memory was exceedingly slow and expensive. As a result, the overall performance speed of these machines was generally not an issue. When CPUs began to get faster in the 1980s, however, the growing disparity between memory speed and CPU speed became increasingly problematic. To address this issue, computer engineers introduced the concept of CPU cache, which improved overall performance by providing the CPU with a small, built-in memory bank in which it could store the vital data to which it needed quick access most often. This advance bridged the gap between memory speed and CPU speed and made it possible for modern computers to take full advantage of their improved processors.

How It Works

CPU cache is a special type of memory that is housed on the same die, or integrated circuit, as the processor itself and serves as a locator for the CPU to store the important information to which it needs access most frequently. Because CPU cache memory is much faster than traditional random access memory (RAM) and runs at a refresh rate that is significantly closer to the CPU's clock speed, the CPU can access key data with relatively little risk of missed cycles.

Modern computers have several levels of CPU cache. Level one cache, which is typically denoted as L1, is the fastest cache level and the only one to work directly with the CPU. While the L1 cache is always located on the processor die itself, the level two (L2) cache and level three (L3) cache are sometimes located on a separate set of motherboard chips. Some modern machines are even beginning to incorporate a level four (L4) level of cache, which is also located off the processor die. When in use, all the levels feed down to L1 to improve performance. Ultimately, the L1 cache is designed to align with the CPU so that the CPU has access to the data it needs to complete its tasks on every cycle. The L2 and L3 caches feed additional data to the L1 cache as necessary. Data found in the L1 cache is also duplicated in the L2 cache. When the CPU receives a request for a certain set of data in a cycle, it searches through the L1 and L2 caches. If the data is not found in either of those locations, the CPU moves on to the L3 cache, the L4 cache if one exists, and finally to system RAM when necessary. This process ultimately helps the CPU to operate at maximum efficiency and reduce the risk of missed cycles as much as possible. Proof of the effectiveness of CPU cache can be seen in its efficiency rate as compared to that of system RAM. Whereas system RAM might only be able to feed data to the CPU once in every fifty cycles at a rate of 2 percent efficiency, cache memory delivers data to the CPU at a rate of up to 98 percent efficiency.

The Future of CPU Cache

Computer engineers continue to look for new ways to improve system performance by experimenting with cache structure and design. The relatively recent introduction of the L4 cache suggests the development and implementation of additional and even larger caches may take place in the future. Whatever direction CPU cache takes in the years to come, its design is sure to be essential to the performance potential of new processors and the computers they run.

Bibliography

Genua, Paul. "A Cache Primer." Freescale Semiconductor. Freescale Semiconductor, Inc. 2004. Web. 22 Feb. 2016. http://www.nxp.com/files/32bit/doc/app‗note/AN2663.pdf

Hruska, Joel. "How L1 and L2 CPU Caches Work, and Why They're an Essential Part of Modern Chips." ExtremeTech. Ziff Davis, LLC. 17 Feb. 2016. Web. 22 Feb. 2016. http://www.extremetech.com/extreme/188776-how-l1-and-l2-cpu-caches-work-and-why-theyre-an-essential-part-of-modern-chips

Morley, Deborah and Charles S. Parker. Understanding Computers: Today and Tomorrow. 15th ed. Stamford: Cengage Learning, 2015. Print.

Stone, Dan. "How Important Is a Processor Cache?" Houston Chronicle. Hearst Newspapers, LLC. Web. 22 Feb. 2016. http://smallbusiness.chron.com/important-processor-cache-69692.html