|
||
 |
||
|
| Cache As Cache Can |
 Email This
 Print ThisView My Personal Library |
|
Hardware October 2002 • Vol.6 Issue 8 Page(s) 13-18 in print issue |
Cache As Cache Can How Do Caches Work? | ||
 Caching is one of the most impressive feats of small-scale engineering, providing speed and efficiency gains at all levels. Caching strategies are some of the more interesting challenges in computer science. And caches save companies millions of dollars a year. Yet for all that, the poor little cache may be the most misunderstood (and perhaps most mispronounced) strategy in computing. A cache (sounds like "cash") is all about saving time. And in PCs, as elsewhere, time is money. It's important to understand what caches do, how the various types of caches work, and how you can make use of caching technology to make your PC run better. We'll help you get the most flash for your cache and put your system's performance back in the black.  The Nuts & Bolts. You first need to understand caching as a concept before delving into the workings of the various caches themselves. There are many different types of caches, and we'll describe their particulars here, but the general practice is similar in all of them. Essentially, a (relatively fast) cache temporarily stores data away from its (relatively slow) source in order to make it more quickly available. For example, your computer's CPU is several times faster than its RAM (also called system memory). If the processor had to go directly to memory each time it needed data, you would end up with quite a lot of idle processor time. In order to avoid that waste, your PC uses a memory cache to store the processor's most frequently requested data so that it's more readily and quickly available. Think of a cache as a tool drawer. You probably store tools at home in several different locations. The tools you use infrequently are probably in one location, like the garage. And tools you use just occasionally may be stored in the back of a utility closet. But you probably keep a couple screwdrivers and maybe a hammer close by because you use them all the time. In some ways, it might make sense to just keep all your tools in one place. But it wouldn't be practical (or affordable) to keep everything close by, so you'd have to run all the way out to the garage every time you needed a screwdriver. Keeping your most frequently used tools nearby just makes you more efficient. That's the idea behind caching. Characteristics of caches. Different types of caches also share more practical attributes. A cache is always smaller, but also much faster, than its source. The need for greater speed is obvious, but why not just make caches larger or, better yet, build all of your components to run at processor speed? First, building every component to operate at top speed would be horrendously costly. The faster something works, the more expensive it is to build. Part of caching, and much of computer science in general, is the art of making trade-offs between price and performance. That's why you pay more for a processor with a larger cache. Second, several important principles allow caches to operate with a fraction of a source's size. For example, locality of reference refers to an application's tendency to loop, or run the same set of instructions repeatedly. Caches take advantage of that tendency by storing instructions they are most likely to need again. Cache policy is the strategy of identifying those instructions. An effective cache policy can take advantage of the locality of reference to make a small cache deliver a high percentage of requested data without having to go back to the source each time. Another shared characteristic of caches is the importance of specialization and proximity to the CPU. The more specialized the component (for fast access and temporary storage), the better it serves as a cache. And the closer a component is to the processor, the faster it can serve up information. For example, an L1 (level 1, or primary) memory cache is built specifically for caching and is located directly on the CPU chip. It is a more effective cache than memory (built for temporary storage but not necessarily explicitly for caching), which is in turn more effective than the hard drive (located even farther from the CPU than RAM and built for permanent storage). Essentially, the farther a component gets from the processor, the lower it falls in the chain of caching.
Types Of Caches. The most common cache types include memory caches, disk caches, cache buffers, browser caches, and server caches. There are others, though, including specialized caches for dual-CPU servers that help these advanced machines share information between processors. Many standard PC components often act as caches: RAM and hard drives, for example, store information from slower sources. Think of these caches as a layered hierarchy of size and speed. As you work back from the processor, each cache gets larger and slower, though always remaining smaller and faster than the source a step farther down the line. The memory cache (usually built into the processor) caches data from system memory, memory caches information from the hard drive or other media, the hard drive caches files from remote Web servers (this is the browser cache), and so on. Memory cache. The memory cache (also called cache memory) is the most explicitly specialized form of caching. Performed by expensive and dedicated hardware components, memory caching is extremely fast and highly efficient. Its purpose is to store the common and often-repeated operations the processor spends most of its time performing. Those instructions can then be pulled directly from the cache rather than having to wait for slower system memory to provide them. For example, in the Windows GUI (graphical user interface), you spend quite a lot of time moving your mouse around. There is a particular set of instructions for translating input data from the mouse (your hand moves left) through the processor, resulting in screen output (the pointer moves left). Imagine if your machine had to go back to memory or, worse yet, to Windows' source code (stored on the hard drive) every time you moved your mouse. The memory cache lets the CPU keep those mouse instructions handy. There are two types of cache memory: L1 and L2 (level two, or secondary). The L1 cache normally operates at the same speed as the processor and is built right into the CPU. It is generally small (the Pentium 4's L1 cache is only 8KB) but, due to its speed and proximity, very effective. L2 cache is sometimes physically "on chip" (though running apart from CPU operations) and other times installed separately on the system's motherboard. L2 cache is generally quite a bit larger than L1 cache (most new systems have at least 256KB of L2 cache) and is the most commonly cited type of cache when measuring PC performance. Some lower-end processors (Intel's Celeron chip, for example) skip the L2 cache altogether, sacrificing performance for a lower price point. Cache memory consists of the data store (the actual area where data is housed, measured as the size of the cache), tag RAM (a record of all the entries in the data store), and the physical circuitry of the components. The size of the L2 cache is probably the most important caching statistic when comparing systems. All other things being equal, a larger L2 cache will be able to hold more instructions for the CPU and thus provide a greater percentage of cache hits (or requests fulfilled by the cache). In the end, that translates to faster performance. L2 distance from the CPU and L1 size also matter, though, so don't ignore them in the search for a bigger L2 cache. Recently, some manufacturers have begun to locate L3 caches off-chip; these are still much faster than memory. In short, buy as much memory cache as you can afford. Disk cache. Disk caching works on much the same principle as memory caching. The CPU caches data in system memory and memory, in turn, caches information from the hard drive. This is so that information not stored in cache memory (called a cache miss) can be still be accessed relatively quickly. The disk cache provides a larger storage area than can be economically provided to the L1 and L2 caches, while still preserving some of the speed benefits that system memory provides. Though RAM serves other purposes, as well, disk caching is one of its major responsibilities. In fact, disk caches often provide more bang for your buck than other types of caching due to the large speed differential between the cache (in RAM) and the source (the hard drive or other storage media). Though the memory cache may be several times faster than RAM, the disk cache is often hundreds of times faster than hard drives and CD-ROM drives. This creates dramatic increases in the speed and efficiency of PC operations. Disk caches operate very similarly to memory caches. Data from the source drive is stored in dedicated portions of system memory, readily accessible to the memory cache and processor should they be required again. The best way to take advantage of disk caching is to make sure that you have as much RAM as your system can hold. Memory is relatively inexpensive and is an easy upgrade. Obviously, the speed of your system's memory also affects the speed of the disk cache. Check with your motherboard or system manufacturer to find the top speed at which your memory will run and make sure you purchase memory that operates at that speed. When purchasing or upgrading a system, get a motherboard that will accept today's fastest memory modules (DDR [double data rate] RAM or Rambus) and don't skimp on the system memory. Cache Buffers. Cache buffers are another step removed from the processor, operating as an integrated part of a hardware component. Although access times to the cache buffer are the same as to the media itself, cache buffers improve internal response time by letting the device retrieve and send data back to memory much more quickly. Many hard drives, removable media drives (such as CD-ROM and DVD-ROM drives), and other components (such as graphics and audio adapters) contain specialized built-in memory that works with the motherboard's controllers to pass cached data directly to memory (and the disk cache) without accessing the media. Though cache buffers are slower than system memory (transfer rates through these controllers are slower than accessing the RAM installed directly on the motherboard), they store frequently used data for quicker retrieval and limit the need to access the (very slow) media directly. When shopping for components, check for the existence and size of onboard memory or cache buffers. The more onboard memory a device has, the more data it can store for quick and easy access. Browser Cache. Browser caches share some of the same principles as other types of caches but operate somewhat differently. The browser cache's purpose is to improve Internet browsing by speeding up page loading while conserving bandwidth and minimizing network traffic. Browser caches make use of a system's hard drive to store Web objects the browser downloads. Each element (buttons, images, scripts, files) of the Web pages you visit is stored in a cache on your hard drive. Then, when you reload a page or when you navigate to a new page that reuses some of the same elements (navigation buttons or company logos, for example), those files are loaded from the cache rather than downloaded all over again. This saves time; you can load files from your hard drive many times faster than you can download them across the fastest Internet connection. The browser also saves your system work and cuts down on network congestion. It is much easier to load files locally than to send and receive network requests for data. Some Web elements, though, shouldn't be cached. Real-time news sites, stock quotes, and online discussion forums are some obvious examples of content where "freshness" matters. Webmasters can exercise some control over browser caching in their code and in their Web server's HTTP (Hypertext Transfer Protocol) headers by setting rules for the cache and by "expiring" pages. In addition, you can control some of your cache's behavior from your browser settings. In Internet Explorer, the browser cache is better known as Temporary Internet Files. From the Tools menu, select Internet Options. Select Settings under Temporary Internet Files. Choose among several options for how often the browser checks the server for newer versions of cached files. Strike a balance between speed and current information by checking either Every Visit or Automatically. Access the same options in Netscape by clicking Edit, choosing Preferences, expanding the Advanced menu, and then selecting Cache. Don't be confused by Netscape's terminology; a memory cache and a disk cache are actually just two levels of browser cache, one stored in system memory and the other on the hard drive. Both, however, work much like IE's Temporary Internet Files. You can also control the size of the browser cache. This is more important in the browser cache than in other caches. Because the browser cache stores files on the hard drive, it can be very large. Normally, a larger cache is a good thing. But there is such a thing as too much cache, because in order to pull data from a cache, that data first has to be searched. And if your cache gets too big, it could theoretically take longer to search for the data than it would to retrieve it from the source in the first place. This only becomes a practical concern when dealing with browser caches, but it can be an issue. And Windows automatically sets your cache size as a percentage of your hard drive space, which doesn't help matters if you have a very large hard drive. When those hundreds of megabytes of cache fill up, browser performance will degrade noticeably. The solution is to empty the cache (clear Temporary Internet Files) and reduce the size to a more reasonable level. For most users, a 64MB browser cache is plenty. Proxy Cache. A proxy cache (also sometimes called a cache server) is very similar to a browser cache. Proxy caches collect and store Web elements requested by a group of users, all of whom connect to the Internet through the proxy. This essentially pools the benefits of a browser cache (quicker downloads, less network traffic, less load on Web servers) among a group of users. A cache server sits between a group of users and the public Internet. As each user requests pages and Web objects, the cache stores those objects on its hard drive. When the proxy cache receives another request for those same elements (either from the same or a different user), it serves them up without having to forward the request to the Web server. Normally, the cache server is much closer to users and accessed across a much faster network than the Internet, so pages download more quickly, the organization uses less bandwidth, and the Web server's traffic load lessens. Everyone benefits. Proxy caches have the added benefit of shared resources: You get the file more quickly because the woman in the next cubicle has already visited the Web site. Cache servers deal with the same expiration and page freshness issues as browser caches, though as dedicated machines they have more flexibility. Again, Webmasters can control some caching behavior in their code and server setup. Proxy cache policy can also be set to define what files should be stored, for how long, and how often they should be compared to files on the Web server. Some caches check with the server for updates on every request (increasing freshness but decreasing speed), some check only when a page has "expired" (determined by the Webmaster or a default cache setting), and others check on a regular schedule (providing more cache hits but risking fewer fresh pages). Most of the time, individual users don't have a choice about whether to use a proxy cache: Most organizations force the use of a proxy if one is present, and administrative staff control settings for caching at the proxy. If you have a choice, though, use the proxy. And if you're creating Web pages, account for cache servers by building expirations into pages that require freshness. More Speed & Efficiency. Caching of every stripe enhances the speed and efficiency of computing operations. From tiny dedicated L1 memory caches to massive and remote cache servers, all caches work on the same basic principle: Store important information where it can be accessed more quickly. If you understand that concept and how it is applied to each type of cache, you'll be a savvier user and a wiser shopper. Which, of course, will save you time and money.  by Gregory Anderson View the graphics that accompany this article. How Caches Work Storage & Caches (NOTE: These pages are PDF (Portable Document Format) files. You will need Adobe Acrobat to view these pages. Download Adobe Acrobat Reader )
|
|
Home Copyright & Legal Information Privacy Policy Site Map Contact Us