Question

I’m constantly hearing about person y had performance issue x which they solved through caching.

Or, how doing x,y,z in your programs code can hurt your caching ability.

Even in one of the latest podcasts, Jeff Atwood talks about how they cache certain values for speedy retrieval.

There seems to be some ambiguity in the terms "cache" and "caching" and it has led me to be confused about it’s meaning in different cases. Whether you are referring to application or database caching, cpu, etc and what that means.

What is caching and what are the different types?

From context I can get a sense of it, to store an oft retrieved value into main memory and have quicklook up access to it. However, what is it really?

This word seems to be used in a lot of different contexts with slightly different meaning (cpu, database, application, etc) and I’m really looking to clear it up.

Is there a distinction between how caching works in your applications vs your database caching?

When someone says that they found a piece of code that would hurt caching and after they fixed it, it improved the speed of their app, what are they talking about?

Is the program’s caching something that is done automatically? How do you allow values to be cached in your programs? I’ve often read users on this site say that they cached a value in their application, I sit here and wonder what they mean.

Also, what does it really mean when someone talks about database caching? Is this simply a feature they turn on in their database? Do you have to explicitly cache values or does the database pick which ones to cache for you?

How do I begin caching items myself to improve performance?

Can you give me some examples of how I can begin caching values in my applications? Or again, is this something that is already done, under the hood and I simply have to write my code in a particular way to allow "caching"?

What about database caching, how do I begin that? I’ve heard about things like memcache. Is this type of utility required to cache in databases?

I’m looking to get a good distinction between caching in applications vs databases, how they are used and how it is implemented in both cases.

Answer 1

Caching is just the practice of storing data in and retrieving data from a high-performance store (usually memory) either explicitly or implicitly.

Let me explain. Memory is faster to access than a file, a remote URL (usually), a database or any other external store of information you like. So if the act of using one of those external resources is significant then you may benefit from caching to increase performance.

Knuth once said that premature optimization is the root of all evil. Well, premature caching is the root of all headaches as far as I’m concerned. Don’t solve a problem until you have a problem. Every decision you make comes at a cost that you’ll pay to implement it now and pay again to change it later so the longer you can put off making a deicsion and changing your system the better.

So first identify that you actually have a problem and where it is. Profiling, logging and other forms of performance testing will help you here. I can’t stress enough how important this step is. The number of times I’ve seen people ‘optimize’ something that isn’t a problem is staggering.

Ok, so you have a performance problem. Say your pages are running a query that takes a long time. If it’s a read then you have a number of options:

• Run the query as a separate process and put the result into a cache. All pages simply access the cache. You can update the cached version as often as is appropriate (once a day, once a week, one every 5 seconds, whatever is appropriate);
• Cache transparently through your persistence provider, ORM or whatever. Of course this depends on what technology you’re using. Hibernate and Ibatis for example support query result caching;
• Have your pages run the query if the result isn’t in the cache (or it’s ‘stale’, meaning it is calculated longer ago than the specified ‘age’) and put it into the cache. This has concurrency problems if two (or more) separate processes all decide they need to update the result so you end up running the same (expensive) query eight times at once. You can handle this locking the cache but that creates another performance problem. You can also fall back to concurrency methods in your language (eg Java 5 concurrency APIs).

If it’s an update (or updates take place that need to be reflected in your read cache) then it’s a little more complicated because it’s no good having an old value in the cache and a newer value in the database such that you then provide your pages with an inconsistent view of the data. But broadly speaking there are four approaches to this:

• Update the cache and then queue a request to update the relevant store;
• Write through caching: the cache provider may provide a mechanism to persist the update and block the caller until that change is made; and
• Write-behind caching: same as write-through caching but it doesn’t block the caller. The update happens asynchronously and separately; and
• Persistence as a Service models: this assumes your caching mechanism supports some kind of observability (ie cache event listeners). Basically an entirely separate process–unknown to the caller–listens for cache updates and persists them as necessary.

Which of the above methodologies you choose will depend a lot on your requirements, what technologies you’re using and a whole host of other factors (eg is clustering and failover support required?).

It’s hard to be more specific than that and give you guidance on what to do without knowing much more detail about your problem (like whether or not you have a problem).