Question

I am using mysql as a db.

Is there any formula for determining the cache size needed to cache the results?

Thanks

Answer 1

This is usually a function of how much spare RAM you have, the size of your database, and how much of the dataset is “popular”.

(Below is assuming that you are talking about an application-level cache, and not I/O caches, buffer caches, or other lower-level caching.)

If you’re sizing your cache for performance (i.e., maximize throughput, and minimize latency), then one simple formula would be to use as much as your spare RAM as possible that is less than your total database size. Another simple formula would be to start somewhere between 1% and 10% of your database size, and grow based on usage.

If you want to carefully calculate how much cache you need, then the most reliable way to do it is by experimentation, i.e., run load-tests against growing cache-sizes and graph the hit-rate (This is especially true if your database load/usage characteristics are complicated.)

For example, you can have a graph where the X-axis is the cache-size, and the Y-axis has both hit-rate and query-latency. Your goal would be to find the minimum X (cache-size) where hit-rate is maximized and query-latency is minimized (these could be different points).

In order to do this correctly, you need a real-world load test i.e., if you keep logs of your queries, you can replay them. You may want to limit your replay of only non-mutating queries only (for simplicity).

Note that instead of loadtests, you can simplify this process by adding a cache (start somewhere between 1% and 10%) to a live database, then watching the hit-rate and query-latency over time. This is a bit easier to do if you don’t have a load-test framework setup, but may be more intrusive on a production system. If the hit-rate is too low (or query-latency too high), then grow the cache. If not, then see if shrinking it makes a noticeable difference.

(Of course, there are edge cases here, which I’m glossing over, but this is the general idea. E.g., sometimes different types of queries can have different costs, and you may need to allocate different caches for different query types.)

Once you do have a cache, you should monitor it’s statistics along with other query stats, such as latency. You may need to grow it over time.

Or, you may find that you need to warm the cache up before you can get any reliable performance. For example, if you rely on your cache to be able to serve out a specific query-load, and your caching component crashes, starting up cold, then your database will get overloaded for some period of time while the cache warms up.

Anyhow, the short answer is that there is no simple solution, and sizing your cache best done by experimentation.