Question

We have a messaging application that maintains an unread/read counter. Currently, we cache this value for one hour, and if a user hits the app and the cache is expired, a count is done on the database tables to refresh the cached value.

The issue is when a high number of users come all at once and have expired caches, which is actually very common, this put tremendous strain on the db to do all the counts live and at the same time.

I would like to find a way to actively maintain this counter in the cache as we add or remove messages, so that the user is only ever hitting the cached value and it never expires. The new problem becomes making sure that count value stays in sync, as it’s possible that the systems that add or remove messages that would trigger a cache update might miss a couple, so the two numbers won’t be in sync anymore.

Some of the options I’ve come up with:

• Doing a forced refresh every X updates. This is fine for active users, but less active users are more likely to have an inaccurate count, and with no cache expiry, that count remains for a long time.

• Have a background job that updates the count. This has the problem that it is spending db resources on inactive users, which is inefficient and time consuming.

Does anyone have any general suggestions for this type of count maintenance?

Answer 1

I’m not 100% sure of the application you have, but if I assume you have something like a mail client where you have number of received messages and number of sent messages, then I have some ideas:

1. Check out the Thundering Herd idea for caching: http://en.wikipedia.org/wiki/Thundering_herd_problem The idea is that you don’t always use the same cache expiration value. Introduce some randomness. Some will expire in 1 hour, some in 58 minutes, some in 1 hour and 2 minutes. This prevents a lot of things from expiring at the same time.

2. Consider invalidating the cache on the write side. In other words, let’s say I send a message to someone. When I send it in the UI, the code should go invalidate the other users cache (their received count just went up by 1) and then re-populate the value. In other words don’t just invalidate the cache with the one application call. Invalidate it and put the new value in cache pro-actively.

3. You can seed the cache on a schedule, but like you said you’d be doing work that isn’t necessary.

4. I’ve noticed a lot of large companies taking problems like this and pushing them out to the UI. You allow for slightly inconsistent data for a period of time, or you do little tricks. For example, when they send a message if you want it to “feel” fast, simply increment their “sent” counter by 1 with Javascript (assuming a web application) even if the counter hasn’t yet been saved to the database.

In other words give instant feedback in the UI and fudge a bit and then asynchronously write the data and wait for it to finish the write/cache updates.