We have a situation where we are calling a function on a API that is rather expensive. Lets call it API.ExpensiveCall()
This ExpensiveCall() is called frequently within a web application. Although it is not noticeable with one user, it becomes noticeable when you have 5 or so simultaneous users.
We want to cache these results. But since API.ExpensiveCall() is essentially a black box to us, we have no way of invalidating our cache, to know when to refresh.
So our proposed solution was to create a windows service, which would simply call this API.ExpensiveCall() every ten seconds, then save the results into a local database that the web application is already using.
This way it would not matter if there is 1 user or 20+ users on the website, the ExpensiveCall() is only called every 10 seconds. It is a very controlled load on the external system the API.ExpensiveCall() is connected to.
The problem
Our Project manager does not agree with this. For some reason he thinks a timed refresh every 10 seconds is a bad idea, because in his opinion puts too much load on the external system.
But if we don’t do anything about it, and leave things the way they are without any sort of caching, it not only degrades the performance of the web application, but it will definitely cause way more than one ExpensiveCall per second on the external system. And that number would multiply depending on the number of users on the web application.
I would like to ask you is this way of caching really such a bad idea? Have you heard of other systems using such a method for caching? And if it is such a bad idea, are there any alternative better ways of caching results from system when it is a black box to you?
EDIT:
Your responses seem to indicate that I should be using the timeout feature of ASP.Net’s memory caching mechanism.
I like the timeout idea. The only (small) issue I see with it now is that when the timeout expires and it is time to call the ExpensiveCall(), it will be a blocking call. As opposed to querying a local table, which is kept up to date by constantly refreshing in a separate process. This is the thing I find attractive with the polling idea. Although I must admit it does feel weird to be polling every 10 seconds, which is why I’m on the fence about it.
Take a look at my response to this question – it describes a way to ensure fresh data in a standard cache. Seems like it might directly address your situation as well.