Question

I am using jQuery’s $.ajax method to retrieve some JSON from an API.

Each time the page is loaded, a call to the API is made, regardless if the user has has received this data before – which means when a large amount of users are on the page, the API limiting would come into effect.

My thought of how to deal with this would be firstly pushing the data to a database (pushing to a PHP script), and then checking the database to see if anything is cached, before going back to the API to get more up to date information if required.

Is this a viable method? What are the alternatives?

It just seems like jQuery is actually a hurdle, rather than doing it all in PHP to begin with, but as I’m learning the language, would like to use it as much as I can!

Answer 1

In order to help distinguish between opinions and recommended techniques, lets first break down your problem to make sure everyone understands your scenario.

Let’s say we have two servers: ‘Server A’ and ‘Server B’. Call ‘Server A’ our PHP web server and ‘Server B’ our API server. I’m assuming you don’t have control over the API server which is why you are calling it separately and can’t scale the API server in parallel to your demand. Lets say its some third party application like flickr or harvest or something… let’s say this third party API server throttles requests per hour by your developer API key effectively limiting you to 150 requests per hour.

When one of your pages loads in the end-users browser, the source is coming from ‘Server A’ (our php server) and in the body of that page is some nice jQuery that performs an .ajax() call to ‘Server B’ our API server.

Now your developer API key only allows 150 requests per hour, while hypothetically you might see 1000 requests inside one hours to your ‘Server A’ PHP server. So how do we handle this discrepancy of loads, given the assumption that we can’t simple scale up the API server (the best choice if possible).

Here are a few things you can do in this situation:

• Just continue as normal, and when jQuery.ajax() returns a 503 service
  unavailable error due to throttling (what most third party APIs do) tell your end user politely that
  you are experiencing higher than normal traffic and to try again
  later. This is not a bad idea to implement even if you also add in
  some caching.
• Assuming that data being retrieved by the API is cache-able, you
  could run a proxy on your PHP server. This is particularly well
  suited when the same ajax request would return the same response
  repeatedly over time. (ex: maybe you are fetching some description
  for an object, the same object request should return the same description response for
  some period of time). This could be a PHP Pass through proxy or a
  lower level proxy like SQUID caching proxy. In the case of a PHP Pass through proxy you would use the “save to DB or filesystem” strategy for caching and could even re-write the expires headers to suite your level of desired cache lifetime.
• If you have established that the response data is cache-able, you can
  allow the client side to also cache the ajax response using
  cache:true. jQuery.ajax() actually defaults to having cache:true, so you simply need to not set cache to false for it to be caching responses.
• If your response data is small, you might consider caching it client-side in a
  cookie. But experience says that most users who clear their temporary
  internet files will also clear their cookies. So maybe the built in caching with jQuery.ajax() is just as good?
• HTML5 local storage for client-side caching is cool, but lacks wide-spread popular support.
  If you control your user-base (such as in a corporate environment) you
  may be able to mandate the use of an HTML5 compliant browser. Otherwise, you
  will likely need a cookie based fallback or polyfill for browsers lacking
  HTML5 local storage. In which case you might just reconsider other options above.

To summarize, you should be able to present the user with a friendly service unavailable message no matter what caching techniques you employ. In addition to this you may employ either or both server-side and client-side caching of your API response to reduce the impact. Server-side caching saves repeated requests to the same resource while client side caching saves repeated requests to the same resources by the same user and browser. Given the scenario described, I’d avoid Html5 LocalStorage because you’ll need to support fallback/polyfill options which make the built in request caching just as effective in most scenarios.

As you can see jQuery won’t really change this situation for you much either way vs calling the API server from PHP server-side. The same caching techniques could be applied if you performed the API calls in PHP on the server side vs performing the API calls via jQuery.ajax() on the client side. The same friendly service unavailable message should be implemented one way or another for when you are over capacity.

If I’ve misunderstood your question please feel free to leave a comment and clarify and/or edit your original question.