Assume there’s a mobile app and a server.
I have question about rate limiting and hoping someone can give some advice on a design as I’m banging my head on how to navigate around rate limit. There must be something I”m missing because the 150 unauthenticated rate limit per IP per hour is extremely low.
Imagine the scenario I want to build is the following (simplified into a trivial example for this discusion). Assume user is signed into Twitter for this entire discussion to remove discussion about oAuth.
Mobile talks to our service to show users twitter friends list. Every time the mobile app is loaded, it will show the entire friends list, and highlighting the new friends that were added within the last 2 days.
That’s it. But the trick is that I want to ensure that the friends list is always up to date in the client, which means our server has to have the most recent up to date friends list.
Periodically, I want my server to automatically scan the Twitter friends list for every user of my app to see if new friends have been added.
Our initial design was getting our server to do all the work with this flow:
- New User signs in on client, gives access token to server
- Server makes call to Twitter REST APIs to get initial friends lists
- Server stores the Twitter Friends IDs and shows responds to the client with that list.
Periodically (e.g. every 48 hours), server checks Twitter REST APIs for friends list for each user and compares it to our cached Twitter friends list we have for them to see who is new and to highlight in the mobile app.
The good thing about this is that all the interaction with twitter to get friends list, compare and peridiocally refresh is on the server. Mobile client just makes a single call to my server and gets friends list.
The problem with this design is that it will work for a single user, but since the rate limit is 150 per hour on un-authenticated calls, I will hit my limit as soon as 151 users user my service (which has a fixed IP).
The only solution I can see is to have the client do the work for each user, then send me the friends list which my server caches. This takes care of Step #2 above. However, for Step #4, I’d have to build something into the client to auto refresh twitter friends and send back to the server.
This is super clumsy to have the client involved at all in this Twitter friends list operation.
At first I thought I was crazy and the public unauthenticated APIs like getting friends lists wouldn’t be subject to rate limiting. However, according to their docs, it is.
Am I missing something obvious or is the only way to solve this is to put heavy logic into the client?
Since you are making request on behalf of users you should make those requests be authenticated as those users. Then requests will count against each users own pool of 350 requests/hour.