I am faced with the following problem. We are required to call the Twitter API to send a status update then we must write a record to a local database, recording that this Twitter update happened.
My question is how can we ensure data consistency if either Twitter or the database calls fail? If we update Twitter and the database call fails we will have no local record of the Twitter call and if the Twitter call fails but the database call succeeds we will have an erroneous local record.
I understand the obvious first steps such as only updating the DB if Twitter call is successful or vice-versa but this only addresses half the problem.
I imagine that this is a common problem but I have tried searching for relevant information without much luck.
Stephen,
You will have a SPOF (single point of failure) in any case – it will be your application. However, for reducing the risk of failure, you can dump your data into a queue (of any sort; even a /tmpfs on the web-server will suffice), and repeat the attempts to write it to DB until it’s available.
If the call to Twitter API fails, you obvoiusly can just put a record of failed attempt to the DB.