I am looking for some recommendations on a good data store for activity feeds. The goal is to have a Twitter/Facebook type feed log consisting of various activities users can do throughout our website. The “wall” or “feed” would updated via AJAX showing what the users of the website are currently doing. It will be written to often and then the most recent will be displayed on the site.
(e.g. John Smith recommended Jane Smith’s article 2 seconds ago)
We currently are storing the feeds in MySQL but performance has been poor and I’m concerned with hindering performance throughout the rest of the website if we are constantly hitting the database to grab the most recent user activity as well as writing the feeds.
Any recommendations would be greatly appreciated!
Make use of the best caching solutions like memcache to increase performance. Other than scaling, there are no performance-increasing possibilities for an activity feed.
I would vote for using http://redis.io/ or http://www.mongodb.org/ as an alternative to MySQL for short-term, almost live activity feeds across a site. And a cron job to dump history of activities into MySQL for record keeping.
A look at tumblr’s or twitters architectures can push you to the right direction as well.