What are some techniques to present an activity feed tailored to a specific user depending on who he/she follows? Here are some examples:
- User B liked User A’s photo
- Users B and User C commended on User A’s picture (aggregation based on who the user follows)
- User B liked 10 photos: [photo1, photo2, etc]
Services like Instagram provide a feed with activity aggregation tailored to specific users. I assume that saving a separate feed for each user on the server will not scale. So the aggregation need to be done on demand but has to be fast.
Are there open source tools for mongodb or redis designed to take care of this?
There are multiple approaches that can be taken.
In MongoDB a common pattern would be to do pre-aggregation for certain types of information – that is, update some field or document when an event happens to make it fast to read back only the final information at query time.
The other option is to use queries – regular and Aggregation Framework to get the data “in real time”.
The third option may be to run periodic MapReduce jobs to calculate various aggregations and then save them for fast access. This trades off speed of access for eventual consistency – as you wouldn’t see the information updated immediately in real time.