Question

I am creating an API using PHP and MongoDB. In this system I got users and each user can upload files. Users can also “follow” each other.

I need to return a feed of all the latest files uploaded by users that the authenticated user is following. I am not really sure how to design and execute this.

This is what I am thinking of implementing.

1. Get all the users that the user is following.
2. Loop through this array of users and get latest 4 files for each user and add them to an array.
3. Somehow order these files (this array) by creation date.
4. Return it.

Is this really the optimal way? Is there a better way? Users are saved in the collection Users and files in the collection files. Following is saved in the collection users.followers.

Answer 1

This is the fan-in vs. fan-out problem. I’d suggest you try fan-out:

Keep a feed collection for your users. When a user uploads a document, insert a new feed item in each of her friends feed item collection. The collection could look like this:

{
    "_id": (some id)
    "UserId": (id of the user who 'owns', i.e. reads this feed)
    "FriendId": (if of the friend who posted the file)
    "FriendName": "John Doe" (name of the fried, denormalized)
    "Timestamp": ...
}

Use a compound index {UserId, Timestamp}.

This approach is write-heavy: If Jane has hundreds of friends, these hundreds of inserts will take their time. On the other hand, uploading a file generally takes a lot of time anyway, so the overhead is negligible, and your reads will be ridiculously simple.

Of course, this can be further optimized with more effort, but it should do fine for quite a bit of traffic.