I am working on a project in which I intend for large numbers of users to store their messages within my database. These messages which will consist of a mix of text and binary data (think an email) will have a many to one relationship with individual users. As mongo db has a document size limit of 16mb for an individual document I was thinking I will need to associate them via linking.
This will require two separate collections which means more round trips and worse performance. Is there any alternative that I am not thinking of?
Well, there’s not really much choice. Linking or embedding are the standard choices. However, in some cases it may be beneficial to implement a hybrid approach. I call it “batches”.
The essence of the approach is: you stick many messages into a single document, just as in embedding. When that document is about to overflow, you start writing to a second document (and have first document link to it).
This way you have better bulk read performance than with single linked documents, but you also can’t query them as efficiently (I imagine that you’d want to query several documents out of the pack, since those are messages).
Assess your query patterns and choose wisely.