Question

I am modeling my data-schema at the moment and I am not sure about if my thought-process is making sense. So I thought I might ask some of the more experienced MongoDB guys here:

---

Let us suppose my application produces up to 10.000 event-documents a day. I want to access them time-based. Like: “Give me all the events of those three days!”.

My RDBMS knowledge I gathered at university first told me: “Do an Events-Collection and give each document the Event’s propertie ‘Date’. Done.”

But then I came across with the idea to do a collection for each day!. Then I could access those Events very fast, by just getting all the events of one single day by just calling its coresponding collection.

Does this makes sense? Can I have hundreds/thousands collections without sacrifising speed/performance?

---

Thank you for advise 🙂

Answer 1

10.000 documents per day is not a very much. Over the course of one year, that is 3.65m documents. That is certainly not a very small collection, but I don’t see much sense in breaking them up.

The downsides in this specific case are

• It’s hard to change your query patterns later on. If you suddenly need hour-precision, you are in trouble. If you want to find all events in the last year with some field x set to y, you’ll have to query 365 or 366 collections.
• Your query patterns will be more complicated, because you have to deal with the different collection names. Also, you need several round-trips to the database.
• Internationalization is very complicated, because “day” is not a well-defined point in time across the globe. Using a UTC DateTime field on the other hand allows you to query in different time zones, should that ever be required.
• Managing a large number of collections can be tedious, working with the shell will be quite annoying.
• Sharding is normally performed on a per-collection-basis. If you have many smaller collections, you can’t do auto-sharding.

However, working with a larger number of collections is possible, though there are limits you should understand. As the docs explain, you can have 12,000 collections w/ one index each with the default settings. See there for more details.

Server Density blogged about their approach, they use a lot of collections too, but they chew 650m documents and they claim that it doesn’t make a big difference performance-wise.