Question

I built a MongoDB. I want to do aggregation by certain grouping. I found this document, which will do that for me. Everything is ok, but certain limitations are pointed out:

1. Output from the pipeline can only contain 16 megabytes. If your result
   set exceeds this limit, the aggregate command produces an error.

2. If any single aggregation operation consumes more than 10 percent of
   system RAM the operation will produce an error.

3. The aggregation system currently stores $group operations in memory,
   which may cause problems when processing a larger number of groups.

How many rows / documents can I process with MongoDB aggregation? I am afraid to use this. Can anyone guide me about this?

Answer 1

I got valid and helpful answer from google groups. Would like to share with you all.

The limitation is not on the number of documents: the limitation is on the amount of memory used by the final result (or an intermediate result).

So: if you aggregate 200 000 documents but the result fits into the 16MB result, then you’re fine. If you aggregate 100 documents and the result does not fit into 16 MB, then you’ll get an error.

Similarly, if you do a sort() or a group() on an intermediate result, and that operation needs more than 10% of available RAM, then you’ll get an error. This is only loosely related to how many documents you have: it’s a function of how big the particular stage of the pipeline is.

Can i increase 16MB via any settings?

Is 16MB limitation only for End-Result OR Is that for that particular
aggregation (means, intermediate results + any temporary holdings +
End result)?

The 16MB limit is not adjustable. This is the maximum size of a document in MongoDB. Since the Aggregation framework is currently implemented as a command, the result from the aggregation must be returned in a single document: hence the 16 MB limit.

see this post