Home/ Questions/Q 8188775
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-07T03:01:58+00:00

We have a system in which we want to maintain stats on large volumes

  • 0

We have a system in which we want to maintain stats on large volumes of logged information split into separate channels, each channel will have an overview document maintaining stats including Max and Min values for the Hours, Days, Months and Years in a cascaded fashion.

We could maintain a single document to hold the stats of a channel for a month, which would have a single Monthly Max/Min, 31 Daily Max/Mins with 24 Hourly max mins for each of the those days.

Using a document for counting using $inc is great for in place atomic updates ,a $max and $min on the update modifier would solve our problem but that’s unfortunately not available.

We can keep the max and min within the task processors and issues $set but that has issues with coordinating those over all of the task processors in the system (potentially on different machines), so we have decided to keep mongo as the main reference. Also the same stats might be being updated by two seperate tasks being run concurrently on different task processor instances.

So we are looking for comment/suggestions on the best solution (we won’t necessarily keep the raw data so post processing is not an option)

  1. Read, update & save each time we need to update?

  2. Use db Eval to execute JavaScript server side to update the max/man cascade by passing in just a single value?

  3. Is there another way?

Many thanks for your help.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I think you’re looking for optimistic locking. You can ensure that updates are atomic by storing a version field in your documents. Consider the following example:

    {_id: xyz, version: 2, max: 10}

    First find the document:

    db.example.find({_id:xyz})

    Store the version number and determine (client-side) whether or not the max value needs to be updated. If so,

    db.example.update({id:xyz, version: 2}, {$set: {max: 12}, $inc: {version: 1}})

    This update will fail if the version has changed between the time that the query and the update took place, thereby preventing concurrency-related data corruption. You can use getLastError to determine if the update failed, and subsequently retry from step one.

    You should avoid db.eval() since this operation requires a global lock.

    Additional resources on optimistic locking can be found here:
    http://www.mongodb.org/display/DOCS/Atomic+Operations#AtomicOperations-%22UpdateifCurrent%22
    http://code.google.com/p/morphia/wiki/MongoNewsletterArticleDec2010

    MongoDB documentation on concurrency: http://www.mongodb.org/display/DOCS/How+does+concurrency+work

    • 0