Question

I am working on a project which has some important data in it. This means we cannot to lose any of it if the light or server goes down. We are using MongoDB for the database. I’d like to be sure that my data is in the database after the insert and rollback the whole batch if one element was not inserted. I know it is the philosophy behind Mongo that we do not need transactions but how can I make sure that my data is really safely stored after insert rather than sent to some “black hole”.

• Should I make a search?

• Should I use some specific mongoDB commands?

• Should I use sharding even if one server is enough for satisfying
  the speed and by the way it doesn’t guarantee anything if the light
  goes down?

What is the best solution?

Answer 1

Your best bet is to use Write Concerns – these allow you to tell MongoDB how important a piece of data is. The quickest Write Concern is also the least safe – the data is not flushed to disk until the next scheduled flush. The safest will confirm that the data has been written to disk on a number of machines before returning.

The write concern you are looking for is FSYNC_SAFE (at least that is what it is called from the point of view of the Java driver) or REPLICAS_SAFE which confirms that your data has been replicated.

Bear in mind that MongoDB does not have transactions in the traditional sense – your rollback will have to be rolled by hand as you can’t tell the Mongo database to do this for you.

The other thing you need to do is either use the relatively new --journal option (which uses a Write Ahead Log), or use replica sets to share your data across many machines in order to maximise data integrity in the event of a crash/power loss.

Sharding is not so much a protection against hardware failure as a method for sharing the load when dealing with particularly large datasets – sharding shouldn’t be confused with replica sets which is a way of writing data to more than one disk on more than one machine.

Therefore, if your data is valuable enough, you should definitely be using replica sets, perhaps even siting slaves in other data centres/availability zones/racks/etc in order to provide the resilience you require.

There is/will be (can’t remember offhand whether this has been implemented yet) a way to specify the priority of individual nodes in a replica set such that if the master goes down the new master that is elected is one in the same data centre if such a machine is available (ie to stop a slave on the other side of the country from becoming master unless it really is the only other option).