I have got 2 mongo databases. 1. Staging, 2. Production.
In the staging we have around 5 collections of seed data, on which we run some batch jobs and populate few more say 3 collections.
The 8 collection becomes seed data for the production which has user information + this seed data.
Is there any better patterns on managing the data push to the staging and from the staging to production. Right now we are trying to mongoexport all the collections and tar.gz it and archive it on network drive on each stage and mongoimport it.
Its very painful and taking long to export,import and archive which on gzipping is around 1.5 GB.
Is there any good patterns to solve this problem?
‘mongoimport’ and ‘mongoexport’ is meant to be used with data from outside systems – all data is translated into plain json and then back again into bson.
If you use ‘mongodump’ and ‘mongorestore’ you should see much better performance as both deal with bson directly which is more compact to store and does not require two translations (once to json and once from json).