Suppose I have ten cron jobs running in parallel which fetch entries from database and process them.
The first cron job gets all the entries from database which have ‘commonId’ field as 1,
the second one gets all the entries from database which have ‘commonId’ field as 2 and so on till 10.
Solution –
When I am adding entries to the database from front-end, I have a file that has the “last” common id saved in it. This id is incremented till 10 and is then again set to 1 for each entry in database.
Hurdle –
There are lots of users on front end and when adding the entries, a same common id is used for two or three consecutive entries. This should not happen. Every entry in database should have serial common id from 1-10.
Is there any solution for this? To get the latest of latest common id?
I’m using MongoDB.
If this isn’t possible. What could be done?
What you want is an ATOMIC operation and MongoDB has ways to do those. See http://www.mongodb.org/display/DOCS/Atomic+Operations
Any reason you specifically want to allocate them round robin like that? Wouldn’t it process faster if you allowed any of the chron jobs to consume any pending job?
I suggest simply relying on the generated MongoID’s (see http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-TheBSONObjectIdDatatype), reverse order by them and find the first unallocated work item, grab it using one of the atomic techniques mentioned. If no work is found, sleep and try again or quit (depending on how often your chron job is rescheduling these workers).