Question

Lets say there are 100 records in the database for a batch job, when batch job runs and pick up those 100 record and then start processing. During process if error occurs at 10th record then should I rollback all 9 records which are already been processed.

How can we design this scenario??? Your suggestions are welcomed.

Answer 1

I believe you’re asking if you should roll back successful records if an error occurs part-way through batch processing.

You want your DB updates to occur in transactions in a way that leaves database records consistent and legal (with respect to DB and business rules) after each transaction is committed or rolled back.

If each item in your list of 100 records can be processed and recorded individually, then I’d suggest using a flag of some sort (this could be a status field, as well) to indicate whether each record has been processed, then loop through the records to update each one. If you encounter an error, note that somewhere (log file, exception, error table… your call) and move on. When you’re done, you’ll have logged which records were successful and which failed. You should then be able to go back and fix whatever caused the problem(s) on bad records and re-process the skipped records.

If all 100 of your records must succeed or fail together, then you’ll need to wrap your updates in a transaction so they all succeed or fail as one. This will work for dozens of records or (maybe) hundreds of records, but trying to scale up to thousands of records in the same transaction could create scalability problems (performance and contention issues), so you’d want a different solution for a pattern like that.