I have an application where users start taking tests and sometimes the users forget to finish/submit their tests. The admins have requested a button that they could click which goes through the system, finds all of the unsubmitted exams, and scores them.
This process is done with a single select statement to load all of the impacted exams. Then, an in-memory process determines each user’s score. The problem is that updating the database with each user’s score is very slow. The update queries take several minutes and hundreds or thousands of update statements are run (depending on how many unsubmitted exams are waiting to be processed). My database server uses MySQL.
Is there any good way to merge these update queries into a single update? Each user receives a unique score for their exam. I’ve found a few links about how to do this, but my fellow developers tell me that these are unmaintainable and too complicated. Most of them deal with adding case statements to the update query (one for each scored exam). Here is an example of one suggestion:
http://dashasalo.com/2009/06/18/update-multiple-rows-with-different-values/
Is there any other idea that I’m missing which might improve the performance of this process? Thanks!
Note: I do have proper indices setup on this table and each individual query takes between 0.2-0.5 seconds to complete. I also suggested a night-time cron job or an automatic cleanup. The admins rejected this stating that they want to control when the cleanup process occurs and no cron/automatic/nightly process is acceptable to them.
When you say that the scoring is done in memory, I presume you mean that an application is selecting the test answers from the database and scoring them.
Once the scores are determined, you could sort the results based on the score and build lists of tests for each score. Using this list, you could build some dynamic SQL to update all tests that have the same score, something like: