Question

I have a desktop app that has 65 modules, about half of which read from or write to an SQLite database. I’ve found that there are 3 ways that the database can throw an SQliteDatabaseError:

1. SQL logic error or missing database (happens unpredictably every now and then)
2. Database is locked (if it’s being edited by another program, like SQLite Database Browser)
3. Disk I/O error (also happens unpredictably)

Although these errors don’t happen often, when they do they lock up my application entirely, and so I can’t just let them stand.

And so I’ve started re-writing every single access of the database to be a pointer to a common “database-access function” in its own module. That function then can catch these three errors as exceptions and thereby not crash, and also alert the user accordingly. For example, if it is a “database is locked error”, it will announce this and ask the user to close any program that is also using the database and then try again. (If it’s the other errors, perhaps it will tell the user to try again later…not sure yet). Updating all the database accesses to do this is mostly a matter of copy/pasting the redirect to the common function–easy work.

The problem is: it is not sufficient to just provide this database-access function and its announcements, because at all of the points of database access in the 65 modules there is code that follows the access that assumes the database will successfully return data or complete a write–and when it doesn’t, that code has to have a condition for that. But writing those conditionals requires carefully going into each access point and seeing how best to handle it. This is laborious and difficult for the couple of hundred database accesses I’ll need to patch in this way.

I’m willing to do that, but I thought I’d inquire if there were a more efficient/clever way or at least heuristics that would help in finishing this fix efficiently and well.

(I should state that there is no particular “architecture” of this application…it’s mostly what could be called “ravioli code”, where the GUI and database calls and logic are all together in units that “go together”. I am not willing to re-write the architecture of the whole project in MVC or something like this at this point, though I’d consider it for future projects.)

Answer 1

Your gut feeling is right. There is no way to add robustness to the application without reviewing each database access point separately.

You still have a lot of important choice at how the application should react on errors that depends on factors like,

• Is it attended, or sometimes completely unattended?
• Is delay OK, or is it important to report database errors promptly?
• What are relative frequencies of the three types of failure that you describe?

Now that you have a single wrapper, you can use it to do some common configuration and error handling, especially:

• set reasonable connect timeouts
• set reasonable busy timeouts
• enforce command timeouts on client side
• retry automatically on errors, especially on SQLITE_BUSY (insert large delays between retries, fail after a few retries)
• use exceptions to reduce the number of application level handlers. You may be able to restart the whole application on database errors. However, do that only if you have confidence as to in which state you are aborting the application; consistent use of transactions may ensure that the restart method does not leave inconsistent data behind.
• ask a human for help when you detect a locking error

…but there comes a moment where you need to bite the bullet and let the error out into the application, and see what all the particular callers are likely to do with it.