(Using Delphi 2010 + latest SQLite in WAL mode)
I’m using DISQLite (Delphi port of SQLite) with my multi-threading client application (yet to be released, so I can change the DB engine if I really have to)
My profiler clearly says it’s a stupid decision, I tracked this down to 2-3 very simple SQL statements that flies when executed in a single-threaded app, but because threads locking/waiting (SQLite really doesn’t perform well with multiple threads trying to write simultaneously)
I did my best to optimize my code / avoid bottlenecks, but after several weeks of hard work, I wonder now if it’s not just easier to dump SQLite & choose a different DB engine(?)
My requirements are:
- ACID
- Very good simultaneous reading / writing (record level) support
- (Very) Fast & stable DB engine
- B-Tree
- Delphi 2010 support
I’m only using basic INSERT/UPDATE/DELETE commands with indexes, nothing fancy. So my SQL requirements are relatively basic (I don’t need join or other “more advanced” sql stuff).
I’m also open to NQL solutions, as long as it support the requirements mentioned above.
My research lead to Berkley DB, which is, if I understood correctly, a modified version of SQLite with concurrent writing support, but the problem is it’s not really for delphi.
I also read about Kyoto Cabinet, but then again, no delphi support 🙁
Any suggestion would be more than welcome,
Thanks!
FWIW, I finally decided to stick to DISQLite along with this “ugly”, hackish, solution:
Made some (not-so-minor) changes to minimize writing to DB inside threads as much as possible (Two DB inserts required in each thread)
When I absolutely had to write something to DB while working inside threads, I took the SQL query parameters & wrote them in a special folder (writing to files is very fast), ie.
C:\my-project\pending-sql\insert_SOME-GUID.txt
Each file would look like this:
Param1|Param2|Param3|Param4|
Once I’m done with the threads (or if my app crashes), I called a routine that scanned this folder, extracted the SQL parameters and run them using prepared statements (wrapped inside a transaction).
Any file containing less than, say, 4 parameters would be considered corrupt and would be skipped.
This is one heck of hackish ugly algorithm (shame on me!), but it works, it’s fast, it’s (sort of) ACID, and I don’t have to spend months learning another DB engine that may (or may not) be suitable.
I just wanted to thank everyone for their help, time pressure makes it impossible for me to switch to another DB engine, at least for this project.