I have made a SQLite database (~700MB, 3 tables, 3 indexes – 1 rtree index and 2 primary keys). I have marked it as a read-only file (on Windows).
Is it safe and performant to execute just SELECT commands on this database from multiple threads?
If so how can it be made more performant (any options or flags to enable, any tiny tunings)?
This application is in C# using System.Data.SQLite (1.0.82.0), compiled for .NET 4.0 on a x64 machine. And It works fine (not necessarily performant or correctly paralleled because I can not/do not know (how to) prove them). Currently I have no real bottleneck but soon I will! I need to search the rtree as fast as possible. (On my machine 4GB, 2 Cores) It takes sometimes more than 5 milliseconds to search the rtree. I have made that part multithreaded to process my data paralleled. And according to structure of the R-Tree (or I think R*-Tree in SQLite’s case) if my database grows to some GB it should be no problem because these trees has low depths and are fast on large datasets. But if any improvements are possible, then it should be considered in this application.
I can not be sure that the part that has been made parallel is really running in parallel and for example SQLite (or System.Data.SQLite) has not an internal lock. In fact in some tests the parallel version runs slower!
This should be safe, provided each thread has its own connection or you use locks to prevent multiple threads from using the same connection at the same time.