I need to store simultaneous readings from 10 sensors into a SQLite database at about 100Hz. Which means I need to store about 1000 readings a second into the database.
I’ve tried putting the database insert method in a Service running in its own process but I can only collect about 3.5 seconds worth of data. I think the Service is being killed at that point.
I am not using database transactions to insert the data. Just individual calls of INSERT INTO… Is it crucial to use transactions?
Is it necessary to run the database inserts in a separate process? Can I just run them in a new thread off the main process?
I can provide code if needed.
You need to use transactions, and “chunk” your requests by performing multiple inserts in the same transaction. You’ll have to play with the number of inserts, but I would start at 1000 inserts per transaction, and tweak it from there.
Transactions are extremely slow, if you want to insert one record at a time. The reason for this is that SQLite waits for the data to be written to disk before completing the transaction, and so has to wait for the disk platters to spin completely around, during which time the disk is essentially inactive.
On a 7200 RPM hard drive, this essentially limits transactions to 60 per second.
http://www.sqlite.org/faq.html#q19