I have a sqlite db on an ARM embedded platform running Linux with somewhat limited resources. Storage device is a microSD card. Sqlite version is 3.7.7.1. The application accessing sqlite is written in C++.
I want to know the number of rows in several tables in regular intervals. I currently use
select count(*) from TABLENAME;
to get this information. I’m having trouble with the performance: When the table sizes reach a certain point (~200K lines), I have a lot of system and iowait load every time I check the table sizes.
When I wrote this, I though looking up the number of rows in a table would be fast as it is probably stored somewhere. But now I’m suspecting that sqlite actually looks through all rows and when I pass the point where the data doesn’t fit into the disk cache anymore I get a lot of io load. This would roughly fit from db size and available memory.
Can anyone tell me if sqlite behaves in the way I suspect?
Is there any way to get the number of table rows without producing this amount of load?
EDIT: plaes has asked about the table layout:
CREATE TABLE %s (timestamp INTEGER PRIMARY KEY, offset INTEGER, value NUMERIC);
From all the information I gathered, count() apparently really needs to scan the table. As plaes has pointed out, this is faster if the count is done on a integer indexed column, but scanning the index is still needed.
What I do now is store the row count somewhere and increment / decrement it manually in the same transactions I use to do inserts and deletes to keep it consistent.