Question

Hey stackoverflow community!

I’m having an issue where a highly involved algorithmic program is using TOO LITTLE cpu utilization: somewhere between 3 and 4%. It is taking very long to return results, and I believe it’s just not working hard enough.

Do any of you geniuses have any ideas why this would occur – if anything I would expect 100% utilization. One additional detail is that the program makes inserts into a sqlite3 database, and thus yes, there are a lot of JNI calls via the sqlite3jdbc library I believe. (Note that I wanted to defer these inserts with a PreparedQuery batch earlier, but this caused major memory problems – there’s a lot of data).

Thanks in advance

UPDATE: Fixed. Yeah, I was just being a doofus, but I didn’t expect that sqlite would start a new transaction and do so much overhead.

I now use a PreparedStatement and queue 32768 entries before insert – seemed like a good enough number to me.

Answer 1

Obviously the database calls are causing delays. Isn’t it an option to create smaller batches and test if that helps?? Maybe you could parallelize the algorithm as well to have a queue somewhere taking results and another thread cleaning out that queue?

edit:

There are also some other problem areas:

• Database optimalization (model)
• Database server configuration
• Disk speed

All these factors should be taken into account