Question

I have one application where "persisting to database" is consuming 85% time of the entire application flow.

I was thinking of using multiple threads to do the insert because inserts are mostly independent here. Is there any way to achieve multi threaded insert using any of JPA implementation ? Or is it worth doing the mutli threaded insert, from improving the performance perspective ?

Note: Inserts are in the range of 10K to 100K records in a single run. Also performance is very very critical here.

Thanks.

Answer 1

Multi-threading insert statements on database won’t really make it perform any faster because in most databases the table requires a lock for an insert. So your threads will just be waiting for the one before it to finish up and unlock the table before the next can insert – which really doesn’t make it any more multi-threaded than with a single thread. If you where to do it, it would most likely slow it down.

If you inserting 10k-100k records you should consider using either batch insert statements or bulk insert commands that are native to the database your using. The fastest way would be the native bulk insert commands but it would require you to not use JPA and to work directly with JDBC calls for the inserts you want to use bulk commands on.

If you don’t want to play around with native bulk commands I recommend using Spring’s JDBCTemplate which has templated batch insert commands. It is very fast and I use it to batch insert 10k-20k entities every 30 seconds on a high transaction system and I am very pleased with the performance.

Lastly, make sure your database tables are optimized with the correct indexes, keys and options. Since your database is the bottleneck this should be one of the first places you look to increase performance.