Question

First of all, is this possibly ready-made?

Hibernate 3.6 , JDBC batch_size 500 , using Hilo generator on the
200.000 entities.

In my example case, I have a request that takes 56 seconds, and I am creating 200,000 entities in the session. So, the session.flush() command takes 32 of those 56 seconds with only one CPU core at %100.

Is there a way to get the list of entities that need to be updated and create the SQL statements, say in four threads?

Answer 1

You cannot simply flush() in different threads, because what flush() does is basically sending all pending SQL INSERT statements to the database using underlying connection. JDBC connections aren’t thread safe, which means you would have to use 4 different connections and thus 4 different transactions. If all inserts need to take place in one transaction, there is nothing you can do here.

If you can live with 4 separate transactions, just create a thread pool and store records in smaller batches. Pool will distribute INSERT operations across several threads.

Also are you sure this will really help? I would guess flush() is not CPU-bound but I/O or network bound. However your experience is different with 100% usage, so I might be wrong. Also try optimizing INSERTs – using stateless session, raw JDBC/Native queries, batch inserts, etc. Spliting into separate threads is much harder.