Question

I’m trying to insert huge amount of data into database, and I want this to be in the most efficient way. Now, I’m storing my objects in an arraylist and when the size of the arraylist becomes 5000 I iterate through all elements and create a one mysql insert stament with multiple values and then I call executeUpdate() function. I’m wondering that is there any faster way for doing it, for example instead of storing data in an array, I may create a local CSV file and use LOAD DATA LOCAL INFILE for once, or I may do the prepared statement still using the arraylist.
Can you advice me that, which one is the best or is there any other ways to do that ?

Thanks.

Answer 1

• For innodb: use a transaction to do all/many of your inserts as a first optimization. That might yield a huge performance gain and might already be sufficient
• The LOAD DATA… is a very good option, too. That will also do everything within a transaction and use your bandwidth wisely.

• You could insert multiple values within one sql command, like:

  INSERT INTO talbe_blubb (col_a,col_b) VALUES (1,3), (2,2);
  

I am not sure but don’t think prepared statements might gain any advantage in case of simple insertions into a table.