Question

I’m developing an application and have over 100,000 records, that have been read in from a file, which need to be inserted into a MySQL database.

What would be the best approach of inserting these to the database?

The approach I’m currently working from is generating a SQL query to insert all the records in one call to the database.

INSERT INTO table (field1, field2)
VALUES
(123, 456),
(125, 984),
...

Is there a limit to the number of records that can be inserted at once? And in terms of performance is this the best method?

I have considered an alternative of splitting the records into a number of queries but I’m unsure if this would have any benefit?

Any advice would be greatly appreciated, thanks!

Answer 1

The way you’re doing it is indeed very efficient. The maximum query length is determined by the max_allowed_packet setting so you may need to split the query into several queries, see http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_max_allowed_packet