Home/ Questions/Q 6150253
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T19:35:26+00:00

I have hundred of thousands of elements to insert into a database. I realized

  • 0

I have hundred of thousands of elements to insert into a database. I realized calling an insert statement per element is way too costly and I need to reduce the overhead.
I recon each insert can have multiple data elements specified such as

INSERT INTO example (Parent, DataNameID) VALUES (1,1), (1,2)

My issue is that since the “DataName” keeps repeating itself for each element I thought it would optimize space if I stored these string names in another table and reference it.
However that causes problems for my idea of the bulk insert which now requires a way to actually evaluate the ID from the name before calling the bulk insert.

Any recommendations?
Should I simply de-normalize and insert the data every time as plain string to the table?
Also what is the limit of the size of the string as the string query amounts to almost 1.2 MB?

I am using PHP with MySQL backend

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You haven’t given us a lot of info on the database structure or size, but this may be a case where absolute normalization isn’t worth the hassle.

    However if you want to keep it normalized and the strings are already in your other table (let’s call it datanames), you can do something like

    INSERT INTO example (Parent, DataNameID) VALUES
(1, (select id from datanames where name='Foo')),
(1, (select id from datanames where name='Bar'))
    • 0