I have 15k objects I need to write to the database everyday. I’m using a cronjob to write to mysql server every night. Each night, 99% of these 15k objects will be the same and identified uniquely in the DB.
I have set up a DB rule stating there will be no duplicate rows via specifying a unique key.
I do NOT want to check for an existing row before actually inserting it.
Therefore, I have opted to INSERT all 15k objects every night and allow mysql to prevent duplicate rows…(of course it will throw errors).
I do this because if I check for a pre-existing row – it will significantly reduce speed.
My question: Is there anything wrong with inserting all 15k at once and allowing mysql to prevent duplicates? (without manually checking for pre-existing rows) Is there a threshold where if mysql errors out 1,000 times that it will lock itself and reject all subsequent queries?
please help!
Using
INSERT IGNORE INTO...will make MySQL discard the row without error and keep the row already present. Maybe this is what you want?If you instead want to overwrite the existing row you can do
INSERT INTO ... ON DUPLICATE KEY UPDATE .... See docs