Question

I have some data in a csv file and I want to insert it into a Mysql table. My question is how to do that in a way that no duplicate values inserted into the table. I have looked at the following reference http://dev.mysql.com/doc/refman/5.1/en/load-data.html and there is a keyword IGNORE but can’t see how to use it to insert unique values only.

Answer 1

Add unique index on the table using one or more columns:

CREATE TABLE `test` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`start` VARCHAR(10) DEFAULT NULL,
`end` VARCHAR(10) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `start` (`start`,`end`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8;

You would use ignore to skip rows that error out due to duplicate index error.

INSERT IGNORE into test (id, start, end) values (1,'test','test');

Using load:

LOAD DATA INFILE 'file_name' IGNORE INTO TABLE test;