Question

Before inserting data into table i need to check against duplicate records and report list of those records.
Query should not insert anything while there is at least one duplicate.

Number of records to insert: ~1000

Rows in table: ~1.000.000

Table:

CREATE TABLE `test` (
  `A` varchar(19) NOT NULL,
  `B` varchar(9) NOT NULL,
  KEY `A` (`A`),
  KEY `B` (`B`)
) ENGINE=InnoDB;

I need to check for both columns:

What is the efficient way of doing this.

Thanks in advance.

P.S. consider using php and mysql

Answer 1

Say duplicate means a username already exists. so you can try:

1] use a for loop to group all usernames that’s in your insert list: pseudo code follows:

  $names="";
  for(each element){
      $names.=$val.", ";
  //remove last comma ","

Then use following sql statement to get all duplicate entries.

  select * from myTable where username in ($names);

having obtained a list of all your duplicates, then insert as

insert into myTable (username, address, otherstuff)
values("henry", "35 skid", "who knows")
ON DUPLICATE KEY UPDATE 
otherstuff = values(otherstuff);