Question

I have to insert some data from external source.
My php function runs periodically and fetches results & insert those into table.

There are often results that are already saved.
So they are again inserted.

How do I discard those results that are duplicate so that only new results are entered & duplicate are discarded.

Answer 1

If you can modify the structure of database tables, the best way would be adding UNIQUE INDEX for column or a number of columns which together uniquely identify a single entry. For example:

ALTER TABLE table ADD UNIQUE INDEX name (column_one, column_two);

If the table already contains duplicate records, the altering attempt will result in an error. In that case, you can use IGNORE:

ALTER IGNORE TABLE table ADD UNIQUE INDEX name (column_one, column_two);

Keep in mind that creating unnecessarily big indexes will result in:

• more disk space taken up
• slower writing queries (INSERT, UPDATE, DELETE)

http://dev.mysql.com/doc/refman/5.1/en/create-index.html
(page describes another way of creating indexes but also references ALTER TABLE)