Question

I have a standard one directional master-slave setup.
The tables have MyIsam storage engine.

The Versions are:

Master
Server version: 5.1.41-3ubuntu12.10-log
Protocol version: 10

1st Slave:
Server version: 5.1.41-3ubuntu12.10
Protocol version: 10

2nd Slave:
Server version: 5.1.49-1ubuntu8.1
Protocol version: 10

I have set up all users beside a dedicated replication user to be read only on the slaves.

The following statement, executed on the master, gets not replicated correctly:

$insert = "INSERT INTO search_affiliate_product
(film_id, ext_id, affiliate_id, `status`, url, created_at)
SELECT film_id, LPAD(imdb, 7, '0'), $affiliate_id, 5, CONCAT('http://www.imdb.com/title/tt', LPAD(imdb, 7, '0'),'/'), NOW() FROM search_film_entity
WHERE film_id NOT IN (SELECT film_id FROM search_affiliate_product WHERE affiliate_id = $affiliate_id)
AND status IN (5, 9)
AND release_year BETWEEN 0 AND $year
AND imdb > 0";

The slave gets this error:

Error 'Duplicate entry '271769' for key 'PRIMARY'' on query. Default database:     'flimmit_search_14'. Query: 'INSERT INTO `search_affiliate_product` (`affiliate_id`,     `ext_id`, `url`, `status`, `film_id`, `created_at`, `updated_at`) VALUES ('16', '1991/JohnnyStecchino', 'http://www.cineman.ch/movie/1991/JohnnyStecchino/review.html', '5', '102164', '2011-10-26 02:30:05', '2011-10-26 02:30:05')'

I also made some other observations:

• The statement gets transferred “as is” over the bin log (relay log and master log), so the explicit data that is inserted is not contained, but the whole statement with the select and subselect is executed on the slave, relying on the local data.

• If i look at the master and slave table and sort by primary key, I see a range of inserts produced by the statement above on both servers. The rows before and after that are in sync. Also a lot of rows in that range that is produced by that statement is equal. But there are also discrepancies.

I thought that the problem may because by concurrent sql transactions. So that the inserting process interferes with other inserts and therefore the values get mixed up, where on the slave this does not happen because the log is processed procedurally.

I also noticed that during the inserting process of the statement some IDs seem to be inserted that are not inserte on the other server, and some duplicates get inserted (duplicates on the field ext_id which is populated by the imdb field from the other table, which is unique…

I am really stuck here.

Any suggestions? I would really like to understand why this is happening.

I thought about selecting all the ids and making single insert statements in the application or lock the table before executing it. I think this could solve it, but i would really like to understand the error.

The next weird thing is that the second slave seems to run fine! only the first slave is getting problems. I just don’t understand it.

Answer 1

this kind of replication can only be done reliably under innodb with the following settings in place on the client

[mysqld]
innodb_flush_log_at_trx_commit=1
sync_binlog=1