Question

I’m trying to sync a mssql database (hosted at the customer office) with the mysql database that is on the hosting platform where there’s the customer’s website.

I’ve already got to synchronize three of the four table(structure and data) of the database successfully.
Troubles come when I attempt to sync the 4th table which has about 2300 records with 2 BLOB fields with the images of each product.
The first thing I’ve tried was a

SELECT * FROM chunkyTable

but many timeout were fired (despite I set the “mssql.timeout” to 7200.

So I set up a paginated synchronization which read 10 records each query and insert them into the mysql database.

Everything works smoothly for the first 150 records.
From the log I make I can see that for the first 15 pages , synchronizing 10 records takes about 2 up to 6 seconds (consider that from the website I connect to the mssql server which is connected with a common ADSL which constitute a quite big bottleneck).
From the 16th page on the time required for synchronizing the 10 records rises up to 5:30 seconds.
I thought at first that this was due to the fact that the first 150 records don’t have any image stored, so I checked what was inserted into the mysql and I discovered that nothing was inserted!

These led me to think that maybe mysql has some limits on the lenght of the argument passed to the mysql_query() function.
The way the query is composed is:

INSERT INTO chunckyTable(field1,field2,...fieldN)
VALUES (val1,val2,val3,...,valN),(val4,val5,val6,...,valM),

and so on

repeated for 10 records.

Do you know if there are some limitations on the maximum length on the sql string passed to mysql_query function (I had a look to the phpinfo() output but there isn’t any configuration about this topic)?

Do you have any similar experience with this kind of synchronization?
Any help would be really appreciated!

Thanks,
Luke

Answer 1

I found what was firing the error.
Actually there were 2 things.

1. I was composing the query this way $query=”\””.mysql_real_escape_string($val).”\”” but the \” was letting the query crash.
2. Mysql @@global.max_allowed_packet on my server is set to 1Mb , but each row contains 2 blob field with ~150Kb image in them. So each row is more than 300Kb since there are also the data of the records and when I was packing the insert of 10 rows in one query I exceeded the limit of max_allowed_packet. To resolve this I insert the row one by one

Thanks anyway for the advise you gave me!