Question

I had to change the blueprint of my webapplication to decrease loading time (http://stackoverflow.com/questions/5096127/best-way-to-scale-data-decrease-loading-time-make-my-webhost-happy).

This change of blueprint implies that the data of my application has to be migrated to this new blueprint (otherwise my app won’t work). To migrate all my MySQL records (thousands of records), I wrote a PHP/MySQL script.

Opening this script in my browser doesn’t work. I’ve set the time limit of the script to 0 for unlimited loading time, but after a few minutes the script stops loading. A cronjob is also not really an option: 1) strange enough it doesn’t load, but the biggest problem: 2) I’m afraid this is going to cost too much resources of my shared server.

Do you know a fast and efficient way to migrate all my MySQL records, using this PHP/MySQL script?

Answer 1

I solved the problem!

Yes, it will take a lot of time, yes, it will cause an increase in server load, but it just needs to be done. I use the errorlog to check for errors while migrating.

How?

1) I added ignore_user_abort(true); and set_time_limit(0); to make sure the scripts keeps running on te server (stops when the while() loop is completed).

2) Within the while() loop, I added some code to be able to stop the migration script by creating a small textfile called stop.txt:

    if(file_exists(dirname(__FILE__)."/stop.txt")) {
    error_log('Migration Stopped By User ('.date("d-m-Y H:i:s",time()).')');
    break;
}

3) Migration errors and duplicates are logged into my errorlog:

error_log('Migration Fail => UID: '.$uid.' - '.$email.' ('.date("d-m-Y H:i:s",time()).')');

4) Once migration is completed (using mail()), I receive an email with the result of migration, so I don’t have to check this manually.

This might not be the best solution, but it’s a good solution to work with!