I have an existing django web app that is in use. I have to radically migrate one key model in my design to a completely new design, but I want to cache all of the existing data for that model and migrate them to the new records in production when ready to deploy.
I can afford to bring my website down for a few hours one night and do whatever I need to do to migrate. What are some sane ways I can do this migration?
It seems any migration would need to: 1) Dump all of the existing data into some format, such as SQL, JSON, XML 2) Migrate the model to the new format 3) Reload the data into the new model using a conversion script
I also thought of trying to store all of the existing data in some other model called ‘OldModel’ (if Model is the name of the existing model) and then migrating the data live.
There is a project to help with migrations that I’ve heard of: South.
Having said that, I admit we’ve not used it. We still plan our migrations using a file of SQL statements. Madness, I know, but it has the advantage of testability. You can run it as many times as necessary during development and staging testing before the ‘big deploy’. It can be source controlled, diffed, etc. It can also, therefore, be called from a larger deployment script. Of course, we back up production before running it 🙂
If your database does journaling, using the old-fashioned method has the added advantage that there is a transaction history that can be rolled back.
Experiments we’ve run with JSON, XML and ‘OldModel’ -> ‘NewModel’ style dumps have scaled pretty poorly. Mind you, YMMV… we have quite a large database. By using a script, you can run on your production database without having to offload or reload vast amounts of data. This way even a complicated migration can take seconds, rather than hours.