On a Postgres backed Rails application running on Heroku I am trying to add a boolean column with default set to false and null set to false (NOT NULL).
The migration takes hours upon hours to run (there are actually about 15 or so such fields that need to get added). Memory usage goes through the roof (5-6gb or more). I think this is because Postgres is wrapping the migration in a transaction and trying to load the whole table into memory so it can rollback on failure.
Without the field default or the NOT NULL the column is added in just a moment.
As an interim solution, I am adding the new column without any restrictions, updating all of the records with a false default value in batches of 1000 records, and then finally altering the column to have the default and NOT NULL requirements.
Now, the migration takes about 3-4 hours to run and uses only about 100mb of RAM.
Is there any better solution out there?
From your timing, I expect that you perform your ‘upgrade’ via RoR, which possibly means ‘row at a time’ processing. I just tested on a 300K rows table with a a primary key and 3 foreign keys, and adding a boolean costs about 5 seconds (too short to benchmark it exact, for me) The query:
Which suggests that your framework produces sub-optimal code. To say the least.