Question

My application relies on having about 48,000 rows of data already present in a database (MySql). It seems to me that with the ActiveRecord abstraction this would present the problem that every time a migration takes place this data would have to be re-inserted. Is this indeed the case? Or is there a workaround?

Answer 1

Ruby on Rails handles migrations by keeping track of changes made to the application schema in a differential manner. The purpose of this is to offer a very controlled mechanism for managing both a database’s schema and data. By using a migration to pre-populate your database after your initial database schema has been created, this migration will only be executed once per application deployment.

Perhaps an example would be the best way to illustrate how rails ActiveRecord::Migrations are handled. Your timestamps will obviously be different, so it’d be best to try it by hand if you’re having trouble following along.

$ rails new example -d mysql
$ cd example/
$ rails generate scaffold Users username:string password:string
  invoke  active_record
  create    db/migrate/20111004022310_create_users.rb
  create    app/models/user.rb
  ...
  ...

This migration creates a table users with two string fields – username and password. Pretty simple. Now, in this example, I’ll be assuming you’ve preconfigured your database.yml.

Creating the database, load the schema, and initial migrations

$ rake db:setup
/Sites/example/db/schema.rb doesn't exist yet. Run 
"rake db:migrate" to create it then try again. If
you do not intend to use a database, you should
instead alter /Sites/example/config/application.rb
to limit the frameworks that will be loaded

We still have to merge initial migrations into the schema. This will load the initial specification for your users model into the database schema, and then the database can be setup.

$ rake db:migrate
==  CreateUsers: migrating ====================================================
-- create_table(:users)
   -> 0.2389s
==  CreateUsers: migrated (0.2390s) ===========================================
$ rake db:setup
-- create_table("users", {:force=>true})
   -> 0.1460s
-- initialize_schema_migrations_table()
   -> 0.5908s
-- assume_migrated_upto_version(20111004022310, "db/migrate")
   -> 0.0010s

At this point, you can easily drop into rails console and see the initial configuration.

$ rails console
ruby-1.9.2-p180 :007 > u = User.new(:username => 'test', :password => 'testing?')
 => #<User id: nil, username: "test", password: "testing?", created_at: nil, updated_at: nil> 
ruby-1.9.2-p180 :008 > u.save!
 => true
ruby-1.9.2-p180 :009 > User.all
 => [#<User id: 2, username: "test", password: "testing?", created_at: "2011-10-04 02:33:56", updated_at: "2011-10-04 02:33:56">] 

Pre-populating the database

Just so you can see what the initial users migration looks like.

$ cat db/migrate/20111004022310_create_users.rb
class CreateUsers < ActiveRecord::Migration
  def self.up
    create_table :users do |t|
      t.string :username
      t.string :password

      t.timestamps
    end
  end

  def self.down
    drop_table :users
  end
end

To populate it, you will need to create a migration to act as a proxy between your preferred data interface and ActiveRecord. If it’s in a database somewhere, I would recommend using something like FasterCSV to import it in the self.up (or up) section of your newly migration. This migration will only be executed once per application deployment, unless you drop your database and start over.

$ rails generate migration prepopulate_users
      invoke  active_record
      create    db/migrate/20111004024648_prepopulate_users.rb
$ cat db/migrate/20111004024648_prepopulate_users.rb 
class PrepopulateUsers < ActiveRecord::Migration
  def self.up
    # Assuming the 'results' is set up as an Enumerable for your data
    results.each do |row|
      u = User.new( :username => row[:username],
                    :password => row[:password] )
      u.save!
    end
  end

  def self.down
  end
end

A word of caution

I will caution you, though, you’re probably in for a long haul (for at least a day or two). I’ve done dirty migrations of data from old applications running on completely different platforms, and you’d better hope your dataset is perfect (and that you’ve mimic’d the data model perfectly) or things will break.

Also, a possible oversight

In case I misunderstood your question, and you’re looking to prepopulate your database with random data, you should take a peek at Forgery, random-data, and/or Faker.

And if I’m completely off-point here, feel free to comment and I’ll adjust my answer accordingly.