Question

I’ll try here since the mailing list for DM doesn’t seem to have much input from other users unfortunately.

I’m reasonably sure this isn’t something we have to do manually, but maybe I’m wrong. I’ve removed ActiveRecord from my project and have started creating models in DataMapper. It’s all working, but I want to write unit tests for my models (and functional for my controllers). However, my test database is not cleaned between test runs (easily proven with a test). AR takes care of this for you, but it seems like the DM guys haven’t considered this in their dm-rails project.

In a desperate attempt to wipe the slate clean, I dropped all tables in my test database. Now instead of my unit tests failing because the environment is dirty, they fail because the schema doesn’t exist. Looking at the rake tasks available to me, I cannot restore my test DB without also wiping my development database. I’m starting to go insane and hoping a fellow DM + Rails 3 user can nudge me in the right direction.

Specifically, when I run my unit tests, all test data should be removed between the test methods. Also, if I make a change to the schema, I should be able to run my tests and they should work.

I tried putting DataMapper.auto_migrate! in a setup callback in my test_helper.rb, but this doesn’t seem to create the schema (the tests still fail due to the tables not existing when they try to insert/select records).

I’ve seen https://github.com/bmabey/database_cleaner, but do we really have to bring an external library into Rails just to do something that DM probably already has (seemingly undocumented) support for? This also doesn’t address the issue of recreating the schema.

Answer 1

The answer came back on the mailing list that it’s basically a do-it-yourself situation, so to save others the hassle if they end up having to do this too:

Create a .rake file under lib/tasks, called something like test_db_setup.rake:

require File.dirname(__FILE__) + '/../../test/database_dumper'

# Custom logic that runs before the test suite begins
# This just clones the development database schema to the test database
# Note that each test does a lightweight teardown of just truncating all tables
namespace :db do

    namespace :test do
        desc "Reset the test database to match the development schema"
        task :prepare do
            Rake::Task['db:schema:clone'].invoke
        end
    end

    namespace :schema do
        desc "Literally dump the database schema into db/schema/**/*.sql"
        task :dump => :environment do
            DatabaseDumper.dump_schema(:directory => "#{Rails.root}/db/schema", :env => Rails.env)
        end

        desc "Clones the development schema into the test database"
        task :clone => [:dump, :environment] do
            DatabaseDumper.import_schema(:directory => "#{Rails.root}/db/schema", :env => "test")
        end
    end

end

task 'test:prepare' => 'db:test:prepare'

This uses the :test:prepare hook that Rails provides, which runs just before the test suite begins. It copies the schema from your development database into .sql files under db/schema/ (one per table/view), then it imports those .sql files into your test database.

You’ll need the utility class I wrote for this to work (currently it’s written for MySQL >= 5.0.1. You’ll have to adjust the logic if you need a different database.

# Utility class for dumping and importing the database schema
class DatabaseDumper
    def self.dump_schema(options = {})
        options[:directory] ||= "#{Rails.root}/db/schema"
        options[:env]       ||= Rails.env

        schema_dir = options[:directory]

        clean_sql_directory(schema_dir)

        Rails::DataMapper.configuration.repositories[options[:env]].each do |repository, config|
            repository_dir = "#{schema_dir}/#{repository}"
            adapter        = DataMapper.setup(repository, config)

            perform_schema_dump(adapter, repository_dir)
        end
    end

    def self.import_schema(options = {})
        options[:directory] ||= "#{Rails.root}/db/schema"
        options[:env]       ||= "test"

        schema_dir = options[:directory]

        Rails::DataMapper.configuration.repositories[options[:env]].each do |repository, config|
            repository_dir = "#{schema_dir}/#{repository}"
            adapter        = DataMapper.setup(repository, config)

            perform_schema_import(adapter, repository_dir)
        end
    end

    private

        def self.clean_sql_directory(path)
            Dir.mkdir(path) unless Dir.exists?(path)
            Dir.glob("#{path}/**/*.sql").each do |file|
                File.delete(file)
            end
        end

        def self.perform_schema_dump(adapter, path)
            Dir.mkdir(path) unless Dir.exists?(path)

            adapter.select("SHOW FULL TABLES").each do |row|
                name    = row.values.first
                type    = row.values.last
                sql_dir = "#{path}/#{directory_name_for_table_type(type)}"

                Dir.mkdir(sql_dir) unless Dir.exists?(sql_dir)

                schema_info = adapter.select("SHOW CREATE TABLE #{name}").first

                sql = schema_info.values.last

                f = File.open("#{sql_dir}/#{name}.sql", "w+")
                f << sql << "\n"
                f.close
            end
        end

        def self.directory_name_for_table_type(type)
            case type
                when "VIEW"
                    "views"
                when "BASE TABLE"
                    "tables"
                else
                    raise "Unknown table type #{type}"
            end
        end

        def self.perform_schema_import(adapter, path)
            tables_dir     = "#{path}/tables"
            views_dir      = "#{path}/views"

            { "TABLE" => tables_dir, "VIEW" => views_dir }.each do |type, sql_dir|
                Dir.glob("#{sql_dir}/*.sql").each do |file|
                    name       = File.basename(file, ".sql")
                    drop_sql   = "DROP #{type} IF EXISTS `#{name}`"
                    create_sql = File.open(file, "r").read

                    adapter.execute(drop_sql)
                    adapter.execute(create_sql)
                end
            end
        end
end

This will also leave the .sql files in your schema directory, so you can browse them if you want a reference.

Now this will only wipe your database (by installing a fresh schema) as the test suite starts up. It won’t wipe the tests between test methods. For that you’ll want to use DatabaseCleaner. Put it in your test_helper.rb:

require 'database_cleaner'

DatabaseCleaner.strategy = :truncation, {:except => %w(auctionindexview helpindexview)}

class ActiveSupport::TestCase
    setup    :setup_database
    teardown :clean_database

    private

        def setup_database
            DatabaseCleaner.start
        end

        def clean_database
            DatabaseCleaner.clean
        end
end

Now you should be good to go. Your schema will be fresh when you start running the tests, you’ll have a copy of your SQL in the db/schema directory, and your data will be wiped between test methods. A word of warning if you’re enticed by the transaction strategy of DatabaseCleaner… this is rarely a safe strategy to use in MySQL, since none of the MySQL table types currently support nested transactions, so your application logic will likely break the teardown. Truncate is still fast, and much safer.