Question

I’ve been following this fantastic tutorial about mass inserting data. All is well, I’ve got my transaction times down from about 30 seconds to less than 1 🙂

I just don’t know how to populate the fields in a child model:

has_many :check, :dependent => :destroy
accepts_nested_attributes_for :check, :reject_if => lambda { |a| a[:value].blank? },   :allow_destroy => true   

Previously, I’ve used this:

...
User.create!(:username => username, :check_attributes => [ {:attribute_name => "User-Password", :value => password, :op => ":="}])   
...

Since moving to a different method, I’ve now got this in my user model:

  def self.activerecord_extensions_mass_insert(validate = true)
        columns = [:username]
        values = []
         10000.times do
           username = ""
           5.times { username << (i = Kernel.rand(62); i += ((i < 10) ? 48 : ((i < 36) ? 55 : 61 ))).chr }
           values.push [username]
         end
        User.import columns, values, {:validate => validate}
  end

I’ve tried using this and a few other variations without success…

  columns = [:username, :check_attributes => [ :attribute_name, :value, :op]]

Any suggestions?

Answer 1

import won’t accept nesting so I think you are out of luck for that method.
Why not have two strings, user_sql and check_sql and then as you loop through manually build each sql statement for an extended insert. then you can run run two queries instead of a bunch.
on the plus side, you will shave off some more time from the indexing penalty of multiple queries.

edited to add code:
you build two sql statements at once so you can have them releate to each other via user_id, then at the end you can execute. If this stuff might ever make it to production, wrap the sql calls in a transaction so you don’t have any accidents.

user_sql = "INSERT into users (id,username,password) VALUES\n"
check_sql = "INSERT into check_attributes (user_id,foo,bar) VALUES\n"
random_data_pool = [('0'..'9'),('A'..'Z'),('a'..'z')].collect(&:to_a).flatten
max_loop = 10

1.upto(max_loop) do |i|
 seperator = (i == max_loop) ? ';' : ",\n"
 username = (1..5).map{ random_data_pool[Kernel.rand(random_data_pool.size)] }.join
 password = (1..5).map{ random_data_pool[Kernel.rand(random_data_pool.size)] }.join
 user_sql += "(#{i},'#{username}','#{password}')" + seperator
 check_sql +="(#{i},true,'potato')" + seperator
end


ActiveRecord::Base.connection.execute(user_sql)
ActiveRecord::Base.connection.execute(check_sql)