Question

In certain cases when I get an ActiveRecord Relation I’m experiencing strange behavior with .each on an ActiveRecord::Relation

It seems to be when ActiveRecord::Relation delegates :each to :to => :to_a (source)

@tasks = Task.find_task(list, {:week_id => 1})

Basically, there’s a lengthy class method that takes an object (list, and a hash with a :week_id)

A bunch of filtering & queries happen within this find_task method, but it ends up returning a relation to @tasks

Then, in the template, I have:

<% @tasks.each do |task| %>
.
.
.
<% end %>

For whatever reason, no matter the size of @tasks, it takes ~3 minutes. I Can replicate this same behavior by calling @tasks.to_a Even if @tasks, an instance of ActiveRecord::Relation is only two records, calling to_a on them takes > 3 minutes.

It doesn’t happen on all :week_ids, only on a specific week_id, for example: :week_id => 1

The SQL executes fine and I get a relation back, it just seems to be a problem with enumerable on a specific ActiveRecord::Relation.

Update

Inside the algorithm (I think this means the class method) I do a LOT of eager loading. So Postgres does a lot of LEFT OUTER JOINs and I’ve indexed all the tables that this needs to happen on.

An explain analyze shows that all scans are index scans and as it turns out the query executes just fine with a lot of eager loading… and I get an eager loaded ‘ActiveRecord::Relation` back in a reasonable amount of time.

Update 2
While this process is taking 3 minutes I see a postgres process run for a few seconds, and then I see this for 3 minutes as my output in top:

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+    COMMAND                                                            
 8685 dylan     20   0 3407m 2.6g  904 R 99.7 69.1   1:14.49 /usr/local/bin/ruby script/rails s

When it finally finishes, The server shows this;

• eager loading: 200 ms, 139 ms then
• a big SQL Query with lots of LEFT OUTER JOINS in 33,000 ms, (long, but not where the majority is) then
• 257,000 ms in the template.

And when I replicate the behavior in the template I see that it takes ~3-4 minutes to call to_a on the @tasks Relation.

So, when my server tells me all that time is spent in the template, and I can see calling an enumerable on the relation takes forever, is that when the query gets executed? Even though in top I can only see the ruby process running?

Answer 1

The problem was with the eager loading. When calling an enumerable method on an ActiveRecord::Relation instance it gets delegated to .to_a, which can take a really long time with a huge set of relations. Even though I was looping through @tasks, I had eager loaded so many objects that .to_a was taking too long.

My short-term fix is to simply eager load fewer objects, even though it ends up hurting me with n+1 queries.