Question

Update:

Originally, this post was using Books as the example entity, with
Books1, Books2, etc. being the
separated table. I think this was a
bit confusing, so I’ve changed the
example entity to be “private
todo_items created by a particular
user.”

This kind of makes Horace and Ryan’s original comments seem a bit off, and
I apologize for that. Please know that
their points were valid when it looked
like I was dealing with books.

Hello,

I’ve decided to use multiple tables for an entity (e.g. todo_items1, todo_items2, todo_items3, etc.), instead of just one main table which could end up having a lot of rows (e.g. just todo_items). I’m doing this to try and to avoid a potential future performance drop that could come with having too many rows in one table.

With that, I’m looking for a good way to handle this in Rails, mainly by trying to avoid loading a bunch of unused associations for each User object. I’m guessing that other have done something similar, so there’s probably a few good tips/recommendations out there.

(I know that I could use a partition for this, but, for now, I’ve decided to go the ‘multiple tables’ route.)

Each user has their todo_items placed into a specific table. The actual “todo items” table is chosen when the user is created, and all of their todo_items go into the same table. The data in their todo items collection is private, so when it comes time to process a users todo_items, I’ll only have to look at one table.

One thing I don’t particularly want to have is a bunch of unused associations in the User class. Right now, it looks like I’d have to do the following:

class User < ActiveRecord::Base
  has_many :todo_items1, :todo_items2, :todo_items3, :todo_items4, :todo_items5
end

class todo_items1 < ActiveRecord::Base
  belongs_to :user
end

class todo_items2 < ActiveRecord::Base
  belongs_to :user
end

class todo_items3 < ActiveRecord::Base
  belongs_to :user
end

The thing is, for each individual user, only one of the “todo items” tables would be usable/applicable/accessible since all of a user’s todo_items are stored in the same table. This means only one of the associations would be in use at any time and all of the other has_many :todo_itemsX associations that were loaded would be a waste.

For example, with a user.id of 2, I’d only need todo_items3.find_by_text('search_word'), but the way I’m thinking of setting this up, I’d still have access to todo_items1, todo_items2, todo_items4 and todo_items5.

I’m thinking that these “extra associations” adds extra overhead and makes each User object’s size in memory much bigger than it has to be. Also, there’s a bunch of stuff that Ruby/Rails is doing in the background which may cause other performance problems.

I’m also guessing that there could be some additional method call/lookup overhead for each User object, since it has to load all of those associations, which in turn creates all of those nice, dynamic model accessor methods like User.find_by_something.

I don’t really know Ruby/Rails does internally with all of those has_many associations though, so maybe it’s not so bad. But right now I’m thinking that it’s really wasteful, and that there may just be a better, more efficient way of doing this.

So, a few questions:

1) Is there’s some sort of special Ruby/Rails methodology that could be applied to this ‘multiple tables to represent one entity’ scheme? Are there any ‘best practices’ for this?

2) Is it really bad to have so many unused has_many associations for each object? Is there a better way to do this?

3) Does anyone have any advice on how to abstract the fact that there’s multiple “todo items” tables behind a single todo_items model/class? For example, so I can call todo_items.find_by_text('search_phrase') instead of todo_items3.find_by_text('search_phrase').

Thank you!

Answer 1

This is not the way to scale.

It would probably be better going with master-slave replication and proper indexing (besides primary key) on fields such as “title” and/or “author” if that’s what you’re going to be looking up books based on. Having it in n-tables, how are you going to know the best place to go looking for the book the user is after? Are you going to go looking through 4 tables?

I agree with Horace: ” don’t try to solve a performance issue before you have figures to prove it.” I suggest, however, that you should really look into adding indexes to your table if you want lookups to be fast. If they aren’t fast, then tell us how they aren’t fast and we will tell you how to make it go ZOOOOOM.