Question

I have a Ruby on Rails app with a PostgreSQL database which has this structure:

class A < ActiveRecord::Base
   has_many :B
end
class B < ActiveRecord::Base
    has_many :C
end
class C < ActiveRecord::Base
    attr_accessible :x, :y :z
end

The are only a few A’s, and they grow slowly (say 5 a month). Each A has thousands of B’s, and each B has tens of thousands of C’s (so each A has millions of C’s).

A’s are independent and B’s and C’s from different A’s will never be needed together (i.e. in the same query).

My problem is that now that I have only a couple of A’s, ActiveRecord queries take pretty long. When the table for C has tens of millions of rows, queries will take forever.

I am thinking about scaling the database horizontally (i.e a table for A’s, one table of B’s and one table of C’s for each A). But I don’t know how to do it. It is a kind of sharding i guess, but I can’t figure out how to create DB tables dynamically and use ActiveRecord to access the data if the table depends on which A im working with.

Thank you very much.

Answer 1

If you have performance concerns with only a few rows, or even with several million rows, you need to take a step back before trying to atmosphere engineer a solution. The problem you are describing is very easily solved by indexing; there is no advantage to creating additional physical tables and you’d be introducing incredible complexity.

As @mu-is-too-short already stated: pay attention to your query plans. Use your tools to analyze performance.

That being said you can use table partitioning to physically and transparently house the storage of data into different sharded tables which is especially useful for data that grows very fast but is only useful in a given time box (like a month). You can also do this with an archive bit flag column to shuttle old or deleted records onto some slower storage (say, standard RAID comprised of spinning rust) while keeping active records on faster storage (like a RAID of SSDs).