Question

I have a very large table (millions of records) containing approximately 8 fields as a primary key. for simplicities sake lets say that the table looks like this:

    key_1 | key_2 | key_3 | ... | key_8 | value

given a value for key_1, I need to fetch all possible values for key_2, key_3, …, key_8
something along the following lines:

    SELECT DISTINCT key_2 FROM table1 WHERE key_1 = 123;
    SELECT DISTINCT key_3 FROM table1 WHERE key_1 = 123;
    ...
    SELECT DISTINCT key_8 FROM table1 WHERE key_1 = 123;

My problem is that this query is significantly slower then my performance needs, and the data in this table is fairly constant and rarely updated(once every few days). Also table_1 could be a slow sub-query. Short of creating an additional table in the database and manually updating it every time the database is updated, is there another solution that can give me fast results. I would need it to work across multiple MySQL Sessions.

Answer 1

Can’t give a definitive answer with the information we have, but let’s start with these:

Do you have an index on key_1?

Without it, each query by itself will already be slow just looking for 123.

Do you have an index on (key_1, key_2)?

Because select distinct key_2 where key_1 = 123 is really fast if it can get all the necessary data from the index alone. No need to access the table.

Are the rows/indexes fixed-size?

Traversing a fixed-size table/row can be faster because one always knows where the x-th record is by just calculating the offset. Variable row sized tables are slower.

Have you tried adding an autoincrement surrogate primary key?

Indexes work way better when all they have to store is the column, and a small primary key. Composite primary keys are slower.

Did you consider a read-only table?

You can pack myisam table for fast access, but they become read-only. It’s a hack that has its uses though.

One step further, have you considered a datawarehouse?

If the tables don’t change often, it might be best to duplicate the information for fast access.

Can you post a show create table statement? Seeing the columns and indexes would help.
Can you post an explain select statement? Seeing which indexes are used would help.