Question

I have two tables that have a foreign key constraint between them

Table event
mysql> describe event;
+------------+------------------+------+-----+---------+-------+
| Field      | Type             | Null | Key | Default | Extra |
+------------+------------------+------+-----+---------+-------+
| sid        | int(10) unsigned | NO   | PRI | NULL    |       |
| cid        | int(10) unsigned | NO   | PRI | NULL    |       |
| signature  | int(10) unsigned | NO   | MUL | NULL    |       |
| timestamp  | datetime         | NO   | MUL | NULL    |       |
| is_deleted | tinyint(1)       | NO   | MUL | 0       |       |
+------------+------------------+------+-----+---------+-------+
5 rows in set (0.00 sec)

Table signature
mysql> describe signature;
+--------------+------------------+------+-----+---------+----------------+
| Field        | Type             | Null | Key | Default | Extra          |
+--------------+------------------+------+-----+---------+----------------+
| sig_id       | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| sig_name     | varchar(255)     | NO   | MUL | NULL    |                |
| sig_class_id | int(10) unsigned | NO   | MUL | NULL    |                |
| sig_priority | int(10) unsigned | YES  |     | NULL    |                |
| sig_rev      | int(10) unsigned | YES  |     | NULL    |                |
| sig_sid      | int(10) unsigned | YES  |     | NULL    |                |
| sig_gid      | int(10) unsigned | YES  |     | NULL    |                |
+--------------+------------------+------+-----+---------+----------------+
7 rows in set (0.00 sec)

event.signature is a foreign key and links to signature.sig_id. Both have indexes as well

Table event is large(say 1M records) while table signature will be comparatively small (Few thousand at most)

Joined Queries that access any signature attribute take a very long time to execute. A look at explain

mysql> explain select event.sid,event.cid,signature.sig_name from event join signature on signature.sig_id=event.signature;
+----+-------------+-----------+------+--------------------------------+-----------------------+---------+-------------------------+------+--------------------------+
| id | select_type | table     | type | possible_keys                  | key                   | key_len | ref                     | rows | Extra                    |
+----+-------------+-----------+------+--------------------------------+-----------------------+---------+-------------------------+------+--------------------------+
|  1 | SIMPLE      | signature | ALL  | PRIMARY,index_signature_sig_id | NULL                  | NULL    | NULL                    |  127 |                          |
|  1 | SIMPLE      | event     | ref  | index_event_signature          | index_event_signature | 5       | snorby.signature.sig_id |   68 | Using where; Using index |
+----+-------------+-----------+------+--------------------------------+-----------------------+---------+-------------------------+------+--------------------------+
2 rows in set (0.00 sec)

While if no signature attribute is accessed

mysql> explain select event.sid,event.cid from event join signature on signature.sig_id=event.signature;
+----+-------------+-----------+-------+--------------------------------+------------------------+---------+-------------------------+------+--------------------------+
| id | select_type | table     | type  | possible_keys                  | key                    | key_len | ref                     | rows | Extra                    |
+----+-------------+-----------+-------+--------------------------------+------------------------+---------+-------------------------+------+--------------------------+
|  1 | SIMPLE      | signature | index | PRIMARY,index_signature_sig_id | index_signature_sig_id | 4       | NULL                    |  127 | Using index              |
|  1 | SIMPLE      | event     | ref   | index_event_signature          | index_event_signature  | 5       | snorby.signature.sig_id |   68 | Using where; Using index |
+----+-------------+-----------+-------+--------------------------------+------------------------+---------+-------------------------+------+--------------------------+
2 rows in set (0.00 sec)

As can be seen if Signature attribute is queried it does a full scan with ALL join type.

Is it possible to rewrite the query to be faster? I ask this because this is a part of multiple table join and joining event with signature is the bottleneck that is slowing down the query tremendously

I am using 5.1.52 MySQL and SQLAlchemy 0.7.8 as ORM

Answer 1

Your query does require a full scan, by definition.

That is, you give no filtering condition. No ... WHERE sig_rev = 17, for example.

Therefore, there is not much to improve here. MySQL picks a table to start with, does a full scan, and per row fetches matching rows from second table.

So the scan is essential. But you may turn it into an index scan instead of a table scan. I am assuming you have an index on the signature column only, and on the sig_id column only.

What you may do is create an additional index on sig_id, sig_name, like this:

ALTER TABLE signature ADD UNIQUE INDEX(sig_id, sig_name);

The index is unique by definition, since it is broader than the PRIMARY KEY, but this is outside the point.

What you may gain now is an execution plan similar to the second example you have posted: an index scan on signature, followed by an index lookup on event.

Make sure to compare and verify that you do get performance boost on this particular query. Check that the new index does not hurt INSERT performance etc.

Good luck.