Question

I have a relatively large table (5,208,387 rows, 400mb data/670mb index),
all columns i use to search with are indexes.
name and type are VARCHAR(255) BTREE INDEX
and sdate is an INTEGER column containing timestamps.

I fail to understand some issues,
first this query is very slow (5sec):

SELECT *
FROM `mytable`
WHERE `name`  LIKE 'hello%my%big%text%thing%'
AND `type` LIKE '%'
ORDER BY `sdate` DESC LIMIT 3

EXPLAIN for the above:

id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   SIMPLE      mytable range   name        name    257 NULL    5191    Using where

while this one is very fast (5msec):

SELECT *
FROM `mytable`
WHERE `name`  LIKE 'hello.my%big%text%thing%'
AND `type` LIKE '%'
ORDER BY `sdate` DESC LIMIT 3

EXPLAIN for the above:

id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   SIMPLE      mytable range   name        name    257 NULL    204 Using where

the amount of rows scanned different makes sense because of the indexes,
but having 5k of indexed rows take 5 seconds seems way too much.

also, ordering by name instead of sdate makes the queries very fast, but I need to order by the timestamp.

Second thing I do not understand is that before
adding the last column to the index,
the db had index of 1.4GB,
not after running an OPTIMIZE/REPAIR the size is just 670MB.

Answer 1

The problem is, only the portion before the first % can take advantage of the index, the rest of the like strings needs to process all rows which match hello% or hello.my% without the help of one. Also, ordering by another column then the index used, probably requires a second pass, or at least a scan rather then an already sorted index. Options to better performance (can be implemented independently from each other) are:

1. Using a full-text index on the name column and using a MATCH() AGAINST() search rather then LIKE with %‘s.
2. Adding the sdate to in index combined (name,sdate) could very well speed up sorting.