Question

I’ve just saw a very slow query in my application. Table ‘News’ has more than 600.000 records.

When I execute:

SELECT news.id FROM `news` WHERE (newstime between '2012-01-16 00:00:00' AND  '2012-01-16 23:59:59') AND ((MATCH(titolo, testo) AGAINST('"Public Administration" "SOMETHING" "ELSE" "ROMA" "MILANO"' IN BOOLEAN MODE))) ORDER BY newstime DESC LIMIT 23 OFFSET 0;
23 rows in set (26.32 sec)

For some reason MySQL is not performing range selection (there are only 10.000 records for each day), it looks like it’s searching on the whole table because when I select from subquery:

 SELECT id FROM(SELECT * from news where newstime between '2012-01-16 00:00:00' AND  '2012-01-16 23:59:59') as N where ((MATCH(titolo, testo) AGAINST('"Public Administration" "FIAT" "SOMETHING" "ELSE" "ROMA" "MILANO"' IN BOOLEAN MODE))) ORDER BY newstime DESC LIMIT 23 OFFSET 0;

23 rows in set (0.09 sec)

My query returns after 0.09 second!

26 -> 0.09 second.

I thought that MySQL will be intelligent enough to select the range of records by newstime and then do fulltext searching but it looks like it’s not the case. Is it normal? Or should I try to still optimize the first query? When I write explain #1 it is telling:

mysql> explain SELECT news.id FROM `news` WHERE (newstime between '2012-01-16 00:00:00' AND  '2012-01-16 23:59:59') AND ((MATCH(titolo, testo) AGAINST('"Public Administration" "FIAT" "SOMETHING" "ELSE" "ROMA" "MILANO"' IN BOOLEAN MODE))) ORDER BY newstime DESC LIMIT 23 OFFSET 0;
+----+-------------+-------+----------+--------------------------------+---------+---------+------+------+-----------------------------+
| id | select_type | table | type     | possible_keys                  | key     | key_len | ref  | rows | Extra                       |
+----+-------------+-------+----------+--------------------------------+---------+---------+------+------+-----------------------------+
|  1 | SIMPLE      | news  | fulltext | index_news_on_newstime,alltext | alltext | 0       |      |    1 | Using where; Using filesort |
+----+-------------+-------+----------+--------------------------------+---------+---------+------+------+-----------------------------+

Why is is using the fulltext key and not newstime?

Show create table news is as following:

 CREATE TABLE `news` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `titolo` varchar(255) DEFAULT NULL,
  `testo` mediumtext,
  `newstime` datetime NOT NULL,
  `created_at` datetime DEFAULT NULL
  PRIMARY KEY (`id`),
  KEY `index_news_on_newstime` (`newstime`),
  FULLTEXT KEY `alltext` (`titolo`,`testo`)
) ENGINE=MyISAM AUTO_INCREMENT=1846714 DEFAULT CHARSET=utf8 |

Why is that?

Answer 1

The answer to “why?” is just that the MySQL query optimizer is not perfect. Sometimes it chooses the less optimal index. You can correct for this by telling MySQL which index to use with index hints

SELECT news.id 
FROM `news` 
USE INDEX (index_news_on_newstime) 
WHERE (newstime between '2012-01-16 00:00:00' AND  '2012-01-16 23:59:59') 
  AND ((MATCH(titolo, testo) AGAINST('"Public Administration" "SOMETHING" "ELSE" "ROMA" "MILANO"' IN BOOLEAN MODE))) 
ORDER BY newstime DESC LIMIT 23 OFFSET 0;

From the docs:
“You can also use FORCE INDEX, which acts like USE INDEX (index_list) but with the addition that a table scan is assumed to be very expensive. In other words, a table scan is used only if there is no way to use one of the given indexes to find rows in the table.”

SELECT news.id 
FROM `news` 
FORCE INDEX (index_news_on_newstime) 
WHERE (newstime between '2012-01-16 00:00:00' AND  '2012-01-16 23:59:59') 
  AND ((MATCH(titolo, testo) AGAINST('"Public Administration" "SOMETHING" "ELSE" "ROMA" "MILANO"' IN BOOLEAN MODE))) 
ORDER BY newstime DESC LIMIT 23 OFFSET 0;