I’m not really sure where to start with cutting down this query in the search section of my site so that it doesn’t take so long. You run a search on a variety of tables, and get filtered results back mainly from the “Items” table, and currently it (and searched like it) take sometimes over 10 seconds. Part of the problem is that the php code that composes the sql is dirty as hell, of course, and I’m not used to having huge queries that I need to optimize in general, but what techniques can I use to determine where to add indexes in a query like this?
The query
SELECT
items.ItemId,
items.Name,
items.BrandCode,
items.BrandCategoryId,
items.CatalogPage,
items.PriceRetail,
items.PriceSell,
items.PriceHold,
items.Descr,
items.GenderId,
products.ImagetnURL,
products.FlagDefault,
products.ProductId,
products.Code AS ProductCode,
products.Name AS ProductName,
brands.Name AS BrandName,
items.FlagStatus AS ItemFlagStatus
FROM
items,
products,
brands,
productsizes,
searchsizechartsizes,
sizechartsizes
WHERE
items.ItemId = products.ItemId AND
items.BrandCode = brands.Code AND
items.FlagStatus != 'U' AND
products.FlagStatus != 'U' AND
items.TypeId = '10' AND
searchsizechartsizes.SearchSizeChartId = '11' AND
searchsizechartsizes.Size = sizechartsizes.Size AND
sizechartsizes.SizeChartId = productsizes.SizeChartId AND
productsizes.ProductId = products.ProductId
GROUP BY
items.ItemId
ORDER BY
items.Name
LIMIT
0, 15;
Logs show 14 second execution time, and 3 million rows examined
# Query_time: 14 Lock_time: 0 Rows_sent: 15 Rows_examined: 2901565
Summary of query results
+--------+----------------------+-----------+-----------------+-------------+-------------+-----------+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------------------------+-------------+-----------+-------------+------------------------------------------+-------------------+----------------+
| ItemId | Name | BrandCode | BrandCategoryId | CatalogPage | PriceRetail | PriceSell | PriceHold | Descr | GenderId | ImagetnURL | FlagDefault | ProductId | ProductCode | ProductName | BrandName | ItemFlagStatus |
+--------+----------------------+-----------+-----------------+-------------+-------------+-
| 3376 | 10-inch Pull On Boot | RW | 2801 | 24 | 189.99 | 189.99 | 189.99 | Full grain brown leather - blah blah blah | 1 | images/rw/rw-2249tn.jpg | Y | 4345 | 2249 | Brown Full Grain Turbo Vegas Leather | Red Wing Work | A |
| 9340 | 11 | RR | NULL | 1 | 300.00 | 300.00 | 300.00 | The Engineer 11" boot from Red Wing Shoes� blah blah blah... | 1 | images/rr/rr-2990tn.jpg | Y | 16749 | 2990 | Black Harness Calfskin | Red Wing Heritage | A
~~~~~~ other results here redacted for space reasons ~~~~~~
15 rows in set (13.33 sec)
Explain of the query
mysql> explain SELECT items.ItemId, items.Name, items.BrandCode, items.BrandCategoryId, items.CatalogPage, items.PriceRetail, items.PriceSell, items.PriceHold, items.Descr, items.GenderId, products.ImagetnURL, products.FlagDefault, products.ProductId, products.Code as ProductCode, products.Name as ProductName, brands.Name as BrandName, items.FlagStatus as ItemFlagStatus FROM items, products, brands, productsizes, searchsizechartsizes, sizechartsizes WHERE items.ItemId = products.ItemId AND items.BrandCode = brands.Code AND items.FlagStatus != 'U' AND products.FlagStatus != 'U' AND items.TypeId = '10' AND (searchsizechartsizes.SearchSizeChartId = '11' AND searchsizechartsizes.Size = sizechartsizes.Size AND sizechartsizes.SizeChartId = productsizes.SizeChartId AND productsizes.ProductId = products.ProductId) group by items.ItemId ORDER BY items.Name LIMIT 0, 15;
+----+-------------+----------------------+--------+------------------------------------------------+-------------+---------+----------------------------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------------+--------+------------------------------------------------+-------------+---------+----------------------------------------+------+---------------------------------+
| 1 | SIMPLE | searchsizechartsizes | ref | PRIMARY,Size | PRIMARY | 4 | const | 2 | Using temporary; Using filesort |
| 1 | SIMPLE | brands | ALL | NULL | NULL | NULL | NULL | 40 | |
| 1 | SIMPLE | sizechartsizes | ref | Size,SizeChartId | Size | 33 | shermanbros.searchsizechartsizes.Size | 217 | Using where |
| 1 | SIMPLE | productsizes | ref | ProductId,SizeChartId | SizeChartId | 5 | shermanbros.sizechartsizes.SizeChartId | 17 | Using where |
| 1 | SIMPLE | products | eq_ref | PRIMARY,FlagStatus,flagstatusanddefault,ItemId | PRIMARY | 4 | shermanbros.productsizes.ProductId | 1 | Using where |
| 1 | SIMPLE | items | eq_ref | PRIMARY,BrandCode,TypeId,FlagStatus,ItemsIndex | PRIMARY | 4 | shermanbros.products.ItemId | 1 | Using where |
+----+-------------+----------------------+--------+------------------------------------------------+-------------+---------+----------------------------------------+------+---------------------------------+
6 rows in set (0.02 sec)
Structure of the items table
mysql> show create table items;
+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| items | CREATE TABLE `items` (
`ItemId` int(11) NOT NULL auto_increment,
`Code` varchar(25) default NULL,
`Name` varchar(100) default NULL,
`BrandCode` char(2) default NULL,
`CatalogPage` int(3) default NULL,
`BrandCategoryId` int(11) default NULL,
`TypeId` int(11) default NULL,
`StyleId` int(11) default NULL,
`GenderId` int(11) default NULL,
`PriceRetail` decimal(6,2) default NULL,
`PriceSell` decimal(6,2) default NULL,
`PriceHold` decimal(6,2) default NULL,
`Cost` decimal(6,2) default NULL,
`PriceNote` longtext,
`FlagTaxable` char(1) default NULL,
`FlagStatus` char(1) default NULL,
`FlagFeatured` char(1) default NULL,
`MaintFlagStatus` char(1) default NULL,
`Descr` longtext,
`DescrNote` longtext,
`ImagetnURL` varchar(50) default NULL,
`ImagefsURL` varchar(50) default NULL,
`ImagelsURL` varchar(50) default NULL,
`DateCreated` date NOT NULL default '0000-00-00',
`DateStatus` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
`AdminNote` text,
PRIMARY KEY (`ItemId`),
KEY `BrandCode` (`BrandCode`),
KEY `Name` (`Name`),
KEY `TypeId` (`TypeId`),
KEY `StyleId` (`StyleId`),
KEY `GenderId` (`GenderId`),
KEY `FlagStatus` (`FlagStatus`),
KEY `ItemsIndex` (`TypeId`,`FlagStatus`,`ItemId`)
) ENGINE=MyISAM AUTO_INCREMENT=10216 DEFAULT CHARSET=latin1 |
+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
Existing indexes on the items table
mysql> show indexes from items;
+-------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| items | 0 | PRIMARY | 1 | ItemId | A | 8678 | NULL | NULL | | BTREE | |
| items | 1 | BrandCode | 1 | BrandCode | A | 36 | NULL | NULL | YES | BTREE | |
| items | 1 | Name | 1 | Name | A | 8678 | NULL | NULL | YES | BTREE | |
| items | 1 | TypeId | 1 | TypeId | A | 17 | NULL | NULL | YES | BTREE | |
| items | 1 | StyleId | 1 | StyleId | A | 41 | NULL | NULL | YES | BTREE | |
| items | 1 | GenderId | 1 | GenderId | A | 3 | NULL | NULL | YES | BTREE | |
| items | 1 | FlagStatus | 1 | FlagStatus | A | 6 | NULL | NULL | YES | BTREE | |
| items | 1 | ItemsIndex | 1 | TypeId | A | 17 | NULL | NULL | YES | BTREE | |
| items | 1 | ItemsIndex | 2 | FlagStatus | A | 52 | NULL | NULL | YES | BTREE | |
| items | 1 | ItemsIndex | 3 | ItemId | A | 8678 | NULL | NULL | | BTREE | |
+-------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
10 rows in set (0.00 sec)
It seems to have an index on various columns already, but those indexes still leave it slow with regards to this search query. And one multi-index on typeid, flagstats, itemid that’s there to optimize a seperate query.
And yeah, as you can see, the items table itself is a beast, which probably doesn’t help.
Looking at your FROM and WHERE clauses:
you have put all of the join logic into the WHERE clause, which is unhelpful when reading the query. Using explicit join syntax instead, we can rewrite as the following:
This makes it more clear what is going on.
It seems like you have a good candidate index for this query in ItemsIndex. (By the way – the index on TypeID alone is redundant and should be dropped, because TypeID is a left prefix of ItemsIndex.) Unfortunately MySQL’s query optimiser seems to think it makes more sense to do the join back-to-front, starting with table searchsizechartsizes. I’m not sure why it would do this, but I notice that table brands apparently has no usable keys defined on it. It might be a good idea to look into this.