Question

I use Sphinx for fulltext search on my app. I use a view to filter the data from a products table and Sphinx indexes this view. Right now I have a hard coded field under the products table with it’s related tags, but that is not so good as tags might change and I would have to manipulate the hard coded field every time.

So I was thinking that I could create the view with GROUP_CONCAT, listing all the related tags for me with a syntax like this:

SELECT p.id AS id, GROUP_CONCAT(t.tag SEPARATOR '|') AS tags,
UNIX_TIMESTAMP(p.created) AS created
FROM products p
INNER JOIN products_tags pt ON pt.product_id = p.id
INNER JOIN tags t ON t.id = pt.tag_id
AND p.deleted = 0
AND (p.images IS NOT NULL OR p.images <> '')
AND p.status LIKE 'listed'
GROUP BY p.id;

The problem with this query is that it takes ages to run. It’s really slow. To retrieve only one record, it takes up to 5 seconds. This is the EXPLAIN output:

1, SIMPLE, pt, ALL, , , , , 165029, Using temporary; Using filesort
1, SIMPLE, p, eq_ref, PRIMARY, PRIMARY, 4, trych_default.pt.product_id, 1, Using where
1, SIMPLE, t, eq_ref, PRIMARY, PRIMARY, 4, trych_default.pt.tag_id, 1, 

I wonder if there is any way to improve the query or maybe a better solution to my problem. Thanks!

Answer 1

You’re probably missing an index. With the following schema, this same query hit the index for every table:

CREATE TABLE `products` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `deleted` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `status` varchar(255) NOT NULL DEFAULT 'listed',
  `images` varchar(255) DEFAULT NULL,
  `created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `tags` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `tag` varchar(255) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `products_tags` (
  `product_id` int(11) NOT NULL,
  `tag_id` int(11) NOT NULL DEFAULT '0',
  PRIMARY KEY (`product_id`,`tag_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Here are the results of the EXPLAIN query:

EXPLAIN SELECT p.id AS id, GROUP_CONCAT(t.tag SEPARATOR '|') AS tags,
UNIX_TIMESTAMP(p.created) AS created
FROM products p
INNER JOIN products_tags pt ON pt.product_id = p.id
INNER JOIN tags t ON t.id = pt.tag_id
WHERE p.deleted = 0
AND (p.images IS NOT NULL OR p.images <> '')
AND p.status LIKE 'listed'
GROUP BY p.id;

+----+-------------+-------+--------+---------------+---------+---------+----------------+------+-------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref            | rows | Extra       |
+----+-------------+-------+--------+---------------+---------+---------+----------------+------+-------------+
|  1 | SIMPLE      | p     | index  | PRIMARY       | PRIMARY | 4       | NULL           |    1 | Using where |
|  1 | SIMPLE      | pt    | ref    | PRIMARY       | PRIMARY | 4       | test.p.id      |    1 | Using index |
|  1 | SIMPLE      | t     | eq_ref | PRIMARY       | PRIMARY | 4       | test.pt.tag_id |    1 |             |
+----+-------------+-------+--------+---------------+---------+---------+----------------+------+-------------+
3 rows in set (0.00 sec)