Question

I have a query for a contact messaging system that is getting exponentially slow the more joins I do.

The table structure is basically a contact table, and a contact field table.

The query joins the contact field table many time, and for each join I do, it takes twice as long.

This is the query.

SELECT  SQL_CALC_FOUND_ROWS
    `contact_data`.`id`,
    `contact_data`.`name`,
    `fields0`.`value` AS `fields0`,
    `fields1`.`value` AS `fields1`,
    `fields2`.`value` AS `fields2`,
    ...etc...
    CONTACT_DATA_TAGS(
        GROUP_CONCAT(DISTINCT `contact_data_tags`.`name`),
        GROUP_CONCAT(DISTINCT `contact_data_assignment`.`user`),
        GROUP_CONCAT(DISTINCT `contact_data_read`.`user`)
    ) AS `tags`,
    GROUP_CONCAT(DISTINCT `contact_data_assignment`.`user`) AS `assignments`,
    `contact_data`.`updated`,
    `contact_data`.`created`
FROM
    `contact_data`
LEFT JOIN contact_data_tags ON contact_data.`id` = contact_data_tags.`data`
LEFT JOIN contact_data_assignment ON contact_data.`id` = contact_data_assignment.`data`
LEFT JOIN contact_data_read ON contact_data.`id` = contact_data_read.`data`
LEFT JOIN contact_data_fields AS fields0 ON contact_data.`id` = fields0.`contact_data_id` AND fields0.`key` = :field1
LEFT JOIN contact_data_fields AS fields1 ON contact_data.`id` = fields1.`contact_data_id` AND fields1.`key` = :field2
LEFT JOIN contact_data_fields AS fields2 ON contact_data.`id` = fields2.`contact_data_id` AND fields2.`key` = :field3
...etc...
GROUP BY contact_data.`id`
ORDER BY `id` DESC

This is the table structure:

CREATE TABLE IF NOT EXISTS `contact_data` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(200) NOT NULL,
  `format` varchar(50) NOT NULL,
  `fields` longtext NOT NULL,
  `url` varchar(2000) NOT NULL,
  `referer` varchar(2000) DEFAULT NULL,
  `ip` varchar(40) NOT NULL,
  `agent` varchar(1000) DEFAULT NULL,
  `created` datetime NOT NULL,
  `updated` datetime NOT NULL,
  `updater` int(10) unsigned DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `name` (`name`),
  KEY `url` (`url`(333)),
  KEY `ip` (`ip`),
  KEY `created` (`created`),
  KEY `updated` (`updated`),
  KEY `updater` (`updater`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `contact_data_assignment` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `user` int(10) unsigned NOT NULL,
  `data` int(10) unsigned NOT NULL,
  `created` datetime NOT NULL,
  `updater` int(10) unsigned DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `unique_assignment` (`user`,`data`),
  KEY `user` (`user`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `contact_data_fields` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `contact_data_id` int(10) unsigned NOT NULL,
  `key` varchar(200) NOT NULL,
  `value` text NOT NULL,
  `updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`),
  KEY `contact_data_id` (`contact_data_id`),
  KEY `key` (`key`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `contact_data_read` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `user` int(10) unsigned NOT NULL,
  `data` int(10) unsigned NOT NULL,
  `type` enum('admin','email') NOT NULL,
  `created` datetime NOT NULL,
  PRIMARY KEY (`id`),
  KEY `user` (`user`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `contact_data_tags` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(200) NOT NULL,
  `data` int(10) unsigned NOT NULL,
  `created` datetime NOT NULL,
  `updater` int(10) unsigned DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `unique_tag` (`name`,`data`),
  KEY `name` (`name`),
  KEY `data` (`data`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8;

DELIMITER $$
CREATE FUNCTION `contact_data_tags`(`tags` TEXT, `assigned` BOOL, `read` BOOL) RETURNS text CHARSET latin1
BEGIN
    RETURN CONCAT(
        ',',
        IFNULL(`tags`, ''),
        ',',
        IF(`tags` IS NULL OR FIND_IN_SET('Closed', `tags`) = 0, 'Open', ''),
        ',',
        IF(`assigned` IS NULL, 'Unassigned', ''),
        ',',
        IF(`read` IS NULL, 'New', ''),
        ','
    );
END$$

DELIMITER ;

Anyone know why it runs so slow? What can I do to make it faster? Do I need to adjust the query (I would prefer not adjust the structure)? Is there any config options I can set to speed it up?

Also strange is that it seems to work faster on my Windows development machine , compared to my Debain production server (almost instant, compared to 30+ seconds).

But the Windows machine is far less powerful than the Debain server (8 core Xeon, 32GB RAM).

Running MySQL 5.1.49 on Debian (which I can’t update), and 5.5.28 on Windows.

So reading that EAV does not perform well in RDBMS (or at least in my case), is the a config option that I could increase to make this run faster (i.e. can I just throw more RAM at it)?

Answer 1

One way to speed up the query would be to link to contact_data_fields once only (on contact_data.id = contact_data_fields.contact_data_id) and change the fields columns to be max expressions – like so:

SELECT  SQL_CALC_FOUND_ROWS
    `contact_data`.`id`,
    `contact_data`.`name`,
    MAX(CASE WHEN fields.`key` = :field1 THEN fields.`value` END) AS `fields0`,
    MAX(CASE WHEN fields.`key` = :field2 THEN fields.`value` END) AS `fields1`,
    MAX(CASE WHEN fields.`key` = :field3 THEN fields.`value` END) AS `fields2`,
    ...etc...
    CONTACT_DATA_TAGS(
        GROUP_CONCAT(DISTINCT `contact_data_tags`.`name`),
        GROUP_CONCAT(DISTINCT `contact_data_assignment`.`user`),
        GROUP_CONCAT(DISTINCT `contact_data_read`.`user`)
    ) AS `tags`,
    GROUP_CONCAT(DISTINCT `contact_data_assignment`.`user`) AS `assignments`,
    `contact_data`.`updated`,
    `contact_data`.`created`
FROM
    `contact_data`
LEFT JOIN contact_data_tags ON contact_data.`id` = contact_data_tags.`data`
LEFT JOIN contact_data_assignment ON contact_data.`id` = contact_data_assignment.`data`
LEFT JOIN contact_data_read ON contact_data.`id` = contact_data_read.`data`
LEFT JOIN contact_data_fields AS fields
       ON contact_data.`id` = fields.`contact_data_id` 
...etc...
GROUP BY contact_data.`id`
ORDER BY `id` DESC