Question

I am running a query joining multiple tables through a date range search and stuck trying to figure out how to further optimize it.

SELECT ACC.name AS account_name, CAMP.account_id AS account_id,CAMP.name AS campaign_name,CAMP.id AS campaign_id,ADG.id AS adgroup_id,ADG.name AS adgroup_name,KW.text AS keyword_name,
SUM(SPENT.billed_clicks) AS billed_clicks,KW.id AS keyword_id,KW.status_id AS status_id FROM account ACC, campaign CAMP,adgroup ADG,adgroup_keyword KW INNER JOIN keyword_spent SPENT
ON KW.id = SPENT.keyword_id WHERE     summary_date >= '2012-03-01' AND summary_date <= '2012-03-04' AND KW.adgroup_id = ADG.id AND ADG.campaign_id = CAMP.id AND CAMP.account_id = ACC.id
GROUP BY keyword_id

The EXPLAIN on this yields the following –

+----+-------------+-------+--------+----------------------------+--------------+---------+---------------------------------+--------+----------------------------------------------+
| id | select_type | table | type   | possible_keys              | key          | key_len | ref                             | rows   | Extra                                        |
+----+-------------+-------+--------+----------------------------+--------------+---------+---------------------------------+--------+----------------------------------------------+
|  1 | SIMPLE      | SPENT | range  | summary_date               | summary_date | 3       | NULL                            | 752191 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | KW    | eq_ref | PRIMARY,FK1948D0E6ED3A5544 | PRIMARY      | 8       | clicksummarydb.SPENT.keyword_id |      1 |                                              | 
|  1 | SIMPLE      | ADG   | eq_ref | PRIMARY,FKBBC2083C29112FD0 | PRIMARY      | 8       | advertisedb.KW.adgroup_id       |      1 |                                              | 
|  1 | SIMPLE      | CAMP  | eq_ref | PRIMARY,FKF7A90110246F33C4 | PRIMARY      | 8       | advertisedb.ADG.campaign_id     |      1 |                                              | 
|  1 | SIMPLE      | ACC   | eq_ref | PRIMARY                    | PRIMARY      | 8       | advertisedb.CAMP.account_id     |      1 |                                              | 
+----+-------------+-------+--------+----------------------------+--------------+---------+---------------------------------+--------+----------------------------------------------+

The keyword_spent table contains more than 1.5 million rows and here is the show create table on it

 | keyword_spent | CREATE TABLE `keyword_spent` (
   `id` bigint(20) NOT NULL auto_increment,
   `summary_date` date NOT NULL,
   `adgroup_id` bigint(20) NOT NULL,
   `keyword_id` bigint(20) NOT NULL,
   `billed_clicks` int(11) default NULL,
   `un_billed_clicks` int(11) default NULL,
   `spent` decimal(20,5) default NULL,
   `last_click_recno` bigint(20) default NULL,
   `campaign_id` bigint(20) NOT NULL,
   `account_id` bigint(20) NOT NULL,
   `total_convs` bigint(20) unsigned default '0',
    PRIMARY KEY  (`id`),
   UNIQUE KEY `keyword_spent_uniq` (`summary_date`,`adgroup_id`,`keyword_id`),
   KEY `idx_account_id` (`account_id`),
   KEY `idx_kw_id` (`keyword_id`),
   KEY `adgroup_id` (`adgroup_id`),
   KEY `campaign_id` (`campaign_id`),
   KEY `summary_date` (`summary_date`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 | 

I don’t understand why near 750,000 rows are being scanned when there are no more than 100,000 records in that date range.

Also, why is it doing a filesort instead of using indexes. ?

Answer 1

File sorts aren’t necessarily bad. As shown in Baron Schwartz’s blog post, file sorts aren’t necessarily about files. It’s just a quick sort that is used when there are no valid indexes available.

As an idea for how to optimize: perhaps have all of the aggregate data be in its own subquery, and join that data? I’m thinking something like this (make adjustments as needed):

SELECT ACC.name AS account_name,
CAMP.account_id AS account_id,
CAMP.name AS campaign_name,
CAMP.id AS campaign_id,
ADG.id AS adgroup_id,
ADG.name AS adgroup_name,
KW.text AS keyword_name,
KW.id AS keyword_id,
JOINED.billed_clicks AS billed_clicks,
JOINED.un_billed_clicks AS un_billed_clicks,
JOINED.total_clicks AS total_clicks,
JOINED.spent AS spent,
JOINED.total_convs AS total_convs
FROM account ACC
INNER JOIN campaign CAMP ON ACC.id = CAMP.account_id
INNER JOIN adgroup ADG ON CAMP.id = ADG.campaign_id
INNER JOIN adgroup_keyword KW ON ADG.id = KW.adgroup_id
INNER JOIN (SELECT
    SUM(billed_clicks) AS billed_clicks,
    SUM(un_billed_clicks) AS un_billed_clicks,
    SUM(billed_clicks) + SUM(un_billed_clicks) AS total_clicks,
    SUM(spent) AS spent,
    SUM(total_convs) AS total_convs,
    id AS keyword_id
    FROM keyword_spent
    GROUP BY keyword_id
) JOINED ON JOINED.keyword_id = KW.id

Hopefully I’m understanding this right. There is one benefit to this solution: the group by/aggregates are kept separate, and you don’t have to worry about group by-ing the other columns, which you never did in the original example.