Question

Can someone help me please or explain to me on how to make my query faster ? This query take almost 10seconds (on my local machine) with almost 1GB data.

Here is my explain and explain resuls

    explain select p.delivery_date, p.delivery_hour, p.resource_id, p.participant_id, p.price, p.date_posted, hour(p.date_posted) as hour_date_posted, date(p.date_posted) as date_date_posted
    ,s.mw
from prices_report as p
left join schedules_report s
on s.delivery_date = p.delivery_date
                    AND s.type_id = p.type_id
            and s.delivery_hour = p.delivery_hour
                    and s.resource_id = p.resource_id
                    and s.participant_id = p.participant_id
                    and hour(s.date_posted) = hour(p.date_posted)
                    and date(s.date_posted) = date(p.date_posted)
WHERE p.delivery_date = '2012-05-22'
AND p.type_id = 'GEN'
ORDER BY p.delivery_date, p.resource_id, p.delivery_hour, p.participant_id, p.type_id, p.date_posted

explain results :
id : 1
select type : simple
table : p
type : ref
possible keys : idx1
key : idx1
key_len : 4
ref : const
rows : 40258
extra : using where

id : 1
select type : simple
table : s
type : ref
possible keys : idx1
key : idx1
key_len : 63
ref : const,APC_DB.p.delivery_hour,APC_DB.p.participant_id,APC_DB.p.resource_id,const
rows : 99
extra :

table structure :

  CREATE TABLE `prices_report` (
  `id` int(11) NOT NULL auto_increment,
  `delivery_date` date default NULL,
  `delivery_hour` int(2) default NULL,
  `participant_id` varchar(10) default NULL,
  `resource_id` varchar(15) default NULL,
  `type_id` varchar(10) default NULL,
  `price` float default NULL,
  `date_posted` datetime NOT NULL,
  `date_created` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  UNIQUE KEY `IDX1` USING BTREE (`delivery_date`,`resource_id`,`delivery_hour`,`participant_id`,`type_id`,`date_posted`)
) ENGINE=MyISAM AUTO_INCREMENT=5261441 DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC;


CREATE TABLE `schedules_report` (
  `id` int(11) NOT NULL auto_increment,
  `delivery_date` date default NULL,
  `delivery_hour` int(2) default NULL,
  `participant_id` varchar(15) default NULL,
  `resource_id` varchar(20) default NULL,
  `type_id` varchar(10) default NULL,
  `mw` float default NULL,
  `loss_factor` float default NULL,
  `date_posted` datetime NOT NULL,
  `date_created` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  UNIQUE KEY `IDX1` USING BTREE (`delivery_date`,`delivery_hour`,`participant_id`,`resource_id`,`type_id`,`date_posted`)
) ENGINE=MyISAM AUTO_INCREMENT=43369 DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC;

Million thanks

Answer 1

The problem seems to be the “fuzzy matching”:

LEFT JOIN ... ON ...
and hour(s.date_posted) = hour(p.date_posted)
and date(s.date_posted) = date(p.date_posted)

forces MySQL to calculate hour(s.date_posted) and date(s.date_posted) for all rows in s, every time it touches a row in p.

Give this a try:

and s.date_posted
  BETWEEN DATE_SUB(p.date_posted, INTERVAL TIME_TO_SEC(MAKETIME(0,MINUTE(p.date_posted),SECOND(p.date_posted))) SECOND)
  AND DATE_ADD(DATE_SUB(p.date_posted, INTERVAL TIME_TO_SEC(MAKETIME(0,MINUTE(p.date_posted),SECOND(p.date_posted))) SECOND), INTERVAL 1 HOUR)

Edit:

If you can live with leap seconds being calculated wrongly, this can be written as a more human readable

and s.date_posted
  BETWEEN DATE_SUB(p.date_posted, INTERVAL 60*MINUTE(p.date_posted)+SECOND(p.date_posted) SECOND)
  AND DATE_ADD(DATE_SUB(p.date_posted, INTERVAL 60*MINUTE(p.date_posted)+SECOND(p.date_posted) SECOND), INTERVAL 1 HOUR)

Edit 2:
Repetition of part of the calculated value in the upper limit of the BETWEEN is on purpouse: MySQL will calculate it only once.

Edit 3:
Now seeing your SHOW CREATE TABLE, I understand you don’t have a separate index on date_posted, but just a combined index. You might want to try

ALTER TABLE `schedules_report` ADD INDEX(date_posted)