Question

I am trying to improve my SQL query so that i could get only the newest information from f_facts table and i’m checking if the information is new by the date_id

for some reason this query isn’t working for me:

REPLACE INTO `r_views`
        SELECT network_id, type_id, topic_id, COUNT(*), date_id, 
        SUM( IF (`d_refer`.`refer` LIKE '%facebook%', 1,0) ) AS `facebook`, 
        SUM( IF (`d_refer`.`refer` LIKE '%twitter%', 1,0) ) AS `twitter`, 
        SUM( IF ( (`d_refer`.`refer` LIKE '%hotmail%' or `d_refer`.`refer` LIKE '%live%' or `d_refer`.`refer` LIKE '%gmail%' or `d_refer`.`refer` LIKE '%mail.%'), 1,0) ) AS `email`, 
        SUM( IF (`d_refer`.`refer` LIKE '%google%', 1,0) ) AS `google`      

        FROM `f_facts` 
        LEFT JOIN `d_date` ON `f_facts`.`date_id` = `d_date`.`id` 
        LEFT JOIN `d_refer` ON `f_facts`.`refer_id` = `d_refer`.`id` 

        WHERE `action_id`=3 AND `type_id` != 17 AND `date_id` > (SELECT `date_id` FROM `r_views` ORDER BY `date_id` DESC LIMIT 1) GROUP BY topic_id, date_id

This is my older sql query that was reading all of the data from f_facts table which is not efficient:

REPLACE INTO `r_views`
        SELECT network_id, type_id, topic_id, COUNT(*), date_id, 
        SUM( IF (`d_refer`.`refer` LIKE '%facebook%', 1,0) ) AS `facebook`, 
        SUM( IF (`d_refer`.`refer` LIKE '%twitter%', 1,0) ) AS `twitter`, 
        SUM( IF ( (`d_refer`.`refer` LIKE '%hotmail%' or `d_refer`.`refer` LIKE '%live%' or `d_refer`.`refer` LIKE '%gmail%' or `d_refer`.`refer` LIKE '%mail.%'), 1,0) ) AS `email`, 
        SUM( IF (`d_refer`.`refer` LIKE '%google%', 1,0) ) AS `google`      

        FROM `f_facts` 
        LEFT JOIN `d_date` ON `f_facts`.`date_id` = `d_date`.`id` 
        LEFT JOIN `d_refer` ON `f_facts`.`refer_id` = `d_refer`.`id` 

        WHERE `action_id`=3 AND `type_id` != 17 GROUP BY topic_id, date_id 

Answer 1

I think you can enhance the query in three points:

1) the email column you can use this instead of the ORs:

SUM( IF ( (`d_refer`.`refer` LIKE '%live%' or `d_refer`.`refer` LIKE '%mail.%'), 1,0) ) AS `email`

2) Do you really need the two Joins? They seem to me a little bit redundant.

3) Add an index on the date