Question

I am writing a query for a very specific report that contains a variable number of columns, based on specific relationships of an item. I am open to suggetions on how to change the query if needs be, but I don’t think it can be. I would prefer to keep this as a single query, as opposed to running it in a loop. The table that is being searched contains around 4 million records, and cannot be archived.

What I would like to know, is why the DATEADD index is not being used on the subquery, although it is being used in the outer query, which is on the same table. I am aware that functions on a field stop MySQL from being able to index, but this is only on the item, not what you are comparing it to.

The result of the report is a number for each specific item (subquery) for each date in the range, where something took place. The date range is generated dynamically. The subquerys should return the results for a single day

We are using MySQL version 5.0.77, which we cannot change as it is managed by our Hosting Provider.

Here is the query:

 SELECT DATE_FORMAT(DATEADD, '%d/%m/%y') AS DATEADD,

   (SELECT COUNT(ID)
      FROM ATABLE AS 
     WHERE ELEMNAME = 'ANELEMENT' AND COMPID = 132
           AND VT.DATEADD BETWEEN CONCAT(DATE(V.DATEADD)," 00:00:00") AND CONCAT(DATE(V.DATEADD)," 23:59:59")))
      AS '132',

   (SELECT COUNT(ID)
      FROM ATABLE AS 
     WHERE ELEMNAME = 'ANELEMENT' AND COMPID = 149
           AND VT.DATEADD BETWEEN CONCAT(DATE(V.DATEADD)," 00:00:00") AND CONCAT(DATE(V.DATEADD)," 23:59:59")))
      AS '149'

 FROM ATABLE AS V
WHERE 1 = 1 AND COMPID = 132
   AND (V.DATEADD >= "2010-09-01 00:00:00"
        AND V.DATEADD <= "2010-10-26 23:59:59")
   AND 1 = 1
   AND ELEMNAME = 'ANELEMENT'
GROUP BY DATE_FORMAT(DATEADD, '%Y-%m-%d')

The number of times the subquery is ran depends on the number of links this item has, and is determined when the query is built.

We have tried:
replacing the between with

"VT.DATEADD <= DATE(V.DATEADD) and VT.DATEADD <= DATE(V.DATEADD) +1" 

however this doesnt work either, changing it to

"VT.DATEADD = DATE(V.DATEADD)" 

does use the index, however doesnt return the correct number of rows, as DATEADD is a datetime. If we change it to:

"VT.DATEADD >= "2010-09-01" AND VT.DATEADD <= "2010-09-02"

The output from Explain is

+----+--------------------+-------+-------+-------------------------+----------+---------+-------+-------+----------------------------------------------+
| id | select_type        | table | type  | possible_keys           | key      | key_len | ref   | rows  | Extra                                        |
+----+--------------------+-------+-------+-------------------------+----------+---------+-------+-------+----------------------------------------------+
|  1 | PRIMARY            | V     | range | DATEADD,COMPID,ELEMNAME | DATEADD  | 8       | NULL  |  1386 | Using where; Using temporary; Using filesort |
|  2 | DEPENDENT SUBQUERY | VT    | ref   | COMPID,ELEMNAME         | ELEMNAME | 103     | const | 44277 | Using where                                  |
+----+--------------------+-------+-------+-------------------------+----------+---------+-------+-------+----------------------------------------------+

Using USE INDEX, or FORCE INDEX (when it is available but not used) uses NULL key

without fixing this, the query runs incredibly slowly, even over a tiny date range and locks the database up.

Answer 1

I don’t know if I’m over simplifying what you want overall, but will this one work for you. It appears you want to know how much activity for two “compid” values within a given date range.

SELECT 
      DATE_FORMAT(DATEADD, '%Y-%m-%d'),
      SUM( if( compid = 132, 1, 0 ) ) as Count132,
      SUM( if( compid = 149, 1, 0 ) ) as Count149
   from
      ATable 
   where
          elemname = "ANELEMENT"
      AND ( compid = 132 or compid = 149 )
      AND DATEADD BETWEEN "2010-09-01 00:00:00" AND "2010-10-26 23:59:59"
   group by
      dateadd