Question

I have this quite long query that should give me some information about shipments, and it works, but it’s performing terribly bad. It takes about 4500ms to load.

SELECT
    DATE(paid_at) AS day,
    COUNT(*) as order_count,
    (
      SELECT COUNT(*) FROM line_items
      WHERE order_id IN (SELECT id from orders WHERE DATE(paid_at) = day)
    ) as product_count,
    (
      SELECT COUNT(*) FROM orders
      WHERE shipping_method = 'colissimo'
      AND DATE(paid_at) = day
      AND state IN ('paid','shipped','completed')
    ) as orders_co,
    (
      SELECT COUNT(*) FROM orders
      WHERE shipping_method = 'colissimo'
      AND DATE(paid_at) = day
      AND state IN ('paid','shipped','completed')
      AND paid_amount < 70
    ) as co_less_70,
    (
      SELECT COUNT(*) FROM orders
      WHERE shipping_method = 'colissimo'
      AND DATE(paid_at) = day
      AND state IN ('paid','shipped','completed')
      AND paid_amount >= 70
    ) as co_plus_70,
    (
      SELECT COUNT(*) FROM orders
      WHERE shipping_method = 'mondial_relais'
      AND DATE(paid_at) = day
      AND state IN ('paid','shipped','completed')
    ) as orders_mr,
    (
      SELECT COUNT(*) FROM orders
      WHERE shipping_method = 'mondial_relais'
      AND DATE(paid_at) = day
      AND state IN ('paid','shipped','completed')
      AND paid_amount < 70
    ) as mr_less_70,
    (
      SELECT COUNT(*) FROM orders
      WHERE shipping_method = 'mondial_relais'
      AND DATE(paid_at) = day
      AND state IN ('paid','shipped','completed')
      AND paid_amount >= 70
    ) as mr_plus_70
    FROM orders
    WHERE MONTH(paid_at) = 11
    AND YEAR(paid_at) = 2011
    AND state IN ('paid','shipped','completed')
    GROUP BY day;

Any idea what I could be doing wrong or what I could be doing better? I have other queries of similar length that don’t take as much time to load as this. I thought this would be faster than for example having an individual query for each day (in my programming instead of the SQL query).

Answer 1

It is because you are using sub-queries where you don’t need them.

As a general rule, where you have a sub-query within a main SELECT clause, that sub-query will query the tables within it once for each row in the main SELECT clause – so if you have 7 subqueries and are selecting a date range of 30 days, you will effectively be running 210 separate subqueries (plus your main query).

(Some query optimisers can resolve sub-queries into the main query under some circumstances, but as a general rule you can’t rely on this.)

In this case, you don’t need any of the orders sub-queries, because all the orders data you require is included in the main query – so you can rewrite this as:

SELECT
    DATE(paid_at) AS day,
    COUNT(*) as order_count,
    (
      SELECT COUNT(*) FROM line_items
      WHERE order_id IN (SELECT id from orders WHERE DATE(paid_at) = day)
    ) as product_count,
    sum(case when shipping_method = 'colissimo' then 1 end) as orders_co,
    sum(case when shipping_method = 'colissimo' AND 
                  paid_amount < 70 then 1 end) as co_less_70,
    sum(case when shipping_method = 'colissimo' AND 
                  paid_amount >= 70 then 1 end) as co_plus_70,
    sum(case when shipping_method = 'mondial_relais' then 1 end) as orders_mr,
    sum(case when shipping_method = 'mondial_relais' AND 
                  paid_amount < 70 then 1 end) as mr_less_70,
    sum(case when shipping_method = 'mondial_relais' AND 
                  paid_amount >= 70 then 1 end) as mr_plus_70
    FROM orders
    WHERE MONTH(paid_at) = 11
    AND YEAR(paid_at) = 2011
    AND state IN ('paid','shipped','completed')
    GROUP BY day;