Question

I’m trying to optimize an SQL query. The problem is that it is very slow! I’m working with sqlite and with medium sized datasets(20000row). My sql syntax look like this:

SELECT DISTINCT date(A.last_update) as lastUpdate,
(SELECT COUNT(*) FROM bgp_update AS B WHERE date(B.last_update) = date(A.last_update) AND B.validity = 1) AS valid,
(SELECT COUNT(*) FROM bgp_update AS B WHERE date(B.last_update) = date(A.last_update) AND B.validity = 0) AS invalid,
(SELECT COUNT(*) FROM bgp_update AS B WHERE date(B.last_update) = date(A.last_update) AND B.validity = -1) AS notFound
FROM (SELECT DISTINCT last_update FROM bgp_update ORDER BY last_update) AS A WHERE last_update BETWEEN '%var' and '%var2';

Below is an example of what do i have:

id   |     last_update   |  Validity

48   | 2009-1-6 18:34:38 |  notFound   
47   | 2009-1-6 18:34:38 |  valid   
46   | 2009-1-6 18:34:38 |  valid    
45   | 2009-1-3 18:34:38 |  invalid   
44   | 2009-1-3 18:34:38 |  invalid   
42   | 2009-1-4 18:34:38 |  notFound 
41   | 2009-1-4 18:34:38 |  notFound
48   | 2009-1-4 18:34:38 |  valid

And the query result would look like:

    Date   |  valid   |  invalid   |    notFound

  2009-1-3 |   0      |     2      |       0
  2009-1-4 |   1      |     0      |       2
  2009-1-6 |   2      |     0      |       1

I need this to generate a line graph out of it. The example would be: line chart!

But as I mentioned the query is very slow! How can i make it faster?

:::: 2. EDIT :::::

The previous descriptions I have simplified the problem. My table create statement look like this:

CREATE TABLE bgp_update
    (id INTEGER PRIMARY KEY,
    ip VARCHAR(64) NOT NULL,
    mask INTEGER NOT NULL,
    asn INTEGER NOT NULL,
    validity INTEGER NOT NULL,
    last_update TIMESTAMP DEFAULT CURRENT_TIMESTAMP)

As you can see my query only use the last_update and the validity fields.

Answer 1

You could use sum over case statements, e.g.:

select last_update,
  sum(case when validity=1 then 1 else 0 end) valid,
  sum(case when validity=0 then 1 else 0 end) invalid,
  sum(case when validity=-1 then 1 else 0 end) notfound
from bgp_update
where last_update between '1/1/2013' and '1/2/2013'
group by last_update
order by last_update

SQL Fiddle