Question

I have a table in SQlite that I’m using to cache some data. I’m totally hung up on writing an SQL query that when given a list of x strings will return a subset y that are not in the table. I understand I could just query one by one and check for an empty result but I’d like to know if there’s a way to do this that returns a nice, neat list of the strings not cached (not in the table).

FWIW I’m using sqlite3 and Python. The query I’ve got so far is something like

select e from cache where cache.fk = x and e in (a, b, c);

which is close but gives the opposite behavior from what I want (it returns values that would be a cache hit).

I also understand that in this case it is slightly inappropriate to use a relational database over a key/value store but even if I were to switch away I’m still interested in seeing how this would be done.

Answer 1

Assuming there’s a set value of things you’re caching and that they come from a table, use not exists:

select 
    v.e 
from
    vals v 
where 
    v.fk = x
    and v.e in (a,b,c)
    and not exists (
        select
            1
        from
            cache c
        where
            c.fk = v.fk
            and c.e = v.e
    )

If you don’t have a table of available values and are just throwing some set back, you’d go with a dynamic table, like this:

select 
    v.e 
from
    (
    select a as e
    union all
    select b as e
    union all
    select c as e
    ) v 
where 
    not exists (
        select
            1
        from
            cache c
        where
            c.fk = x
            and c.e = v.e
    )