Question

I’m trying to write a SQL statement that will allow me to select a series of articles from a table based on their keywords. What I’ve got so far is a token table, an article table, and a many-to-many table for tokens & articles:

tokens
  rowid
  token

token_article
  token_rowid
  article_rowid

articles
  rowid

What I’m doing is taking a search query, splitting it up by spaces, then select all articles that contains those keywords. So far I’ve come up with this:

select * from 
    (select * from tokens 
        inner join token_article on 
             tokens.rowid = token_article.token_rowid and 
             token = 'ABC'
    ) as t1, 

    (select * from tokens 
        inner join token_article on 
            tokens.rowid = token_article.token_rowid and 
            token = 'DEF'
    ) as t2 

where t1.article_rowid = t2.article_rowid and t2.article_rowid = articles.rowid

Which works but of course its doing a select on all articles that match ABC and all articles that DEF then selecting them.

Now I’m trying to figure out a better way. What I imagine in my mind that would work would be to select all the articles that match ABC and from those match any with DEF. This is what I imagine it to look like but does not work (receive error message “no such columns: tokens.rowid”)

select * from 
    (select * from
        (select * from tokens 
                inner join token_article on 
                    tokens.rowid = token_article.token_rowid and
            token = 'ABC'
        ) 
    inner join token_article on 
            tokens.rowid = token_article.token_rowid and
    token = 'DEF'
)

Answer 1

Because there is more than one way to do this…this method uses GROUP BY and HAVING clauses. The query is looking for all articles that have either the ABC or DEF token, but then grouping by the article ID where the count of tokens for the article is equal to the number of tokens being queried.

Note that I’ve used MSSQL syntax here, but the concept should work in most SQL implementations.

Edit: I should point out that this has a fairly clean syntax as you add more tokens to the query. If you add more tokens, then you just need to modify the t.token_in criteria and adjust the HAVING COUNT(*) = x clause accordingly.

DECLARE @tokens TABLE
(
    rowid INT NOT NULL,
    token VARCHAR(255) NOT NULL
)

DECLARE @articles TABLE
(
    rowid INT NOT NULL,
    title VARCHAR(255) NOT NULL
)

DECLARE @token_article TABLE
(
    token_rowid INT NOT NULL,
    article_rowid INT NOT NULL
)

INSERT INTO @tokens VALUES (1, 'ABC'), (2, 'DEF')
INSERT INTO @articles VALUES (1, 'This is article 1.'), (2, 'This is article 2.'), (3, 'This is article 3.'), (4, 'This is article 4.'), (5, 'This is article 5.'), (6, 'This is article 6.')
INSERT INTO @token_article VALUES (1, 1), (2, 1), (1, 2), (2, 3), (1, 4), (2, 4), (1, 5), (1, 6)

-- Get the article IDs that have all of the tokens
-- Use this if you just want the IDs
SELECT a.rowid FROM @articles a
INNER JOIN @token_article ta ON a.rowid = ta.article_rowid
INNER JOIN @tokens t ON ta.token_rowid = t.rowid
WHERE t.token IN ('ABC', 'DEF')
GROUP BY a.rowid
HAVING COUNT(*) = 2 -- This should match the number of tokens

rowid
-----------
1
4

-- Get the articles themselves
-- Use this if you want the articles
SELECT * FROM @articles WHERE rowid IN (
    SELECT a.rowid FROM @articles a
    INNER JOIN @token_article ta ON a.rowid = ta.article_rowid
    INNER JOIN @tokens t ON ta.token_rowid = t.rowid
    WHERE t.token IN ('ABC', 'DEF')
    GROUP BY a.rowid
    HAVING COUNT(*) = 2 -- This should match the number of tokens
)

rowid       title
----------- ------------------
1           This is article 1.
4           This is article 4.