Question

I have two tables that make up a full text index of article content for search purposes. One of the tables is just a primary key associated with a word, whereas the other records the article it occurred in and its location in the document. A single word can conceivably appear many times in the same document with different locations, so the same word id can occur several times in the word_locations table.

Here are the structures:

words:

id          bigint
word        tinytext

word_location:

id          bigint(20)  
wordid      bigint(20)  
location    int(11) 
article_id  int(11)

What i need to write is a query that will find the count of occurrences for each word for any one profile. I need to preserve a zero value for wordids that don’t appear at all, so I assume this needs to be a left join. However, whenever I try to add a where query to limit off article, any wordids that don’t appear at all are not included in the result set.

I have tried:

select words.wordid, COUNT(word_location.wordid) as appears from words left join word_location on word.id = word_location.wordid where article_id = %s GROUP BY wordid

But this query does not return zeros for words that don’t appear at all.

How can I modify this left join?

Thanks in advance!

EDIT:

Here is an example data set and the result sets for the different queries.

Example article content:

Bob’s Restaurant is one of the finest restaurants in greater
County where you can enjoy the finest Turkish Cuisine.

So the vocabulary table, after being adjusted by the application to exclude stop words, will have in its vocabulary rows for Bob, Restaurant, finest, greater, county, enjoy, Turkish, and cusine. (I’m using this actual article since it’s the first in the set, so the ids actually appear starting from integer 1.

The query provided by @Mark Bannister produces this result set:
wordid – word – occurances:

128 clifton 0
1   bob's   2
2   restaurant  2
3   one 1
4   finest  3
5   restaurants 2
6   greater 1
9   county  1
12  enjoy   3
13  turkish 6
14  cuisine 1

The result set is correct per se – but id 128 doesn’t appear in the document at all and is the only thing in the result set with occurance 0. The goal is to have the entire vocabulary returned with number of occurrences from the document (this is roughly 2500 different words)

My original problematic query from before the edit above actually returned the same result set, but without ANY 0 occurance rows at all.

Answer 1

You need to include your article selection in your join condition:

select words.wordid, COUNT(word_location.wordid) as appears 
from words 
left join word_location on word.id = word_location.wordid and article_id = ? 
GROUP BY wordid

Including the restriction on article_id in the WHERE clause effectively turns your left join back into an inner join.