I have an SQLite database with a virtual table “myTable” created using FTS4, with one column “myColumn” of text using | as a separating character.
I query this database with SELECT * FROM myTable WHERE myColumn MATCH 'out to'; and I’m getting hits with entries that look like "...out|to..." but with no “out to” substrings.
The same thing happens when I replace | with punctuation like ;. (Note the SQLite docs make explicit that you can’t use _ as a separating character.)
Why is that and how do I prevent this?
FTS’s default tokenizer drops any punctuation from the indexed data, and also from your input query. It would find a match for a entry “… out to …” even if you query
MATCH 'out/to'The solution to this is using phrase searches,
MATCH '"out to"'. It won’t work if you wish to find those words in a record that does not have them in that order.