Question

I have a search form. If the user makes a typo like ager instead of anger, it should still show the relevant results instead of displaying 0 results found.

I came across the PHP levenshtein function and the example that they have given with array is exactly what I want [except that the user can input a sentence rather than one word], but I would like to implement it with database, but have no idea as to how go about implementing it with database.

This is my code:

if(!empty($search))
{
    try {
        $query = $this->_db->prepare($sql);
        $query->execute();
        if(!$query->rowCount()==0)
        {
            $foundRows = $this->_db->query("SELECT FOUND_ROWS()")->fetchColumn();
            while($row = $query->fetch(PDO::FETCH_ASSOC))
            {
                $cQuote =  $this->highlightWords(htmlspecialchars($row['cQuotes']),$search);
                $search_result[] = array('success' => true, 'totalRows' => $foundRows, 'cQuotes' => $cQuote, 'vAuthor' => $this->h($row['vAuthor']), 'vBookName' => $this->h($row['vBookName']), 'vRef' => $this->h($row['vRef']));
            }
            $response = json_encode($search_result);
            echo $response;
            return TRUE;
        }
        else
        {
            $ex =  "No results found for " .$search;
            $this->errorMsg($ex);
        }
        $query->closeCursor();
    }
    catch (Exception $ex){
        $ex =  "Problem: " .$ex;
        $this->errorMsg($ex);
    }
}
else
{
    $ex =  "Please enter something";
    $this->errorMsg($ex);
}

I should add that I’m using MySQL + PDO.

Answer 1

For this to work, you’d need three things:

• A Levenshtein-distance implementation on the MySQL end
• An index of words found in each text
• Split the search query into words, a LEFT JOIN for each word and an HAVING clause

An example database schema:

text

+---------+----------------------------------------------+
| text_id | text                                         |
+---------+----------------------------------------------+
|       1 | The quick brown fox jumps over the lazy dog  |
|       2 | The slow brown foxes jump over the lazy dogs |
+---------+----------------------------------------------+

word

+-------+---------+
| word  | text_id |
+-------+---------+
| fox   |       1 |
| foxes |       2 |
| dog   |       1 |
| dogs  |       2 |
+-------+---------+

Once you have that, say someone searches for “foxs dogg“, you’d build a query like this one:

SELECT text FROM text
    LEFT JOIN word w1 ON w1.text_id = text.text_id AND LEVENSHTEIN(w1.word, "foxs") < 3
    LEFT JOIN word w2 ON w2.text_id = text.text_id AND LEVENSHTEIN(w2.word, "dogg") < 3
    GROUP BY text.text_id
    HAVING COUNT(*) = 2

…where:

• Each word has a LEFT JOIN (e.g.: foxs and dogg)
• You have an HAVING clause that contains the total number of words (e.g.: HAVING COUNT(*) = 2)
• The maximum distance for each word is specified (e.g.: LEVENSHTEIN(...) < 3)

The above would return both entries.

Here’s another example:

SELECT text FROM text
    LEFT JOIN word w1 ON w1.text_id = text.text_id AND LEVENSHTEIN(w1.word, "foxs") < 3
    LEFT JOIN word w2 ON w2.text_id = text.text_id AND LEVENSHTEIN(w2.word, "slows") < 3
    GROUP BY text.text_id
    HAVING COUNT(*) = 2

The above would return only text_id = 2.

Now, before you go crazy implementing this, you should know that multiple JOIN clauses, like the above, on a table having millions of entries (words), will have a very big performance impact.

While this is a working example, you really should look for an already implemented search algorithm, like Solr’s SpellCheck component.