I’m working on some stuff for an in-house CRM. The company’s current frontend allows for lots of duplicates. I’m trying to stop end-users from putting in the same person because they searched for ‘Bill Johnson’ and not ‘William Johnson.’ So the user will put in some information about their new customer and we’ll find the similar names (including fuzzy names) and match them against what is already in our database and ask if they meant those things… Does such a database or technology exist?
I’m working on some stuff for an in-house CRM. The company’s current frontend allows
Share
I implemented such a functionality on one website. I use double_metaphone() + levenstein() in PHP. I precalculate a double_metaphone() for each entry in the dabatase, which I lookup using a SELECT of the first x chars of the ‘metaphoned’ searched term.
Then I sort the returned result according to their levenstein distance. double_metaphone() is not part of any PHP library (last time I checked), so I borrowed a PHP implementation I found somewhere a long while ago on the net (site no longer on line). I should post it somewhere I suppose.
EDIT: The website is still in archive.org:
http://web.archive.org/web/20080728063208/http://swoodbridge.com/DoubleMetaPhone/
or Google cache:
http://webcache.googleusercontent.com/search?q=cache:Tr9taWl9hMIJ:swoodbridge.com/DoubleMetaPhone/+Stephen+Woodbridge+double_metaphon
which leads to many other useful links with source code for double_metaphone(), including one in Javascript on github: http://github.com/maritz/js-double-metaphone
EDIT: Went through my old code, and here are roughly the steps of what I do, pseudo coded to keep it clear:
1) Precompute a double_metaphone() for every word in the database, i.e., $word=’blahblah’; $soundslike=double_metaphone($word);
2) At lookup time, $word is fuzzy-searched against the database: $soundslike = double_metaphone($word)
4) SELECT * FROM
tableWHEREsoundlikeLIKE $soundlike (if you have levenstein stored as a procedure, much better: SELECT * FROM table WHERE levenstein(soundlike,$soundlike) < mythreshold ORDER BY levenstein(word,$word) ASC LIMIT … etc.It has worked well for me, although I can’t use a stored procedure, since I have no control over the server and it’s using MySQL 4.20 or something.