Question

function levenshtein(a, b) {
  var i,j,cost,d=[];

  if (a.length == 0) {return b.length;}
  if (b.length == 0) {return a.length;}

  for ( i = 0; i <= a.length; i++) {
    d[i] = new Array();
    d[ i ][0] = i;
  }

  for ( j = 0; j <= b.length; j++) {
    d[ 0 ][j] = j;
  }

  for ( i = 1; i <= a.length; i++) {
    for ( j = 1; j <= b.length; j++) {
      if (a.charAt(i - 1) == b.charAt(j - 1)) {
        cost = 0;
      } else {
        cost = 1;
      }

      d[ i ][j] = Math.min(d[ i - 1 ][j] + 1, d[ i ][j - 1] + 1, d[ i - 1 ][j - 1] + cost);

      if (i > 1 && j > 1 && a.charAt(i - 1) == b.charAt(j - 2) && a.charAt(i - 2) == b.charAt(j - 1)) {
        d[i][j] = Math.min(d[i][j], d[i - 2][j - 2] + cost)
      }
    }
  }

  return d[ a.length ][b.length];
}

function suggests(suggWord) {
  var sArray = [];
  for(var z = words.length;--z;) {
    if(levenshtein(words[z],suggWord) < 2) { 
      sArray.push(words[z]);
    }   
  }
}

Hello.

I’m using the above implementation of Damerau-Levenshtein algorithm. Its fast enough on a normal PC browser, but on a tablet it takes ~2/3 seconds.

Basically, I’m comparing the word sent to a suggest function to every word in my dictionary, and if the distance is less than 2 adding it to my array.

The dic is an array of words approx size 600,000 (699KB)
The aim of this is to make a suggest word feature for my Javascript spell checker.

Any suggestion on how to speed this up? Or a different way of doing this?

Answer 1

One thing you can do if you are only looking for distances less than some threshold is to compare the lengths first. For example, if you only want distances less than 2, then the absolute value of the difference of the two strings’ lengths must be less than 2 as well. Doing this will often allow you to avoid even doing the more expensive Levenshtein calculation.

The reasoning behind this is that two strings that differ in length by 2, will require at least two insertions (and thus a resulting minimum distance of 2).

You could modify your code as follows:

function suggests(suggWord) {
  var sArray = [];
  for(var z = words.length;--z;) {
    if(Math.abs(suggWord.length - words[z].length) < 2) {
      if (levenshtein(words[z],suggWord) < 2) { 
        sArray.push(words[z]);
      }
    }   
  }
}

I don’t do very much javascript, but I think this is how you could do it.

Part of the problem is that you have a large array of dictionary words, and are doing at least some processing for every one of those words. One idea would be to have a separate array for each different word length, and organize your dictionary words into them instead of one big array (or, if you must have the one big array, for alpha lookups or whatever, then use arrays of indexes into that big array). Then, if you have a suggWord that’s 5 characters long, you only have to look through the arrays of 4, 5, and 6 letter words. You can then remove the Match.Abs(length-length) test in my code above, because you know you are only looking at the words of the length that could match. This saves you having to do anything with a large chunk of your dictionary words.

Levenshtein is relatively expensive, and more so with longer words. If it is simply the case that Levenshtein is too expensive to do very many times, especially with longer words, you may leverage off another side effect of your threshold of only considering words that either exactly match or that have a distance of 1 (one insertion, deletion, substitution, or transposition). Given that requirement, you can further filter candidates for the Levenshtein calculation by checking that either their first character matches, or their last character matches (unless either word has a length of 1 or 2, in which case Levensthein should be cheap to do). In fact, you could check for a match of either the first n characters or the last n characters, where n = (suggWord.length-1)/2. If they don’t pass that test, you can assume that they won’t match via Levenshtein. For this you would want primary array of dictionary words ordered alphabetically, and in addition, an array of indexes into that array, but ordered alphabetically by their reversed characters. Then you could do a binary search into both of those arrays, and only have to do Levenshtein calculation on the small subset of words whose n characters of their start or end match the suggWord start or end, and that have a length that differs by at most one character.