Home/ Questions/Q 9078025
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-16T19:31:04+00:00

I have 200,000 strings. I need to find the similar strings among that set.

  • 0

I have 200,000 strings. I need to find the similar strings among that set. I expect the number of similar strings to be very low in the set. Please help out with an efficient data structure.

I can use a simple hash if I am looking for exact matching strings. But, ‘similarity’ is custom defined in my case: two strings are treated similar if 80% of the chars in them are same, order does not matter.

I don’t want to call the function finding “similarity” ~(200k*100k) times. Any suggestions like techniques to preprocess the strings, efficient data structures are welcome. Thanks.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I learnt that >=0.85 distance ratio is possible only if the string-length difference between two strings is <=3. That means, we can group the strings with length difference <=3.

    This drastically reduced the number of string in each group. So, the number of overall comparisons are reduce to slight less than 50% (of 200k*100k) in my data set.
    Moreover, dividing the the data set into multiple small sets helps to do parallel-processing which further reduces the overall runtime.

    Reduction percentage might vary with the sample data set, i.e. worst case happens when all the string are with length difference <=3.

    [Thanks to Inbar Rose for stimulating this thought]

    In my case, the histogram looked as below:

    histogram

    • 0