Question

I want to remove stop words in java.

So, I read stop words from text file.

and store Set

Set<String> stopWords = new LinkedHashSet<String>();
BufferedReader br = new BufferedReader(new FileReader("stopwords.txt"));
        String words = null;
        while( (words = br.readLine()) != null) {
            stopWords.add(words.trim());
            }
        br.close();

And, I read another text file.

So, I wanna remove to duplicate string in text file.

How can I?

Answer 1

You want to remove duplicate words from file, below is the high level logic for same.

• Read File
• Loop through file content(i.e one line at a time)
  • Have string tokenizer for that line based on space
  • Add each each token to your set. This will make sure that you have only one entry per word.
  • Close file

Now you have set that contains all the unique word of file.