Question

I need to test the below mentioned method by calling it locally by a main method

public TokenFilter create(TokenStream input) {
  if (protectedWords != null){
      input = new KeywordMarkerFilter(input,protectedWords);
  }

return new KStemFilter(input);

}

The problem I’m facing is I need to pass a string as input, but I’m not sure how to parse it as a token stream.

Please help.

Answer 1

To get TokenString from a search text, you have to use Analyzer for that:

Analyzer analyzer = ...; // your analyzer
TokenStream tokenStream = analyzer.tokenStream(null, new StringReader(searchText));

Note that it should be the same analyzer that is used to build the index.