Question

I need to write a parser for textfiles (at least 20 kb), and I need to determine if words out of a set of words appear in this textfile (about 400 words and numbers). So I am looking for the most efficient possibilitie to do this (if a match is found, i need to do some further processing of this and it’s previous line).

What I currently do, is to exclude lines that do not contain any information for sure (kind of metadata lines) and then compare word by word – but i don’t think that only comparing word by word is the most efficient possibility.

Can anyone please provide some tips/hints/ideas/…

Thank you very much

Answer 1

How about the following:

Put all your keywords in a HashSet (Set<String> keywords;)
Read the file one line at once
  For each line in file:
  Tokenize to words
  For each word in line:
  If word is contained in keywords (keywords.containes(word))
    Process actual line
    If previous line is available
        Process previous line
  Keep track of previous line (prevLine = line;)