Question

The task I have is to (somewhat efficiently) read line-by-line through a very large, continuously growing file. Here’s basically what I’m doing now:

BufferedReader rd = //initialize BufferedReader
String line;
while(true){
    while((line=rd.readLine())==null){
        try{
            Thread.sleep(1000);
        }catch(InterruptedException e){
            //handle exception
        }
    }
    //process line
}

So my BufferedReader just hangs at the end of the file until there is more stuff to be read. This works pretty well, but there’s one problem – If readLine is called while the process that writes to the file is in the middle of writing a line. Then the first call to readLine will return the first section of the line, and the next call will return the second section. But I really need those two parts together, I need complete lines.

More specifically, my problem happens when the following interleaving of events happens:

1. Writing process writes most of a line
2. readLine() is called
3. Writing process finishes that line and adds a newline character
4. readLine() is called

The result is that each readLine() picks up a section of the entire line that the writing process is outputting. It is behaving as expected in doing so because each time it is called, it reaches the end of the file so returns what it has read.

So essentially the functionality I need is a BufferedReader that returns null one readLine earlier; one that does not give you a line until there is a line-break after it, not just EOF after it. So if it finds EOF, it does not return the line up to that point, it returns null, and returns that line once the file has been written to and there’s a new line after it.

I could probably implement a crude way of doing this by dealing with the FileReader more directly and essentially re-writing BufferedReader, but I don’t know much about how to do this efficiently. My implementation probably won’t be as fast as the real BufferedReader, and I’d like to avoid slowing down the program for the times when there is data to be read.

Answer 1

You could start with the source of BufferedReader and rewrite the
String readLine(boolean ignoreLF) method which causes the trouble if it finds EOF before the end of line. (Unfortunatly it can’t be inhertited due to package scope)