Question

I’m trying to read in a large (700GB) file and incrementally process it, but the network I’m working on will occasionally go down, cutting off access to the file. This throws a java.io.IOException telling me that “The specified network name is no longer available”. Is there a way that I can catch this exception and wait for, say, fifteen minues, and then retry the read, or is the Reader object fried once access to the file is lost?

If the Reader is rendered useless once the connection is lost, is there a way that I can rewrite this in such a way as to allow me to “save my place” and then begin my read from there without having to read and discard all the data before it? Even just munching data without processing it takes a long time when there’s 500GB of it to get through.

Currently, the code looks something like this (edited for brevity):

class Processor {
    BufferedReader br;

    Processor(String fname) {
        br = new BufferedReader(new FileReader("fname"));
    }

    void process() {
        try {
            String line;
            while((line=br.readLine)!=null) {
                ...code for processing the line goes here...
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Thank you for your time.

Answer 1

You can keep track of read bytes in a variable. For example here I keep track in a variable called read, and buff is char[]. Not sure if this is possible using the readLine method.

read+=br.read(buff);

Then if you need to restart, you can skip that many bytes

br.skip(read);

Then you can keep processing away. Good luck