Question

Is there a way to have one thread in java make a read call to some FileInputStream or similar and have a second thread processing the bytes being loaded at the same time?
I’ve tried a number of things – my current attempt has one thread running this:

FileChannel inStream;
try {
   inStream = (new FileInputStream(inFile)).getChannel();
} catch (FileNotFoundException e) {
    e.printStackTrace();
}
int result;
try {
     result = inStream.read(inBuffer);
} ...

And a second thread wanting to access the bytes as they are being loaded. Clearly the read call in the first thread blocks until the buffer is full, but I want to be able to access the bytes loaded into the buffer before that point. Currently, everything I try has the buffer and it’s backing array unchanged until the read completes – this not only defeats the point of this threading but also suggests the data is being loaded into some intermediate buffer somewhere and then copied into my buffer later, which seems daft.

One option would be to do a bunch of smaller reads into the array with offsets on subsequent reads, but that adds extra overhead.

Any ideas?

Answer 1

When you read data sequentially, the OS will read ahead the data before you need it. As the system is doing this for you already, you may not get the benefit you might expect.

why can’t I just make my Filechannel or FileInputStream “flow” into my ByteBuffer or some byte array?

That is sort of what it does already.

If you want a more seamless loading of the data, you can use a memory mapped files as it “appears” in the memory of the program immediately and is loaded in the background as you use it.