Question

Suppose you need to deal with 2 threads, a Reader and a Processor.

Reader will read a portion of the stream data and will pass it to the Processor, that will do something.

The idea is to not stress the Reader with too much of data.

In the set up, i

// Processor will pick up data from pipeIn and will place the output in pipeOut
Thread p = new Thread(new Processor(pipeIn, pipeOut));
p.start();

// Reader will pick a bunch of bits from the InputStream and place it to pipeIn
Thread r = new Thread(new Reader(inputStream, pipeIn));
r.start();

Needless to say, neither pipe is null, when initialized.

I am thinking … When Processor has been started it attempts to read from the pipeIn, in the following loop:

    while (readingShouldContinue) {
        Thread.sleep(1); // To avoid tight loop

        byte[] justRead = readFrom.getDataCurrentlyInQueue();
        writeDataToPipe(processData(justRead));
    }

If there is no data to write, it will write nothing, should be no problem.

The Reader comes alive and picks up some data from a stream:

    while ((in.read(buffer)) != -1) {
        // Writes to what processor considers a pipeIn
        writeTo.addDataToQueue(buffer); 
    }

In Pipe itself, i synchronize access to data.

public byte[] getDataCurrentlyInQueue() {

    synchronized (q) {
        byte[] a = q.peek();
        q.clear();
        return a;
    }
}

I expect the 2 threads to run semi in parallel, interchanging activities between Reader and a Processor. What happens however is that

• Reader reads all blocks up front
• Processor treats everything as 1 single block

What am i missing please?

Answer 1

You reinvented parts of the java concurrent library. it would make things a lot easier if you modeled your threads with BlockingQueue instead of synchronizind things yourself.

Basically your producer would put chunks on the BlockingQueue und your consumer would while(true) loop over the queue and call get(). That way the producer would block/wait until there is a new chunk on the queue.