Question

Yes, this is yet another question on producer/consumer in Java.

My variant is that I have N producers started by the produceDataRows method and M consumers started by the consumeDataRows method.
Both methods start their own instance of the ThreadPoolExecutor class, submit the respective number of producer/consumer tasks and then wait until their executors complete.

So, here is my code:

final BlockingQueue<Row> allRows = new LinkedBlockingQueue<Row>();
ExecutorService exec = Executors.newFixedThreadPool(2);
FutureTask<Object> producer = new FutureTask<Object>(new Callable<Object>() {
  @Override
  public Object call() throws Exception {
    produceDataRows(allRows);
    return null;
  }
});
FutureTask<Object> consumer = new FutureTask<Object>(new Callable<Object>() {
  @Override
  public Object call() throws Exception {
    consumeDataRows(allRows);
    return null;
  }
});
exec.execute(producer);
exec.execute(consumer);
producer.get();
consumer.get();

The problem is that consumer.get() returns, but consumeDataRows is never called. The produceDataRows, on the other hand is called.

What am I missing?

Thanks.

EDIT 1

Following the reply by Gray, I have rewritten the code as follows:

ExecutorService exec = Executors.newFixedThreadPool(2);
Callable<Object> producer = new Callable<Object>() {
  @Override
  public Object call() throws Exception {
    produceDataRows(allRows);
    return null;
  }
};
Callable<Object> consumer = new Callable<Object>() {
  @Override
  public Object call() throws Exception {
    consumeDataRows(allRows);
    return null;
  }
};
exec.submit(producer);
exec.submit(consumer);
exec.shutdown();
exec.awaitTermination(10, TimeUnit.DAYS);

The same effect – the code terminates, but consumeDataRows is never called. I have three different ThreadPoolExecutor instances here – one in produceDataRows, one in consumeDataRows and the last one here.

Thanks.

EDIT 2

There is something wrong with my produceDataRows method, because if I comment it out, then execution visits the call method of both callables. Trying to figure it out now.

Answer 1

Edit:

I think you have a race condition here. Since the producer and the consumer are both running at the same time, I bet the consumer goes to see what rows are in allRows and gets none because the producer has not done anything yet.

You will need to run the producer ahead of time or have the consumer use allRows.poll(long timeout, TimeUnit unit) to wait for the results of the producer. Maybe the producer could set a volatile boolean producedAll = true once it had produced all of the the rows and completed, and have the consumer looping with .poll(...) waiting for the producedAll to be true.

---

You have your classes a bit confused. You should be calling exec.submit(...) on your Callable not instantiating and executing a FutureTask. submit(...) then returns a Future which you can call get() on.

What you are doing now is instantiating your own FutureTask which is done internally by the ExecutorService — never by the caller. When you are then calling get() on your FutureTask, it is just returning immediately and not doing any joining or anything. It works (unfortunately) because FutureTask is a Runnable.

Your code should be doing something like:

Future<Object> producer = exec.submit(new Callable<Object>() {
    @Override
    public Object call() throws Exception {
       produceDataRows(allRows);
       return null;
    }
});
Future<Object> consumer = exec.submit(new Callable<Object>() {
    ...
});
producer.get();
consumer.get();

Btw, typically I use Callable<Void> if I don’t care about the return from a Callable since I just want the Exception. I still return null but Void is useful when you want a no-value object.