Question

I’ve got a Java client that needs to recursively call a server to retrieve a large data graph – requiring approximately one thousand calls. I have no control of the server, and this is required for a time-critical crash recovery scenario.

My problem is that I need my original thread to block until all calls have completed.

The RecursiveAction and ForkJoinPool abstractions from java.util.concurrent are precisely what I need, except that they are designed for CPU parallelism and forbid use of blocking I/O.

So, what would be the best way to implement recursive network calls, with the initiating thread blocking until all calls have completed?

Additional context info:

• I can’t modify the server.
• The server allows and supports this kind of heavy querying.
• I will restrict the number of concurrent network calls to something like 10-30.
• Caching the data on disk is not feasible.

Additional thoughts: would a single-phase Phaser be appropriate, in conjunction with a ThreadPoolExecutor? Call tasks would call Phaser.register(), make the call, submit child tasks and then call Phaser.arrive(). The initiating thread would call Phaser.awaitAdvance(1). Would this be the most appropriate approach?

Answer 1

Got this working well using JDK1.7 Phasers. The pattern I used was something like this:

private void loadGraphFromServer() {
    final Phaser phaser = new Phaser(1); // "1" registers the calling thread
    for (final Item item : getDataListFromServer()) {
        phaser.register();
        executorService.submit(new Runnable() {
            public void run() {
                try {
                    getMoreDataFromServer(item.getSomeId());
                    // more nested loops/tasks/calls here...
                }
                finally {
                    phaser.arrive();
                }
            }
        });
    }
    phaser.arriveAndAwaitAdvance(); // blocks until all tasks are complete
}