Question

I’ve been trying to pair NIO with multithreaded read handling in order to make a scalable server. I cannot use any frameworks such as Netty or MINA because of several low level client-server protocol details that would be too much effort to implement in either. I was just examining my code and I realized there is a potential race condition in this snippet:

//executes in selector thread
public void runSelector() {
    //...
    Set<SelectionKey> keys = selector.selectedKeys();
    for (Iterator<SelectionKey> keyIter = keys.iterator(); keyIter.hasNext(); ) {
        final SelectionKey key = keyIter.next();
        keyIter.remove();
        if (key.isValid() && key.isReadable()) { //point A
            //maybe some other short calculations
            ((SocketChannel) key.channel()).read(buffer); //point B
            workerThreadPool.submit(new Runnable() { public void run() { processRead(key, buffer); } });
        }
    }
    //...
}

//executes in a worker thread
private void processRead(SelectionKey key, ByteBuffer buf) {
    //... somewhere
    key.cancel();
    //...
}

It is a highly improbable event, but it is entirely possible that in a worker thread, I call key.cancel() while the selector thread is between the two points I commented in the runSelector() method. Keep in mind that this could be deployed on a high concurrency machine, and the CPU core that runs the selector thread could be bogged down. Is it a valid concern that key.cancel() could be called in a worker thread in between key.isReadable() and channel.read() in the selector thread and cause a CancelledKeyException? Should I have some kind of thread safe collection that stores all the keys that are to be canceled so that runSelector() can cancel all of them at the end of an iteration? How do the more professional projects like Netty or MINA handle such a case?

Answer 1

Lock the resource,

synchronized(key) {
    ((SocketChannel) key.channel()).read(buffer); //point B
}

If this looks odd look at this Oracle tutorial on concurrency.