Question

I have two threads, one reading streams and producing data-objects from their content.

I then have to pass them to the second thread, writing them into a database.

I want to separate the two tasks (that’s why I took two threads) because I/O-Performance on the streams and the database varies. In some cases, the stream is fast, sometimes, the db has some lag.

So I thought of putting some (very simple!) data-storage between them.

My first idea was a FILO-solution, has to be Thread-safe and should be fast (no fancy stuff, just put and pull). The order does not matter. At some time there could be around 100,000 Entries in the storage (that would be the case after around one second ping with the db).

Each object has a small footprint, <1kb.

What would you recommend to me?

Answer 1

This is exactly the producer-consumer pattern. Take a look at BlockingQueue and its implementations.

Here is sample code from my blog:

public class Producer implements Runnable {
    private BlockingQueue queue;

    public Producer(BlockingQueue queue) {
        this.queue = queue;
    }

    @Override
    public void run() {
        //Produce 1000 products
        for (int i = 0; i < 1000; i++) {
            queue.put(new Product());
            System.out.println("PRODUCED PRODUCT");
        }
    }
}
public class Consumer implements Runnable {
    private BlockingQueue queue;

    public Consumer(BlockingQueue queue) {
        this.queue = queue;
    }

    @Override
    public void run() {
        while (true) {
            Product product = queue.take();
            System.out.println("CONSUMED PRODUCT");
        }
    }
}

And here is the calling code:
BlockingQueue queue = new ArrayBlockingQueue(50);

Producer producer = new Producer(queue);
new Thread(producer).start();

Consumer consumer = new Consumer(queue);
new Thread(consumer).start();