Question

I have some project where I have a single producer thread which writes events into a buffer, and an additional single consumer thread which takes events from the buffer. My goal is to optimize this thing for a single dual-core machine to achieve maximum throughput.

Currently, I am using some simple lock-free ring buffer (lock-free is possible since I have only one consumer and one producer thread and therefore the pointers are only updated by a single thread).

#define BUF_SIZE 32768

struct buf_t { volatile int writepos; volatile void * buffer[BUF_SIZE]; 
    volatile int readpos;) };

void produce (buf_t *b, void * e) {
    int next = (b->writepos+1) % BUF_SIZE;
    while (b->readpos == next); // queue is full. wait
    b->buffer[b->writepos] = e; b->writepos = next;
}

void * consume (buf_t *b) {
    while (b->readpos == b->writepos); // nothing to consume. wait
    int next = (b->readpos+1) % BUF_SIZE;
    void * res = b->buffer[b->readpos]; b->readpos = next;
    return res;
}

buf_t *alloc () {
    buf_t *b = (buf_t *)malloc(sizeof(buf_t));
    b->writepos = 0; b->readpos = 0; return b;
}

However, this implementation is not yet fast enough and should be optimized further. I’ve tried with different BUF_SIZE values and got some speed-up. Additionaly, I’ve moved writepos before the buffer and readpos after the buffer to ensure that both variables are on different cache lines which resulted also in some speed.

What I need is a speedup of about 400 %. Do you have any ideas how I could achieve this using things like padding etc?

Answer 1

I’ve implemented the optimizations in the first code block of cafs answer. They actually gave some little speed up (thank you), however it is not yet enough. The second optimization which results in the cache being filled by column instead of by row results in a worse performance.

The idea of the consumer lagging behind the producer did not gave any speedup.

Now, I’m at 300%.

An additional change I have made was some hack regarding the volatile writepos and readpos variables:

void produce (void * e) {
    unsigned int oldpos = buffer.writepos;
    unsigned int next = (oldpos+1) % BUF_SIZE;
    while (next == buffer.rpos) { // rpos is not volatile
        buffer.rpos = buffer.readpos;
        usleep(1);
    }
    buffer.buffer[oldpos] = e; buffer.writepos = next;
}

and similar for the consume().

Additional changes to the struct lead to the following new buffer struct (in the global scope as it was suggested in one answer instead of on the heap).

#define STRIDE 16
#define STEPS 524288

struct buf_t {
    volatile unsigned int writepos;
    int offset [STRIDE - 1];
    unsigned int wpos;
    int offset2 [STRIDE - 1];
    volatile void * buffer[BUF_SIZE];
    int offset4 [STRIDE];
    volatile unsigned int readpos;
    int offset3 [STRIDE - 1];
    unsigned int rpos;
}

which gave the 300% speedup which was missing and pushed it below the performance limit I had to achieve.

If you have some additional hacks which could be used to increase the performance further, don’t hesitate to post them also 🙂

Thanks for help.