Question

I need to have two buffers (A and B) and when either of the buffers is full it needs to write its contents to the “merged” buffer – C. Using memcopy seems to be too slow for this operation as noted below in my question. Any insight?’

I haven’t tried but I’ve been told that memcopy will not work. This is an embedded system. 2 buffers. Both of different sizes and when they are full dumb to a common ‘C’ buffer which is a bigger size than the other two.. Not sure why I got down rated..

Edit: Buffer A and B will be written to prior to C being completely empty.

The memcopy is taking too long and the common buffer ‘C’ is getting over run.

Answer 1

The only way you can merge two buffers without memcpy is by linking them, like a linked list of buffer fragments (or an array of fragments).

Consider that a buffer may not always have to be contiguous. I’ve done a lot of work with 600dpi images, which means very large buffers. If you can break them up into a sequence of smaller fragments, that helps reducing fragmentation as well as unnecessary copying due to buffer growth.

In some cases buffers must be contiguous, if your API / microcontroller mandates it. For example, Windows bitmap functions require continuity. You could try to use the C realloc function, but it might internally work like the combination of malloc+memcpy+free. Either way, as others have said earlier, memcpy is supposed to be the fastest possible way of copying contiguous buffers.

If the buffer must be contiguous, you could reserve a large address space and commit it on demand. The implementation depends on the platform. For example, on Win32 the VirtualAlloc function can do that. This gives you a very large contiguous buffer, of which only a portion is allocated (committed). Later you can commit further pages as the buffer needs to grow. This trick requires the concept of virtual memory, which may not be available on a microcontroller.