Question

I have a class as follows:

typedef struct grid_cell_type {
int x;
int y;
grid_cell_type(int x0, int y0){
    x=x0;
    y=y0;
}

} grid_cell;

I’ll be pumping approximately 100 million of these through a queue.

Right now, this happens as follows:

my_queue.push(new grid_cell(x0,y0));

The individual piece-wise allocation of all these objects seems as though it is probably not as quick as some mass-allocation.

Any thoughts as to the best strategy to pursue here?

Answer 1

These are small and self-contained objects – put them directly in the queue instead of putting the pointers.

• In fact, on a 64-bit system and assuming int is 32-bit (which it is, for example, under Visual C++), the pointer will be as large as the object itself! So even if you have a bulk allocator, you still pay this price.
• The general memory allocator will not just be expensive time-wise, it will also have a per-object overhead, which in this case will dwarf the object itself (does not apply for bulk allocator).

While you could devise a fairly efficient “bulk” allocation scheme, I think it’s simpler to sidestep the issue and altogether avoid the individual object allocations.

— EDIT —

You can push elements to the std::queue like this:

struct grid_cell {

    grid_cell(int x0, int y0) {
        x=x0;
        y=y0;
    }

    int x;
    int y;

};

// ...

std::queue<grid_cell> q;

q.push(grid_cell(0, 0));
q.push(grid_cell(0, 1));
q.push(grid_cell(0, 2));
q.push(grid_cell(1, 0));
q.push(grid_cell(1, 1));
q.push(grid_cell(1, 2));

For the std::priority_queue, you’d need to decide how you want to order the elements.

— EDIT 2 —

@Richard Your code is quite different.

• For each push, your code would allocate a new block of dynamic memory, construct the object in it (i.e. assign x and y) and then push the pointer to that block of memory to the queue.
• My code constructs the object directly in its “slot” within the larger memory block that was pre-allocated by the queue itself. And as you already noted, few big allocations
  are better than many small ones.

Your code is:

1. prone to memory leaks
2. you pay for extra storage for pointers,
3. prone to memory fragmentation and
4. there is a per-object overhead, as I already mentioned.

A specialized bulk allocator could remove the last two problems but why not remove them all?

— EDIT 3 —

As for speed, the general dynamic memory allocation is expensive (about 40-50 machine instructions for best allocators).

The specialized block allocator would be much faster, but you still have an issue of memory latency: keeping everything nicely together is guaranteed to achieve better cache locality and be much more suitable for CPU’s prefetching logic than repeatedly “jumping” between the queue and the actual objects by de-referencing pointers.