Question

I have an existing algorithm and I need to optimize it sligtly if it is possible. Changing a lot in this algorithm is not an option at the moment. The algoritm works with instance of std::vector< std::vector<unsigned char> >. It looks like this:

typedef std::vector<unsigned char> internal_vector_t;
std::vector< internal_vector_t > internal_vectors; 

while (fetching lots of records) {
   internal_vector_t tmp;
   // reads 1Mb of chars in tmp...
   internal_vectors.push_back(tmp);
   // some more work
}

// use this internal_vectors

The algorithm inserts a lot of times in internal_vectors instances of internal_vector_t using push_back(). Most of instances of internal_vector_t have size 1 Mb. Since the size of the internal_vectors is unknown no reserve() is done beforehand.

The first thing that I don’t understand is what is happening when internal_vectors reachs its current capacity, needs to allocate a new block and copy its current content in the bigger block of memory. Since most of the blocks are 1Mb in size copying is a long operation. Should I expect that a compiler (gcc 4.3, MS VC++ 2008) will manage to optimize it in order to avoid copying?

If copying is unavoidable will changing to std::deque help? I consider std::deque because I still need accessing by index like internal_vectors[10]. Like this:

typedef std::vector<unsigned char> internal_vector_t;
std::deque< internal_vector_t > internal_vectors; 
// the same while

As far as I understand std::deque does not need relocate that was once allocated. Am I right that std::deque in this situation will requere less allocation and copying on push_backs?

---

Update:
1) According to DeadMG MSVC9 does this type of optimization (The Swaptimization – TR1 Fixes In VC9 SP1). gcc 4.3 probably doesn’t do this type of optimization.

2) I have profiled the version of the algorithm that use std::deque< std::vector<unsigned char> > and I see that its performace is better.

3) I have also made use of using swap that was suggested by Mark Ransom. Using this improved the performance:

   internal_vector_t tmp;
   internal_vectors.push_back(empty);
   tmp.swap(internal_vectors.back());

Answer 1

MSVC9 implements something known as “swaptimization” for it’s Standard containers. It’s a weaker version of move semantics. When the external vector is resized, it will not copy the internal vectors.

However, you’d do best simply upgrading your compiler to MSVC10 or GCC (4.5, I think it is) which will give you move semantics, which makes such operations vastly more efficient. Of course, a std::deque is probably still the smarter container, but move semantics are performance-beneficial in many, many places.