Question

I want to store ~3,000,000 double values at unsigned int indices under VS2010 C++. I use a std::tr1:unordered_map<unsigned int, double> for this purpose. Unfortunately, when I try to store value number 2^21, an exception is thrown (as if there is only space for 2^21-1 , i.e., some index can only use 20 Bits). I tried to rehash before storing the values, which did not work either.

Finally I ended up with some very basic test program (which showed even a little different behavior, but anyway):

    std::tr1::unordered_map<unsigned int, float> mapOut;
    //mapOut.rehash(SOMESIZE);
    for (unsigned int i=0; i<3000000; i++)
    {
        if (i%1000==0) std::cout << i << std::endl;
        mapOut[i] = 0.0;
    }

Some cases I checked:

1) If I do not rehash at all, the program takes a long break after the output according to i == 32000 (eventually 2^15) and then continues to i == 262000 (2^18). There it holds forever (with 100% CPU load, memory not increasing).

2) If I do a rehash(1000), it comes to i == 65000 (2^16) and holds forever (CPU load 100%, memory not increasing).

3) If I do a rehash(3000000), the loop is successfully finished, but the program never exits – i.e., obviously there is some problem with the destructor.

What is going on there, and even more important: What should I do about it?!

Many thanks for your help!

Answer 1

The following bug on Connect appears to be related to your problem: Visual C++: std::unordered_map Destructor Performance in Debug Configuration

The same problem occurs not just in the destructor, but also when the unordered_map is resized. It seems to have something to with invalidating iterators if iterator debugging is enabled.

They say it has been fixed for VC11. There are also several workarounds listed, but I haven’t tried them.

A very simple way to solve the performance problem for both debug and release builds is to set _SECURE_SCL=0 and _HAS_ITERATOR_DEBUGGING=0 in the project options under “Configuration Options / C/C++ / Preprocessor / Preprocessor definitions”, which completely disables all iterator debugging. This does also disable some security checks, however, so you need to be more careful in your own code. I believe both of these are the default under release builds already, so you shouldn’t need to change anything there.

This appears to be related to the problem of your sample code. I’m not sure about the original problem since you didn’t say what exception was thrown.