Question

I need to store a huge number of elements in a std::vector (more that the 2^32-1 allowed by unsigned int) in 32 bits. As far as I know this quantity is limited by the std::size_t unsigned int type. May I change this std::size_t by casting to an unsigned long? Would it resolve the problem?

If that’s not possible, suppose I compile in 64 bits. Would that solve the problem without any modification?

Answer 1

size_t is a type that can hold size of any allocable chunk of memory. It follows that you can’t allocate more memory than what fits in your size_t and thus can’t store more elements in any way.

Compiling in 64-bits will allow it, but realize that the array still needs to fit in memory. 2³² is 4 billion, so you are going to go over 4 * sizeof(element) GiB of memory. More than 8 GiB of RAM is still rare, so that does not look reasonable.

I suggest replacing the vector with the one from STXXL. It uses external storage, so your vector is not limited by amount of RAM. The library claims to handle terabytes of data easily.

(edit) Pedantic note: size_t needs to hold size of maximal single object, not necessarily size of all available memory. In segmented memory models it only needs to accommodate the offset when each object has to live in single segment, but with different segments more memory may be accessible. It is even possible to use it on x86 with PAE, the “long” memory model. However I’ve not seen anybody actually use it.