Question

Here is a simple C++ program which takes in a size arguments and allocates an array of integers of that size. program is compiled using g++ under Linux running on a virtual machine with 32bit architecture.
When the application is called with argument (array size) above 1073741823* I get

Segmentation fault (core dumped)

error and with a value slightly smaller than that I get.

terminate called after throwing an instance of ‘std::bad_alloc’
what(): std::bad_alloc Aborted (core dumped)

here is the code:

/* dynamicAlloc.cpp */
#include <iostream> 
#include <stdlib.h> //for atoi
#include <cstdlib> // for rand() and srand()
#include <ctime> // for time()

using namespace std;
#define STR_LEN 256

int main(int argc, char* argv[])
{
    srand(time(0)); 

    unsigned int iArraySize = 1;
    if(argc < 2)
        return -1;

    iArraySize = atoi(argv[1]);

    int *pnValue = new int[iArraySize];
    if(pnValue == NULL)
    {
        cout << "cannot allocate array" << endl;
        return -2;
    }

    for(unsigned int iCounter = 0; iCounter < iArraySize; iCounter++)
    {
        pnValue[iCounter] = rand(); 
    }

    delete[] pnValue; 
    return 0;
}

Why am I getting two different errors?

also if I have a very big data that requires billions of data to be processed/massaged, do I have to use a database to handle such large amount of data or is there another method of handling large data sets?

thank you for reading

*1073741823 = (2 ^ 32(bit address)) / 4(int size in byte)) – 1

UPDATE

The output of ulimit -a is:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3808
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3808
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

The Ram size is configured to be 512MB. I thought since every program has it’s own virtual page (for a 32bit arch it’s 4GB I think) then I can use all of that virtual memory.

Answer 1

You don’t have 4 gigabytes of memory available to you on a 32 bit machine. You have 4 gig total. What your program can access is less than that.

You get a segfault with that huge value because you are creating an illegal address. The underlying machinery doesn’t even get a chance to try to allocate that memory. It goes kaboom before even attempting to make the allocation.

With a slightly lesser value, the address is valid but something is preventing you from allocating that large a chunk of memory. You might not have that much memory, or you might have a limit that prevents you from taking that much memory. If it’s a soft limit you can raise it. If it’s a hard limit, you need sys admin privileges to raise it.