Question

There is a function that calls itself recursively infinitely.

This function has some arguments too.

For every function call the arguments and return address are pushed on the stack .

For each process there is fixed size of stack space that cannot grow dynamically like heap. And I guess each thread also has its own stack.

Now if a function is called recursively infinitely and process runs out of stack space , what will happen?

Will program crash? Will OS handle the situation? There is 4GB of address space so why cannot OS do something to increase stack size.

Answer 1

stack overflow.

In UNIX and compatible the process will get terminated throwing SIGSEGV or SIGSTKFLT signal.

In Windows the process will get terminated throwing exception STATUS_STACK_OVERFLOW.