what are my options when i need to allocate virtual memory beyond the limits of system memory?
paging file is unlimited(limited by available disk space) so why could not i use it to allocate memory beyond the limits of physical memory limit or OS memory limits? why did they limit the virtual memory with the boundaries of address space? i know CPU could work with physical memory but why not OS handles this for me when i am accessing some portion of virtual memory?
The OS does take care of this for you. You do not have to think about when physical memory is fully utilized because the virtual memory system hides this from you. The limit on the address space of an application is a result of the number of bits that are allocated for storing memory addresses in the architecture.
Edit Re: comments
I think you may be confusing the amount of memory available to a given process with the total amount of virtual memory being managed by the OS, which is shared among processes. There is a limit on the total amount of committed virtual memory by all processes, but this commit limit is NOT the same as the address space limit for an individual process. The total commit limit is loosely the amount of physical memory + the pagefile size. So it’s possible to tune your pagefile size to increase or decrease this number.