I am trying to mmap some large files but failed : I can’t mmap any file that’s larger than around 2.7 GB.
By doing ulimit -a, it says my virtual memory is unlimited. I then did cat /proc/meminfo it says my system has 3GB Memfree and 2GB Swapfree.
I am assuming I could mmap a file up to 5GB. I tried everything I learned on the internet, including using MAP_NORESERVE flag and set overcommit_memory to 1. It’s still the same, Cannot allocate memory. What could possibly go wrong?
I am using 64 bit cpu and a 32 bit linux system. Please let me know if you need anymore information. I would really appreciate any advice or tips.
Your application simply doesn’t have enough address space (useable memory addresses) to map a file that large.
Under Linux, a 32 bit process running on a 32 bit kernel typically has around 3GB of address space available, and a 32 bit process running on a 64 bit kernel has 4GB of address space available. If you need more than that, you will need to compile your application as a 64 bit process. This will give you at least 128TB of address space.