Question

My application is mostly organised in layers so I found that something like the APR memory pools would be the best way.

While reading on SO about C++ placement new posts here & here, and a more generic C allocation question I was thinking about hand-crafting a hierarchical pool allocator as suggested in one post, but in the pure NYI tradition I’m first asking if something like this already exists.

It could also have the nice property of being able to give back unused memory to the OS (since allocation could be done with mmap(MAP_ANON)) or could be allocating from the stack as suggested Ferrucico here.

Answer 1

I know of another good hierarchical memory allocator, but it calls malloc underneath the covers.

talloc is a hierarchical pool based memory allocator with destructors. It is the core memory allocator used in Samba4, and has made a huge difference in many aspects of Samba4 development.

To get started with talloc, I would recommend you read the talloc guide.

That being said, Glibc’s malloc already uses mmap(MAP_ANON) for allocation larger than mmap_threshold, which you can set via mallopt(M_MMAP_THRESHOLD, bytes). By default it is dynamically adjusted between

/*
  MMAP_THRESHOLD_MAX and _MIN are the bounds on the dynamically
  adjusted MMAP_THRESHOLD.
*/

#ifndef DEFAULT_MMAP_THRESHOLD_MIN
#define DEFAULT_MMAP_THRESHOLD_MIN (128 * 1024)
#endif

#ifndef DEFAULT_MMAP_THRESHOLD_MAX
  /* For 32-bit platforms we cannot increase the maximum mmap
     threshold much because it is also the minimum value for the
     maximum heap size and its alignment.  Going above 512k (i.e., 1M
     for new heaps) wastes too much address space.  */
# if __WORDSIZE == 32
#  define DEFAULT_MMAP_THRESHOLD_MAX (512 * 1024)
# else
#  define DEFAULT_MMAP_THRESHOLD_MAX (4 * 1024 * 1024 * sizeof(long))
# endif
#endif

Watch out if you lower it; by default no more than #define DEFAULT_MMAP_MAX 65536 pieces will be allocated using mmap. This can be changed with mallopt(M_MMAP_MAX, count), but using many mmaps has an overhead.

The environment variables MALLOC_MMAP_THRESHOLD_ etc. will also set these options.

Obviously, memory that malloc allocates with mmap is freed with munmap. I’m not sure if any of this is documented anywhere outside of Glibc’s source code, or has any compatibility guarantees.