Question

Just the question stated, how can I use mmap() to allocate a memory in heap? This is my only option because malloc() is not a reentrant function.

Answer 1

Why do you need reentrancy? The only time it’s needed is for calling a function from a signal handler; otherwise, thread-safety is just as good. Both malloc and mmap are thread-safe. Neither is async-signal-safe per POSIX. In practice, mmap probably works fine from a signal handler, but the whole idea of allocating memory from a signal handler is a very bad idea.

If you want to use mmap to allocate anonymous memory, you can use MAP_ANON/MAP_ANONYMOUS.

p = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

At one point this was not strictly portable, and systems differed in which spelling they supported so you should write preprocessor conditionals to use whichever is available, but POSIX has since adopted both spellings as standard.

The traditional but ugly version, from long ago before MAP_ANON was a thing, is:

int fd = open("/dev/zero", O_RDWR);
p = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
close(fd);

Note that MAP_FAILED, not NULL, is the code for failure.