Question

For a c++ arm application I need to trace the memory allocations. To this I am using the gcc memory hooks. For now I am printing the allocations and deallocations, see code below.

However, the malloc‘s and free‘s don’t add up. Sometimes I see a free on a memory block that pass through didn’t the malloc hook before. Or memory is freed twice. Of course this could be a bug in my code,although I don’t get a segfault. But I also see that malloc sometimes returns a pointer that it has returned before and there has been no free in meantime (at least my free hook wasn’t called).

So my guess is that certain malloc‘s and free's are not passed through my hooks. Note that when I only trace the c++ allocations then things do add up nicely.

Does anyone have any ideas?

#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>
#include <new>
#include <unistd.h>
#include <string.h>
#include <malloc.h>

pthread_mutex_t lock = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

static void push_memhooks();
static void pop_memhooks();

static void *malloc_hook(size_t size, const void *ret)
{

    pthread_mutex_lock(&lock);

    pop_memhooks();

    void *mem = malloc(size);

    if (mem) {
        printf("malloc %p\n", mem);
    }

    push_memhooks();

    pthread_mutex_unlock(&lock);

    return mem;
}

static void *realloc_hook(void* ptr, size_t size, const void *ret)
{
    pthread_mutex_lock(&lock);

    pop_memhooks();

    void* mem = realloc(ptr, size);

    if (mem) {
        printf("realloc %p -> %p\n", ptr, mem);
    }

    push_memhooks();

    pthread_mutex_unlock(&lock);

    return mem;
}

static void* memalign_hook(size_t boundary, size_t size, const void *ret)
{
    pthread_mutex_lock(&lock);
    pop_memhooks();

    void* mem = memalign(boundary, size);

    if (mem) {
        printf("memalign %p\n", mem);
    }

    push_memhooks();

    pthread_mutex_unlock(&lock);

    return mem;
}

static void free_hook(void *mem, const void *ret)
{
    pthread_mutex_lock(&lock);

    pop_memhooks();

    free(mem);

    printf("free %p\n", mem);

    push_memhooks();

    pthread_mutex_unlock(&lock);
}

void *operator new(size_t size)
{
    void* mem = malloc(size);

    if (!mem) {
        throw std::bad_alloc();
    }

    return mem;
}

void operator delete(void* mem)
{
    free(mem);
}

void *operator new[](size_t size)
{       
    void* mem = malloc(size);

    if (!mem) {
        throw std::bad_alloc();
    }

    return mem;
}

void operator delete[](void* mem)
{
    free(mem);
}

static int memhooks = 0;

static void push_memhooks()
{
    if (++memhooks == 1) {
        __malloc_hook = malloc_hook;
        __realloc_hook = realloc_hook;
        __free_hook = free_hook;
        __memalign_hook = memalign_hook;
    }
}

static void pop_memhooks()
{
    if (--memhooks == 0) {
        __malloc_hook = NULL;
        __realloc_hook = NULL;
        __free_hook = NULL;
        __memalign_hook = NULL;
    }
}

static void install_memhooks ()
{
    push_memhooks();
}

void (*__malloc_initialize_hook)(void) = install_memhooks;

For example, I get the following output when I grep the trace for a pointer which shows the strange behaviour.

<snip>
malloc 0x8234818
free 0x8234818
malloc 0x8234818
malloc 0x8234818
free 0x8234818
<snip>

Notice the two consecutive malloc’s.

Solution: As Chris mentioned in his answer, there’s a race condition in the code above. Unfortunately, the malloc hooks cannot be safely used in a multithreaded environment when removing and reinstalling the hooks the way I do. For the same reason, mcheck cannot be used in multithreaded apps (http://sources.redhat.com/bugzilla/show_bug.cgi?id=9939).

Implementing malloc/realloc/free and calling the libc versions using dlsym(RTLD_NEXT, "malloc") didn’t work either. First, dlsym calls calloc, so special care is needed to prevent infinite recursion here. Second, when calling the libc malloc, the process hangs. Furthermore, I see that my __malloc_initialize_hook doesn’t get called. So I guess that by providing my own malloc implementation, the libc malloc isn’t properly initialized.

My current solution has the dlmalloc implementation embedded to remove the dependency on the libc malloc. Now I don’t have to continuously remove/reinstall the malloc hooks. I install the hooks once and my hooks allocate memory using dlmalloc.

Answer 1

If you’re running in a multithreaded environment, you have a race condition that might cause you to miss calls to malloc/free. When your malloc_hook function is called, it unhooks all the hooks, calls malloc and then rehooks the hooks. If some other thread calls malloc/free while the hooks are unhooked, you won’t see that call. Your mutex doesn’t help, as while unhooked, a malloc/free call won’t call your hook function, so won’t wait on the mutex.

edit

My preferred way of hooking/intecepting malloc in a program is to just intercept the calls from my program by using macros, without worrying about calls from within the stdlib. Create a wrap_malloc file with:

#define malloc(sz)     wrap_malloc(sz, __FILE__, __LINE__)
#define free(p)        wrap_free(p, __FILE__, __LINE__)
#define realloc(p, sz) wrap_realloc(p, sz, __FILE__, __LINE__)
#define calloc(s1, s2) wrap_calloc(s1, s2, __FILE__, __LINE__)

then compile all my code with -imacros wrap_malloc. The file that defines wrap_malloc and friends just needs the appropriate #undefs, but no other changes to the code are needed.