Question

I have a strange problem freeing allocated memory in my mpi program:

Here is a code sample that produces the error for me:

void *out, *in;
int cnt = 2501; //if cnt<=2500: works perfectly. cnt>2500: crashes at free!

if((out = malloc(cnt * sizeof(double))) == NULL) 
    MPI_Abort(MPI_COMM_WORLD, MPI_ERR_OP);
if((in = malloc(cnt * sizeof(double))) == NULL) 
    MPI_Abort(MPI_COMM_WORLD, MPI_ERR_OP);

//Test data generation
//usage of MPI_Send and MPI_Reduce_local
//doing a lot of memcpy, assigning pointer synonyms to in and out, changing data to in and out

free(in);    //crashes here with "munmap_chunk(): invalid pointer" 
free(out);   //and here (if above line is commented out) with "double free or corruption (!prev)"

I ran it using valgrind:

 mpirun -np 2 valgrind  --leak-check=full --show-reachable=yes  ./foo

and got the following:

==6248== Warning: ignored attempt to set SIGRT32 handler in sigaction();
==6248==          the SIGRT32 signal is used internally by Valgrind
cr_libinit.c:183 cri_init: sigaction() failed: Invalid argument

==6248== HEAP SUMMARY:
==6248==     in use at exit: 0 bytes in 0 blocks
==6248==   total heap usage: 1 allocs, 1 frees, 25 bytes allocated
==6248== 
==6248== All heap blocks were freed -- no leaks are possible
==6248== 
=====================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   EXIT CODE: 134

Any ideas about how to track this error down? Note that it only appears if cnt>2500!

Answer 1

If you’re using GNU glibc, you can set the environment variable MALLOC_CHECK_ to 2 before running your program, to enable extra checking on memory-allocation calls—details here.