Question

First of all, I’m not sure this is a specific platform error. I’m using openssl library to generate big prime number and private/public keys. It ran fine when prime number is relatively small (128 bits), but I can’t compile and got an error when the prime number was getting bigger (256, 512 bits).

DH * params = DH_new(); 
params = DH_generate_parameters(512, 5, NULL, NULL);

DH_generate_key(params); // can't get through this point when prime bits getting bigger

above code produced an error:

 malloc: *** error for object 0x7ffba8403c88: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

I’m using openssl 1.0.1 on MAC OX 10.8.2. I’d appreciate for any advices.

UPDATE: Another way to produce this kind’s error is creating a thread before above code. Either way, it always gave me error at sub-routine DH_generate_key.

Answer 1

Error messages related to heap corruption such as this message from malloc() indicate that you have memory management bugs, such as using memory after freeing it, or overrunning the allocated memory. It is normal for such malloc() error messages to occur at different locations than where the actual programming error happened. Run your application in Valgrind to get some useful information on what’s going wrong.

Unrelated to the error, but you have a memory leak in these two lines of your code above:

DH * params = DH_new(); 
params = DH_generate_parameters(512, 5, NULL, NULL);

The pointer to the allocated DH structure is overwritten in the second line with a new DH structure returned by DH_generate_parameters(). The buffer allocated by DH_new() is lost. This is not the cause of the error message though.