Question

I have a backtrace with something I haven’t seen before. See frame 2 in these threads:

Thread 31 (process 8752):
#0  0x00faa410 in __kernel_vsyscall ()
#1  0x00b0b139 in sigprocmask () from /lib/libc.so.6
#2  0x00b0c7a2 in abort () from /lib/libc.so.6
#3  0x00752aa0 in __gnu_cxx::__verbose_terminate_handler () from /usr/lib/libstdc++.so.6
#4  0x00750505 in ?? () from /usr/lib/libstdc++.so.6
#5  0x00750542 in std::terminate () from /usr/lib/libstdc++.so.6
#6  0x00750c65 in __cxa_pure_virtual () from /usr/lib/libstdc++.so.6
#7  0x00299c63 in ApplicationFunction()

Thread 1 (process 8749):
#0  0x00faa410 in __kernel_vsyscall ()
#1  0x00b0ad80 in raise () from /lib/libc.so.6
#2  0x00b0c691 in abort () from /lib/libc.so.6
#3  0x00b4324b in __libc_message () from /lib/libc.so.6
#4  0x00b495b6 in malloc_consolidate () from /lib/libc.so.6
#5  0x00b4b3bd in _int_malloc () from /lib/libc.so.6
#6  0x00b4d3ab in malloc () from /lib/libc.so.6
#7  0x08147f03 in AnotherApplicationFunction ()

When opening it with gdb and getting backtrace it gives me thread 1. Later I saw the weird state that thread 31 is in. This thread is from the library that we had problems with so I’d believe the crash is caused by it.

So what does it mean? Two threads simultaneously doing something illegal? Or it’s one of them, causing somehow abort() in the other one?

The OS is Linux Red Hat Enterprise 5.3, it’s a multiprocessor server.

Answer 1

Looks like it could be heap corruption, detected by malloc in thread 1, causing or caused by the error in thread 31.

Some broken piece of code overwriting a.o. the vtable in thread 31 could easily cause this.