Question

I’m getting this error reported when I run my test program:

*** glibc detected *** /home/me/work/co/myprog/build/myprog_test: free(): invalid next size (fast): 0x00000000006d7320 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7ffff706ab96]
/home/me/work/co/myprog/build/myprog_test(_ZN9__gnu_cxx13new_allocatorIdE10deallocateEPdm+0x20)[0x47715e]
/home/me/work/co/myprog/build/myprog_test(_ZN5boost7numeric5ublas15unbounded_arrayIdSaIdEED2Ev+0x38)[0x474aea]
/home/me/work/co/myprog/build/myprog_test(_ZN5boost7numeric5ublas6vectorIdNS1_15unbounded_arrayIdSaIdEEEED2Ev+0x19)[0x473177]
/home/me/work/co/myprog/build/myprog_test(_ZN5boost7numeric5ublas13vector_assignINS1_13scalar_assignENS1_6vectorIdNS1_15unbounded_arrayIdSaIdEEEEENS1_11zero_vectorIdS6_EEEEvRT0_RKNS1_17vector_expressionIT1_EENS1_10sparse_tagE+0x2bf)[0x47b2d9]
/home/me/work/co/myprog/build/myprog_test(_ZN5boost7numeric5ublas13vector_assignINS1_13scalar_assignENS1_6vectorIdNS1_15unbounded_arrayIdSaIdEEEEENS1_11zero_vectorIdS6_EEEEvRT0_RKNS1_17vector_expressionIT1_EE+0x26)[0x4782f4]
/home/me/work/co/myprog/build/myprog_test(_ZN5boost7numeric5ublas6vectorIdNS1_15unbounded_arrayIdSaIdEEEE6assignINS1_11zero_vectorIdS4_EEEERS6_RKNS1_17vector_expressionIT_EE+0x23)[0x4771fd]
/home/me/work/co/myprog/build/myprog_test(_ZN5boost7numeric5ublas6vectorIdNS1_15unbounded_arrayIdSaIdEEEEaSINS1_11zero_vectorIdS4_EEEERS6_RKNS1_16vector_containerIT_EE+0x4e)[0x474b52]
/home/me/work/co/myprog/build/myprog_test(_ZN9DataPointC1Ev+0x101)[0x4682d7]
/home/me/work/co/myprog/build/myprog_test(_ZN16Manager13processSampleEP6Sample+0x40)[0x46a49e]
/home/me/work/co/myprog/build/myprog_test(_ZN16Manager7addDataEP6Samplei+0x108)[0x46a8b6]
/home/me/work/co/myprog/build/myprog_test(_ZN16Manager14SetLogFileDataEP6Samplei+0x90)[0x46904a]
/home/me/work/co/myprog/build/myprog_test(_ZN36MyprogTest_testGetValuesToFilter_Test8TestBodyEv+0x6b)[0x4706a5]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8internal38HandleSehExceptionsInMethodIfSupportedINS_4TestEvEET0_PT_MS4_FS3_vEPKc+0x52)[0x49eb29]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS_4TestEvEET0_PT_MS4_FS3_vEPKc+0x45)[0x499b76]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing4Test3RunEv+0xb9)[0x487461]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8TestInfo3RunEv+0xf6)[0x487bb8]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8TestCase3RunEv+0xe1)[0x48815f]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8internal12UnitTestImpl11RunAllTestsEv+0x272)[0x48ce3e]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8internal38HandleSehExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0x52)[0x49ffcb]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0x45)[0x49aa62]
/home/me/work/co/myprog/build/myprog_test(_ZN7testing8UnitTest3RunEv+0x5e)[0x48bbee]
/home/me/work/co/myprog/build/myprog_test(main+0x3e)[0x4a8b76]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7ffff700d76d]
/home/me/work/co/myprog/build/myprog_test[0x466399]
======= Memory map: ========
00400000-004d2000 r-xp 00000000 08:01 11152935                           /home/me/work/co/myprog/build/myprog_test
006d1000-006d2000 r--p 000d1000 08:01 11152935                           /home/me/work/co/myprog/build/myprog_test
006d2000-006d3000 rw-p 000d2000 08:01 11152935                           /home/me/work/co/myprog/build/myprog_test
006d3000-006f4000 rw-p 00000000 00:00 0                                  [heap]
7ffff6fec000-7ffff71a1000 r-xp 00000000 08:01 12452670                   /lib/x86_64-linux-gnu/libc-2.15.so
7ffff71a1000-7ffff73a0000 ---p 001b5000 08:01 12452670                   /lib/x86_64-linux-gnu/libc-2.15.so
7ffff73a0000-7ffff73a4000 r--p 001b4000 08:01 12452670                   /lib/x86_64-linux-gnu/libc-2.15.so
7ffff73a4000-7ffff73a6000 rw-p 001b8000 08:01 12452670                   /lib/x86_64-linux-gnu/libc-2.15.so
7ffff73a6000-7ffff73ab000 rw-p 00000000 00:00 0
7ffff73ab000-7ffff73c0000 r-xp 00000000 08:01 12455438                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff73c0000-7ffff75bf000 ---p 00015000 08:01 12455438                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff75bf000-7ffff75c0000 r--p 00014000 08:01 12455438                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff75c0000-7ffff75c1000 rw-p 00015000 08:01 12455438                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff75c1000-7ffff76bc000 r-xp 00000000 08:01 12452678                   /lib/x86_64-linux-gnu/libm-2.15.so
7ffff76bc000-7ffff78bb000 ---p 000fb000 08:01 12452678                   /lib/x86_64-linux-gnu/libm-2.15.so
7ffff78bb000-7ffff78bc000 r--p 000fa000 08:01 12452678                   /lib/x86_64-linux-gnu/libm-2.15.so
7ffff78bc000-7ffff78bd000 rw-p 000fb000 08:01 12452678                   /lib/x86_64-linux-gnu/libm-2.15.so
7ffff78bd000-7ffff799f000 r-xp 00000000 08:01 794196                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7ffff799f000-7ffff7b9e000 ---p 000e2000 08:01 794196                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7ffff7b9e000-7ffff7ba6000 r--p 000e1000 08:01 794196                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7ffff7ba6000-7ffff7ba8000 rw-p 000e9000 08:01 794196                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7ffff7ba8000-7ffff7bbd000 rw-p 00000000 00:00 0
7ffff7bbd000-7ffff7bd5000 r-xp 00000000 08:01 12452672                   /lib/x86_64-linux-gnu/libpthread-2.15.so
7ffff7bd5000-7ffff7dd4000 ---p 00018000 08:01 12452672                   /lib/x86_64-linux-gnu/libpthread-2.15.so
7ffff7dd4000-7ffff7dd5000 r--p 00017000 08:01 12452672                   /lib/x86_64-linux-gnu/libpthread-2.15.so
7ffff7dd5000-7ffff7dd6000 rw-p 00018000 08:01 12452672                   /lib/x86_64-linux-gnu/libpthread-2.15.so
7ffff7dd6000-7ffff7dda000 rw-p 00000000 00:00 0
7ffff7dda000-7ffff7dfc000 r-xp 00000000 08:01 12452684                   /lib/x86_64-linux-gnu/ld-2.15.so
7ffff7fd6000-7ffff7fdb000 rw-p 00000000 00:00 0
7ffff7ff7000-7ffff7ffb000 rw-p 00000000 00:00 0
7ffff7ffb000-7ffff7ffc000 r-xp 00000000 00:00 0                          [vdso]
7ffff7ffc000-7ffff7ffd000 r--p 00022000 08:01 12452684                   /lib/x86_64-linux-gnu/ld-2.15.so
7ffff7ffd000-7ffff7fff000 rw-p 00023000 08:01 12452684                   /lib/x86_64-linux-gnu/ld-2.15.so
7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0                          [stack]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

I’ve run it through gdb, and the line that’s causing it is indirectly this constructor call, when it gets to the line in the constructor

DataPointPtr data(new DataPoint());
=>
gyro_stdev = zero_vector;

With these definitions for DataPoint and Ptr:

typedef boost::numeric::ublas::vector<double> vector;

struct DataPoint {
  DataPoint(void);
  long timestamp;
  bool derived;

  vector accel;
  vector accel_filtered;

  vector gyro_filtered;
  vector gyro_stdev;

  vector gyro;

  vector velocity;
  vector position;
  vector orientation;

  vector gravity;
};

typedef boost::shared_ptr<DataPoint> DataPointPtr;

boost::numeric::ublas::zero_vector<double> zero_vector(3);

DataPoint::DataPoint(void) {
  derived = false;

  accel = zero_vector;
  accel_filtered = zero_vector;

  gyro_filtered = zero_vector;
  gyro_stdev = zero_vector;

  gyro = zero_vector;

  position = zero_vector;
  velocity = zero_vector;
  orientation = zero_vector;
  gravity = zero_vector;

}

I’ve also run this same program through valgrind and I get an incredible amount of output, most of which looks something like this, varying only depending on which trigonometric function I’m using (sin, tan, atan2 etc.)

==4161== Conditional jump or move depends on uninitialised value(s)
==4161==    at 0x536CC9D: __sin_sse2 (s_sin.c:115)
==4161==    by 0x46B23C: AccelGyroManager::getGyroEstimate(long, boost::numeric::ublas::vector<double, boost::numeric::ublas::unbounded_array<double, std::allocator<double> > >, boost::numeric::ublas::vector<double, boost::numeric::ublas::unbounded_array<double, std::allocator<double> > >) (AccelGyroManager.cxx:659)
==4161==    by 0x46BD53: AccelGyroManager::deriveData(int) (AccelGyroManager.cxx:753)
==4161==    by 0x46957A: AccelGyroManager::GetEstimate(long, float*, float*, float*, float*, float*, float*) (AccelGyroManager.cxx:294)
==4161==    by 0x467150: AgmanDllTest_testDeriveInterpolatedData_Test::TestBody() (agman_test.cxx:144)
==4161==    by 0x49EB28: void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2090)
==4161==    by 0x499B75: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2126)
==4161==    by 0x487460: testing::Test::Run() (gtest.cc:2162)
==4161==    by 0x487BB7: testing::TestInfo::Run() (gtest.cc:2338)
==4161==    by 0x48815E: testing::TestCase::Run() (gtest.cc:2445)
==4161==    by 0x48CE3D: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4237)
==4161==    by 0x49FFCA: bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2090)
==4161==

This problem is a bit confusing to me. Is it possible that I’m running out of memory? This exact same constructor call succeeds about 400 times before this point in the program.

And valgrind’s reference to s_sin.c is also confusing, I can’t find this file anywhere..

Any advice or suggestions on what to look at or poke around with are much appreciated.

Answer 1

So I found my problem. As Mats Petersson suggested, my problem was in fact an array overrun. It turns out that I had this little gem buried in my utilities for my unit tests:

SampleArray createPredictableSamples(int count) {
  SampleArray samples(new Sample[100]);
  for (int i = 0; i < count; ++i) {
  ... do stuff
  }
  return samples;
}

(notice the static array creation of size 100, when the method is parameterized…)

So just remember, test code has bugs too!!

Here’s the method I used to discover this.

When I posted my question, I had made a bunch of changes since the last time I ran the tests. The output from running

valgrind --tool=memcheck --leak-check=yes ./test

was a little overwhelming because of this. So I backed out of most of those changes (using my version control to keep them available), and then re applied them one by one as functional sets.

Each time I introduced one change set, I re-ran the tests, and used valgrind to check for memory errors. Finally, I made one change (changing my test array size) that produced a huge amount of output. I searched through that (as valgrind suggests) for “writes” to uninitialized memory (i.e. for buffer overruns). And lo and behold, it lead me right to the problem, line-numbers and all.

Along the way, I also found a couple of other memory leaks in my test code. Now, I still have more work to do cleaning up test code for memory leaks, but I at least have gotten rid of the buffer overrun.

Thanks Mats, David, and vonbrand for pointing me in the right direction!