Question

I’m trying to debug a simple cross-platform commandline program (a C parser, itself written in C) and running into something strange.

On Windows, when I run it on a small dataset (the source code of glib) it completes successfully, and when I run it on a large dataset (the source code of the Linux kernel) it exits with an out of memory error. I’m not sure whether the latter is a bug in my code or just a consequence of not just having optimized the memory consumption yet, so I’ve been trying to run it on Linux so I can get some feedback from valgrind.

On Linux (Ubuntu 11.04 x64 in VirtualBox), when I run my program on a small dataset it completes successfully, and when I run it on a large dataset Linux locks up hard enough I have to reset the entire virtual box (mouse pointer still moves but other than that it’s completely unresponsive; Windows task manager says the virtual box is using one hundred percent of a CPU core but not allocating memory).

I wouldn’t have expected a bug in my code to crash Linux unless I was writing something like a device driver, and when I try simple test cases that allocate too much memory, go into an infinite loop or both, Linux can handle them just fine. What kind of bug should I be looking for, or what am I missing?

Answer 1

This is most likely an infinite loop (easily done in a parser), which could easily take up 100% cpu or 100% ram.

Attach a debugger!

e.g. gdb
http://www.gnu.org/s/gdb/

gdb comes with gcc on Ubuntu etc…

Here’s a how-to: http://www.unknownroad.com/rtfm/gdbtut/gdbtoc.html

EDIT: just saw you already tried gdb. So, try running strace on it, it might give you a hint.

Further to that, try adding log messages to see how far the program gets (primitive, but it’ll work eventually!)