Question

I’ve avoided trying to do anything with operator new for many years, due to my sense that it is a quagmire on Windows (especially using MFC). Not to mention, unless one has a very compelling reason to mess with global (or even class) new and delete, one should not.

However, I have a nasty little memory corruption bug, and I’d very much like to track it down. I get messages from the CRT debug allocator indicating that previously freed memory was overwritten. This message is displayed only during a later allocation call, when it tries to reuse a block (I believe this is how it works, anyway).

Due to the portion of code in question, the error message and point of corruption are very unrelated. All I know is “something somewhere overwrote some memory that was previously freed with a single null byte.” (I ascertained this by using the debugger and watching the memory referred to by the debug heap over several different runs).

Having exhausted the obvious ideas as to where the culprit might be, I’m left with trying to do something more rigorous. It occurred to me that it would be ideal if I could cause each freed block to become a no-access page of memory so that the writer would be immediately caught by the CPU’s MMC! A little bit of searching later, and I found someone who had implemented something along those lines:

http://www.codeproject.com/Articles/38340/Immediate-memory-corruption-detection

His code is buried under a ton of reinvent-the-wheel code, but extracting the core concept was pretty easy, and I have done so.

The problem I have now is that MFC redfines new as DEBUG_NEW, and then further defines a slew of debug interfaces down to the CRT. In addition, it does define the global operator new and delete. Hence, as far as C++ is concerned, the “user” is trying to replace global operator new and delete twice, and hence I get a linker error to the effect ‘symbol already defined.’

Looking around the internet, and SO, I see a few promising articles, but none which ultimately have anything positive to say about replacing global operator new/delete for MFC.

How to properly replace global new & delete operators
Is it possible to replace the memory allocator in a debug build of an MFC application?

I am already aware:

• MFC/CRT already provides rich debugging tools for memory allocation.

Well, it provides what it provides – such as the message that got me rolling down this path in the first place. I now know that a corruption is occurring, but that’s awfully weak-sauce!

What I would like to supply is guarded-allocation (or even just guarded deallocation). This is clearly possible by using a lot of virtual address space and placing every allocation in isolation, which is horribly wasteful of memory. Okay, yeah, can’t see the down side when this is debug-only code useful for special-purpose moments like now.

So, I’m desperately seeking solutions to the following

1. Force the compiler to be copacetic with my global operator new/delete despite the CRT/MFC supplied one.
2. Find another way to hook the MFC/CRT _heap_alloc_dbg chain to bottom out at using my own code in place of theirs, for the pen-ultimate allocation (i.e. I’ll allocate via the OS’s VirtualAlloc/VirtualFree to supply memory for new and/or malloc).

Does anyone know of answers, or good articles to read that might shed some light on how that these may be accomplished?

Other ideas:

1. Replace the CRT’s new/delete at runtime using a thunk technique.
2. Some other approach entirely?!

Further investigation:

• This article is pretty cool… it gives a way for me to patch the global new/delete operators at runtime. However, as the article points out, it’s a bit hackish (however, since I only need this for debug builds, that’s not a big deal) http://zeuxcg.blogspot.com/2009/03/fighting-against-crt-heap-and-winning.html
  • So although this is getting at what I want (a mechanism to replace the CRT memory allocation functions), this implementation is pretty far out of date, and so far my attempts to make it work have run into myriad issues. I think it’s just too hacked to the version it was originally created for, and only for a relatively simple console use (i.e. C, not even C++, and jettisoning most of the debugging features provided by the microsoft CRT). Hence, although a super-cool idea, one that ultimately would cost many hours of effort to make work with the current VS2010 dev studio, and hence not worth it (to me).
• Apparently there is a well-known version of this idea: http://en.wikipedia.org/wiki/Electric_Fence Which unfortunately even the Windows port I found http://code.google.com/p/electric-fence-win32/ fails to override the CRT properly, but asks that you modify all of your source code to access the electric fence heap allocation code. 🙁

Update 5/3/2012:

• And now I discover that Windows already provides an implementation of Electric Fence, accessible via GFLAGS debugging tool http://support.microsoft.com/kb/286470 This can be turned on and off external to the application being tested. It’s essentially the same technology as I was interested in, and has the features in the DUMA project (a branch of the Electric Fence – http://duma.sourceforge.net/

Answer 1

The MSVCRT debug heap is actually pretty good and has some useful features you can use, such as breakpoint on the nth allocation etc.

http://msdn.microsoft.com/en-us/library/974tc9t1(v=VS.80).aspx

Among other things you can insert an allocation hook which outputs debugging information etc which you can use to debug this sort of issue.

http://msdn.microsoft.com/en-us/library/z2zscsc2(v=vs.80).aspx

In your case all you really need to do is output the address, and file and line of each allocation. Then when you experience a corrupt block, find the block whose address immediately precedes it, which will almost certainly be the one which overran. You can use the memory view in the Visual Studio debugger to look at the memory address which was corrupted and see the preceding blocks. That should tell you all you need to know to find out when it was allocated.

The debug heap also has a numerical allocation ID on each block allocated, and can break on the nth! allocation, so if you can get a reasonably consistent repro, so the same numerical block is corrupted each time, then you should be able to use the “break on nth” functionality to get a full call stack for the time it was allocated.

You may also find _CrtCheckMemory useful to find out if corruption has occurred much earlier. Just call it periodically, and once you have the bug bracketed (error didn’t occur in one, did occur in the other) move them closer and closer together.