Question

what’s happening is i’m reading encryption packets, and I encounter a corrupted packet that gives back a very large random number for the length.

size_t nLengthRemaining = packet.nLength - (packet.m_pSource->GetPosition() - packet.nDataOffset);

seckey.SecretValues.m_data.resize(nLengthRemaining);

In this code m_data is a std::vector<unsigned char>. nLengthRemaining is too large due to a corrupted data packet, therefore the resize function throws. The problem isn’t that resize throws (we handle the exceptions), but that resize has corrupted memory already and this leads to more exceptions later.

What I want to do is know if the length is too large before I call resize, then only call resize if it’s ok. I have tried putting this code before the call to resize:

std::vector<unsigned char>::size_type nMaxSize = seckey.SecretValues.m_data.max_size();
if(seckey.SecretValues.m_data.size() + nLengthRemaining >=  nMaxSize) {
    throw IHPGP::PgpException("corrupted packet: length too big.");
}
seckey.SecretValues.m_data.resize(nLengthRemaining);

This code is using the std::vector max_size member function to test if the nLengthRemaining is larger. That must not be reliable though, because nLengthRemaining is still less than nMaxSize, but apparently still large enough to cause resize to have a problem (nMaxSize was 4xxxxxxxxx and nLengthRemaining is 3xxxxxxxxx).

Also, I haven’t determine what exception resize is throwing. It’s not a std::length_error and it’s not a std::bad_alloc. What exception it’s throwing really isn’t too important to me, but i’m curious to know.

btw, just so you know, this code does work correctly in normal cases. This case of a corrupted data packet is the only place where it goes crazy. Please help! thanks.

UPDATE:

@Michael. For now i’ll just ignore the packet if it’s larger than 5 MB. I’ll discuss with other team members about possibly validating the packets (it may already be there and I just don’t know it). I’m starting to think it really is a bug in our version of STL, the exception it throws isn’t even a std::exception, which surprized me. I’ll try to find out from my supervisor what version of STL we’re running too (how would I check?).

ANOTHER UPDATE:
I just prove that it is a bug in the STL version I’m using on my Visual Studio 6 development machine. I wrote this sample app:

// VectorMaxSize.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <vector>
#include <iostream>
#include <math.h>
#include <typeinfo>

typedef std::vector<unsigned char> vector_unsigned_char;

void fill(vector_unsigned_char& v) {
    for (int i=0; i<100; i++) v.push_back(i);
}


void oput(vector_unsigned_char& v) {
    std::cout << "size: " << v.size() << std::endl;
    std::cout << "capacity: " << v.capacity() << std::endl;
    std::cout << "max_size: " << v.max_size() << std::endl << std::endl;
}

void main(int argc, char* argv[]) {
    {
        vector_unsigned_char v;

        fill(v);

        try{
            v.resize(static_cast<size_t>(3555555555));
        }catch(std::bad_alloc&) {
            std::cout << "caught bad alloc exception" << std::endl;
        }catch(const std::exception& x) {
            std::cerr << typeid(x).name() << std::endl;
        }catch(...) {
            std::cerr << "unknown exception" << std::endl;
        }

        oput(v);    
        v.reserve(500);
        oput(v);
        v.resize(500);
        oput(v);
    }

    std::cout << "done" << std::endl;
}

On my VS6 dev machine it has the same behavior has the encryption project, it causes all kinds of havoc. When I build and run it on my Visual Studio 2008 machine, resize will throw a std::bad_alloc exception and the vector will not be corrupted, just as we would have expected! Time for some EA Sport NCAA football hehe!

Answer 1

I think that vector::max_size() is pretty much always a ‘hard coded’ thing – it’s independent of how much memory the system/library is prepared to dynamically allocate. Your problem seems to be a bug in the vector implementation that corrupts things when an allocation fails.

‘Bug’ might be too strong of a word. vector::resize() is defined in terms of vector::insert() and the standard says this about vector::insert():

If an exception is thrown other than by the copy constructor or assignment operator of T there are no effects

So it seems like there may be times when the resize() operation is allowed to corrupt a vector, but it would still be nice if the operation were exception safe (and I think it wouldn’t be out of line to expect the library to do that, but maybe it’s harder than I imagine).

You seem to have a couple reasonable options:

• change or update to a library that doesn’t have the corruption bug (what compiler/library version are you using?)
• instead of checking against vector::max_size() set nMaxSize to your own reasonable maximum and do what you have above but using that threshold instead.

---

Edit:

I see that you’re using VC6 – there’s definitely a bug in vector::resize() that might have something to do with your problem, though looking at the patch I honestly don’t see how (actually it’s a bug in vector::insert(), but as mentioned, resize() calls insert()). I’d guess it would be worthwhile to visit Dinkumwares’ page for bug fixes to VC6 and apply the fixes.

The problem might also have something to do with the <xmemory> patch on that page – it’s unclear what the bug is that’s discussed there, but vector::insert() does call _Destroy() and vector<> does define the name _Ty so you might be running into that problem. One nice thing – you won’t have to worry about managing the changes to the headers, as Microsoft is never touching them again. Just make sure the patches make it into version control and get documented.

Note that Scott Meyers in “Effective STL” suggests using SGI’s or STLPort’s library to get better STL support than comes with VC6. I haven’t done that so I’m not sure how well those libraries work (but I also haven’t used VC6 with STL very much). Of course, if you have the option to move to a newer version of VC, by all means do it.

---

One more edit:

Thanks for the test program…

VC6’s _Allocate() implementation for the default allocator (in <xmemory>) uses a signed int to specify the number of elements to allocate, and if the size passed in is negative (which apparently is what you’re doing – certainly in the test program you are) the _Allocate() function forces the requested allocation size to zero and proceeds. Note that a zero-sized allocation request will pretty much always succeed (not that vector checks for a failure anyway), so the vector::resize() function merrily tries to move its contents into the new block, which isn’t quite big enough to say the least. So the heap gets corrupted, it’ll likely hit a invalid memory page, and regardless – your program is hosed.

So the bottom line is don’t ever ask VC6 to allocate more than INT_MAX objects in one go. Probably not a great idea in most circumstances (VC6 or otherwise).

Also, you should keep in mind that VC6 uses a pre-standard idiom of returning 0 from new when an allocation fails rather than throwing bad_alloc.