Question

I’m writing a small zlib wrapper via P/Invoke calls. It runs perfectly on a 64-bit target (64-bit C# build, 64-bit DLL), but throws an AccessViolationException on a 32-bit target (32-bit C# build, 32-bit DLL).

Here’s the C# signature and code which throws the exception:

[DllImport(Program.UnmanagedDll, CallingConvention = CallingConvention.Cdecl)]
private static extern ZLibResult ZLibDecompress(byte[] inStream, uint inLength, byte[] outStream, ref uint outLength);

internal enum ZLibResult : byte {
        Success = 0,
        Failure = 1,
        InvalidLevel = 2,
        InputTooShort = 3
}

internal static ZLibResult Decompress(byte[] compressed, out byte[] data, uint dataLength) {
    var len = (uint) compressed.Length;
    fixed (byte* c = compressed) {
        var buffer = new byte[dataLength];
        ZLibResult result;
        fixed (byte* b = buffer) {
            result = ZLibDecompress(c, len, b, &dataLength);
        }
        if(result == ZLibResult.Success) {
            data = buffer;
            return result;
        }
        data = null;
        return result;
    }
}

And here’s the C code (compiled with MinGW-w64):

#include <stdint.h>
#include "zlib.h"

#define ZLibCompressSuccess         0
#define ZLibCompressFailure         1

__cdecl __declspec(dllexport) uint8_t ZLibDecompress(uint8_t* inStream, uint32_t inLength,
                                                     uint8_t* outStream, uint32_t* outLength)
{
    uLongf oL = (uLongf)*outLength;
    int result = uncompress(outStream, &oL, inStream, inLength);
    *outLength = (uint32_t)oL;
    if(result == Z_OK)
        return ZLibCompressSuccess;
    return ZLibCompressFailure;
}

I’ve looked over everything and can’t figure out why an access violation would be happening on a 32-bit build and not on a 64-bit build. ZLibDecompress works fine decompressing the same stream when called from a C app, but throws an access violation when called from my C# app.

Does anyone know why this could be happening?

EDIT:
Updated my code, still getting an access violation on 32-bit builds, but not 64-bit.

C# Code:

[DllImport(Program.UnmanagedDll, CallingConvention = CallingConvention.Cdecl)]
private static extern ZLibResult ZLibDecompress(
    [MarshalAs(UnmanagedType.LPArray)]byte[] inStream, uint inLength,
    [MarshalAs(UnmanagedType.LPArray)]byte[] outStream, ref uint outLength);

internal static ZLibResult Decompress(byte[] compressed, out byte[] data, uint dataLength) {
    var buffer = new byte[dataLength];
    var result = ZLibDecompress(compressed, (uint)compressed.Length, buffer, ref dataLength);
    if(result == ZLibResult.Success) {
        data = buffer;
        return result;
    }
    data = null;
    return result;
}

C Code:

__declspec(dllexport) uint8_t __cdecl ZLibDecompress(uint8_t* inStream, uint32_t inLength,
                                 uint8_t* outStream, uint32_t* outLength) {
    uLongf oL = (uLongf)*outLength;
    int result = uncompress(outStream, &oL, inStream, inLength);
    *outLength = (uint32_t)oL;
    if(result == Z_OK)
        return ZLibCompressSuccess;
    return ZLibCompressFailure;
}

Answer 1

    fixed (byte* b = buffer) {
        result = ZLibDecompress(c, len, b, &dataLength);
    }

No, that can’t work. The fixed keyword provides a highly optimized way to ensure that the garbage collector moving objects doesn’t cause trouble. It doesn’t do it by pinning the object (like the documentation says), it does it by exposing the b variable to the garbage collector. Which then sees it referencing the buffer and updates the value of b when it moves buffer.

That however can’t work in this case, a copy of the b value was passed to ZlibDecompress(). The garbage collector cannot update that copy. The outcome will be poor when a GC occurs while ZLibDecompress() is running, the native code will destroy the integrity of the garbage collected heap and that will eventually cause an AV.

You cannot use fixed, you must use GCHandle.Alloc() to pin the buffer.

But don’t do that either, you are helping too much. The pinvoke marshaller is already very good at pinning objects when necessary. Declare the instream and outstream arguments as byte[] instead of byte*. And pass the arrays directly without doing anything special. Also, the outlength argument should be declared ref int.