Question

UPDATE: There is now an accepted answer that “works”. You should never, ever, ever, ever use it. Ever.

---

First let me preface my question by stating that I’m a game developer. There’s a legitimate – if highly unusual – performance-related reason for wanting to do this.

---

Say I have a C# class like this:

class Foo
{
    public int a, b, c;
    public void MyMethod(int d) { a = d; b = d; c = a + b; }
}

Nothing fancy. Note that it is a reference type that contains only value types.

In managed code I’d like to have something like this:

Foo foo;
foo = Voodoo.NewInUnmanagedMemory<Foo>(); // <- ???
foo.MyMethod(1);

What would the function NewInUnmanagedMemory look like? If it can’t be done in C#, could it be done in IL? (Or maybe C++/CLI?)

Basically: Is there a way – no matter how hacky – to turn some totally arbitrary pointer into an object reference. And – short of making the CLR explode – damn the consequences.

(Another way to put my question is: “I want to implement a custom allocator for C#”)

This leads to the follow-up question: What does the garbage collector do (implementation-specific, if need be) when faced with a reference that points outside of managed memory?

And, related to that, what would happen if Foo had a reference as a member field? What if it pointed at managed memory? What if it only ever pointed at other objects allocated in unmanaged memory?

Finally, if this is impossible: Why?

---

Update: Here are the “missing pieces” so far:

#1: How to convert an IntPtr to an object reference? It might be possible though unverifiable IL (see comments). So far I’ve had no luck with this. The framework seems to be extremely careful to prevent this from happening.

(It would also be nice to be able to get the size and layout information for non-blittable managed types at runtime. Again, the framework tries to make this impossible.)

#2: Assuming problem one can be solved – what does the GC do when it encounters an object reference that points outside of the GC heap? Does it crash? Anton Tykhyy, in his answer, guesses that it will. Given how careful the framework is to prevent #1, it does seem likely. Something that confirms this would be nice.

(Alternatively the object reference could point to pinned memory inside the GC heap. Would that make a difference?)

Based on this, I’m inclined to think that this idea for a hack is impossible – or at least not worth the effort. But I’d be interested to get an answer that goes into the technical details of #1 or #2 or both.

Answer 1

I have been experimenting creating classes in unmanaged memory. It is possible but has a problem I am currently unable to solve – you can’t assign objects to reference-type fields -see edit at the bottom-, so you can have only structure fields in your custom class.
This is evil:

using System;
using System.Reflection.Emit;
using System.Runtime.InteropServices;

public class Voodoo<T> where T : class
{
    static readonly IntPtr tptr;
    static readonly int tsize;
    static readonly byte[] zero;

    public static T NewInUnmanagedMemory()
    {
        IntPtr handle = Marshal.AllocHGlobal(tsize);
        Marshal.Copy(zero, 0, handle, tsize);
        IntPtr ptr = handle+4;
        Marshal.WriteIntPtr(ptr, tptr);
        return GetO(ptr);
    }

    public static void FreeUnmanagedInstance(T obj)
    {
        IntPtr ptr = GetPtr(obj);
        IntPtr handle = ptr-4;
        Marshal.FreeHGlobal(handle);
    }

    delegate T GetO_d(IntPtr ptr);
    static readonly GetO_d GetO;
    delegate IntPtr GetPtr_d(T obj);
    static readonly GetPtr_d GetPtr;
    static Voodoo()
    {
        Type t = typeof(T);
        tptr = t.TypeHandle.Value;
        tsize = Marshal.ReadInt32(tptr, 4);
        zero = new byte[tsize];

        DynamicMethod m = new DynamicMethod("GetO", typeof(T), new[]{typeof(IntPtr)}, typeof(Voodoo<T>), true);
        var il = m.GetILGenerator();
        il.Emit(OpCodes.Ldarg_0);
        il.Emit(OpCodes.Ret);
        GetO = m.CreateDelegate(typeof(GetO_d)) as GetO_d;

        m = new DynamicMethod("GetPtr", typeof(IntPtr), new[]{typeof(T)}, typeof(Voodoo<T>), true);
        il = m.GetILGenerator();
        il.Emit(OpCodes.Ldarg_0);
        il.Emit(OpCodes.Ret);
        GetPtr = m.CreateDelegate(typeof(GetPtr_d)) as GetPtr_d;
    }
}

If you care about memory leak, you should always call FreeUnmanagedInstance when you are done with your class.
If you want more complex solution, you can try this:

using System;
using System.Reflection.Emit;
using System.Runtime.InteropServices;


public class ObjectHandle<T> : IDisposable where T : class
{
    bool freed;
    readonly IntPtr handle;
    readonly T value;
    readonly IntPtr tptr;

    public ObjectHandle() : this(typeof(T))
    {

    }

    public ObjectHandle(Type t)
    {
        tptr = t.TypeHandle.Value;
        int size = Marshal.ReadInt32(tptr, 4);//base instance size
        handle = Marshal.AllocHGlobal(size);
        byte[] zero = new byte[size];
        Marshal.Copy(zero, 0, handle, size);//zero memory
        IntPtr ptr = handle+4;
        Marshal.WriteIntPtr(ptr, tptr);//write type ptr
        value = GetO(ptr);//convert to reference
    }

    public T Value{
        get{
            return value;
        }
    }

    public bool Valid{
        get{
            return Marshal.ReadIntPtr(handle, 4) == tptr;
        }
    }

    public void Dispose()
    {
        if(!freed)
        {
            Marshal.FreeHGlobal(handle);
            freed = true;
            GC.SuppressFinalize(this);
        }
    }

    ~ObjectHandle()
    {
        Dispose();
    }

    delegate T GetO_d(IntPtr ptr);
    static readonly GetO_d GetO;
    static ObjectHandle()
    {
        DynamicMethod m = new DynamicMethod("GetO", typeof(T), new[]{typeof(IntPtr)}, typeof(ObjectHandle<T>), true);
        var il = m.GetILGenerator();
        il.Emit(OpCodes.Ldarg_0);
        il.Emit(OpCodes.Ret);
        GetO = m.CreateDelegate(typeof(GetO_d)) as GetO_d;
    }
}

/*Usage*/
using(var handle = new ObjectHandle<MyClass>())
{
    //do some work
}

I hope it will help you on your path.

Edit: Found a solution to reference-type fields:

class MyClass
{
    private IntPtr a_ptr;
    public object a{
        get{
            return Voodoo<object>.GetO(a_ptr);
        }
        set{
            a_ptr = Voodoo<object>.GetPtr(value);
        }
    }
    public int b;
    public int c;
}

Edit: Even better solution. Just use ObjectContainer<object> instead of object and so on.

public struct ObjectContainer<T> where T : class
{
    private readonly T val;

    public ObjectContainer(T obj)
    {
        val = obj;
    }

    public T Value{
        get{
            return val;
        }
    }

    public static implicit operator T(ObjectContainer<T> @ref)
    {
        return @ref.val;
    }

    public static implicit operator ObjectContainer<T>(T obj)
    {
        return new ObjectContainer<T>(obj);
    }

    public override string ToString()
    {
        return val.ToString();
    }

    public override int GetHashCode()
    {
        return val.GetHashCode();
    }

    public override bool Equals(object obj)
    {
        return val.Equals(obj);
    }
}