Question

Heyup. Long time lover of protobuf.net.

Quick question though. I have a highly multithreaded C# application which is deserialising perhaps 100 objects / second, amounting to about 50MB/sec. I am seeing very large memory usage, well over and above that which I am deserialising. I’ve run the application through the ‘Red Gate ANTS Memory Profiler’ and it’s showing me a massive amount of Generation 2 memory objects due to protobuf (Over 50% of application usage). Most objects are int values and are linked with:

- TypeModel.TryDeserializeList()
- ProtoBuf.Meta.BasicList

Any help reducing this gen 2 memory usage would be appreciated.

Marc

Answer 1

It sounds to me that the root T here is the array itself, i.e.

int[] values = Serializer.Deserialize<int[]>(source);

If that is the case, then currently it uses a slightly sub-optimal path for that scenario (for the reason of: using the same code-path even on platforms that have weak meta-programming/reflection models, such as iOS). I will try to spend a few hours tidying that at some point, but in answer to your question – you should be able to avoid the issue here simply by adding a parent object:

[ProtoContract]
public class MyDataWrapper { // need a new name...
    [ProtoMember(1)]
    public int[] Values { get;set; }
}

and then:

int[] values = Serializer.Deserialize<MyDataWrapper>(source).Values;

This is actually fully compatible with data already serialized via Serialize<int[]>, as long as the field-number used is 1. One additional benefit of this approach is that if desired you could use the “packed” sub-format (only available for lists/arrays of primitives such as int); although maybe that still isn’t a great idea in this case due to the large length (it may require buffering when serialising).

---

Additional context; “v1” here basically uses MakeGenericType to switch into to something like the above on the fly; however since this approach is not available in many of the additional platforms that “v2” targets, it uses a less elegant approach here. But now tht it is pretty stable there, I could re-add the optimised version when running on full .NET 2.0 or above.