Question

I’m trying to see if there is a way I can do away with Reading in my Stream to a MemoryStream before deserialising the Object that has been stored via ProtoBuf.Net.

I can’t use the Protobuf.Net With LengthPrefix features because I am coming back and adjusting the records in the File as new records are written so that the file can be traversed backward or forwards. The Serialized Class requires no changes because I keep the 3 length separate to actual class.

This is the code I’m currently using to read an entry in:

Private Function ReadEntry(ByVal br As BinaryReader) As PacketErrorLogEntry
    Dim activeRecord As PacketErrorLogEntry

    Dim OffsetPrevious As UInt32 = br.ReadUInt32()
    Dim RecordLength As UInt32 = br.ReadUInt32
    Dim OffsetNext As UInt32 = br.ReadUInt32
    Using ms As New MemoryStream
        Dim readLength As UInt32
        Dim bytesRead As UInt32
        Dim writeBuffer As Byte() = CType(Array.CreateInstance(GetType(Byte), _ 
                                          4096), Byte())
        Dim bytesToRead As UInt32 = CType(writeBuffer.Length, UInt32)
        If bytesToRead > RecordLength Then
            bytesToRead = RecordLength
        End If
        bytesRead = 0

        While readLength < RecordLength
            bytesRead = CType(br.BaseStream.Read(writeBuffer, 0, _ 
                              CType(bytesToRead, Integer)), UInt32)
            ms.Write(writeBuffer, 0, CType(bytesRead, Integer))
            readLength += bytesRead
        End While

        ms.Flush()
        ms.Position = 0
        activeRecord = Serializer.Deserialize(Of PacketErrorLogEntry)(ms)
        activeRecord.PreviousRecordLocation = OffsetPrevious
        activeRecord.NextRecordLocation = OffsetNext
    End Using
    'activeRecord = Serializer.Deserialize(Of PacketErrorLogEntry)(br.BaseStream, RecordLength)
    'activeRecord.PreviousRecordLocation = OffsetPrevious
    'activeRecord.NextRecordLocation = OffsetNext


    Return activeRecord
End Function

What I was hoping I might be able to achieve is that by passing the Length to Read to the Deserialize function then I’d be able to do away with the whole MemoryStream block and just get my object back.

I use the BinaryReader/Writer for the Length/Offsets so that I can come back later and just overwrite those positions with updated values.

Answer 1

If you are using v2, then this is available on the TypeModel API (which is actually the primary API; the Serializer.Deserialize<T> API just calls into RuntimeTypeModel.Default.Deserialize). There are overloads that accept the number of bytes to consume. Once such method (on a TypeModel instance, most likely RuntimeTypeModel.Default) is:

/// <summary>
/// Applies a protocol-buffer stream to an existing instance (which may be null).
/// </summary>
/// <param name="type">The type (including inheritance) to consider.</param>
/// <param name="value">The existing instance to be modified (can be null).</param>
/// <param name="source">The binary stream to apply to the instance (cannot be null).</param>
/// <param name="length">The number of bytes to consume.</param>
/// <returns>The updated instance; this may be different to the instance argument if
/// either the original instance was null, or the stream defines a known sub-type of the
/// original instance.</returns>
public object Deserialize(Stream source, object value, Type type, int length);

I should also note that the “with length prefix” API also allow different prefix-styles, with fixed-length (either big-endian or little-endian) int32 available. But your current approach should work fine too.

If you are using v1, you may have to manufacture a length-limited stream…. or just borrow the one that v1 uses internally (note that v2 doesn’t use this approach – it tracks the remaining active bytes on a single stream plus buffer).