Home/ Questions/Q 8440823
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-10T08:24:09+00:00

I’m using the current ServiceStack with protobuf serialization. When adding an ICacheClient to cache

  • 0

I’m using the current ServiceStack with protobuf serialization.

When adding an ICacheClient to cache my responses, the binary answer sent from the cache client has a different encoding/binary serialization than the original response without any cache client.

This is leading to deserialization problems on the client side, where I’m using a precompiled deserializer.

For example, this protobuf exception occurs at the client side when using an ICacheClient on the ServiceStack server side:

OverflowException: Number overflow.
ProtoBuf.ProtoReader.TryReadUInt32VariantWithoutMoving (Boolean trimNegative, System.UInt32& value)
ProtoBuf.ProtoReader.ReadUInt32Variant (Boolean trimNegative)
ProtoBuf.ProtoReader.ReadUInt32 () MyModelSerializer.Read (My.Models.MyDataModel , ProtoBuf.ProtoReader )

(I believe this is a random exception, pretty sure there can occur other exceptions as well. Take this as an example.)

This behaviour is the same with both MemoryCacheClient and Redis cache client.

This is how I initialize protobuf:

ContentTypeFilters.Register(ContentType.ProtoBuf,
                        (reqCtx, res, stream) => ProtoBuf.Serializer.NonGeneric.Serialize(stream, res),
                         ProtoBuf.Serializer.NonGeneric.Deserialize);

The Cache Client is initialized like this:

container.Register<ICacheClient>(new MemoryCacheClient());

(Or Redis following this guide).

This is how the cached protobuf-serialized data which goes over the wire looks like and differs from the uncached one:
Binary diff
Big Image

The uncached response seems to be encoded/serialized differently.

I there anything I can do about this encoding issue to make Redis or the Memory-ICacheClient compatible with protobuf?

Update: I investigated a bit, these are my findings:

  • To be stored within the cache provider (whether it be Redis os Memory), the protobuf binary stream is “converted” to a string via a StreamReader in HttpResponseFilter->SerializeToString using “UTF8 without BOM”-Encoding, which itself is called via CacheClientExtensions->Cache().

  • First Problem: CacheClientExtensions->Cache() then returns the to-string-serialized DTO, making it yet already impossible for protobuf to deserialize. First Solution: Return the original DTO in CacheClientExtensions->Cache(). But that would only work for the first non-cached response, since the cached response won’t yet be properly de-serialized. This takes us to the

  • Second Problem: Getting data out of the cache again requires a correct to-string-serialization of protobuf’s binary data to put it into the cache in the first place. I know it would be working using via base64.

  • Third Problem: Currently there seems to be no way to replace the current stream-to-string conversion via StreamReader in HttpResponseFilter->SerializeToString at runtime, am I right?

  • Fourth Problem: When getting the data back from the cache, it has to be base64-decoded again.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I was digging through the ServiceStack code, but found no existing possibility to realize my requirement (I really hope I didn’t overlook something).

    So I put some changes to ServiceStack which enable the use of custom TextSerializers which are then used to serialize-to-string bevor pushing the DTOs into the cache clients.

    See my diff here:
    https://github.com/derFunk/ServiceStack/commit/93f62c7d7e44a303c88a81c3096b9757daba4c7c

    It’s working for my purposes, though it may not be fully tested and is yet very protobuf focussed.

    I would appreciate if some ServiceStack ninjas could review what I’m doing, maybe it’ll end up in a pull request.

    • 0