Question

I have serialized a list of objects with protobuf-net.

Theoretically, the .bin file can contain millions of objects.

Let’s assume the objects are of a class containing the following:

public string EventName;

I have to take a query and create a list containing the objects matching the query.
What is the correct way to extract the matching objects from the serialized file using LINQ?

Answer 1

The protobuf format is a linear sequence of items; any indexing etc you way can only be applies separately. However, IEnumerable<T> is available; you might find that:

var item = Serializer.DeserializeItems<YourType>(source)
       .First(item => item.Id == id);

does the job nicely; this:

• is lazily spooled; each item is yielded individually, so you don’t need a glut of memory
• is short-circuited; if the item is found near the start, it’ll exit promptly

Or for multiple items:

var list = Serializer.DeserializeItems<YourType>(source)
    .Where(item => item.Foo == foo);

(add a ToList to te end of the above if you want to buffer the matching items in memory, or use without a ToList if you just want to parse it once in a forwards-only way)