Question

I’m working on an application that record the state of a compplex system.
I’ve to record these states to xml files (using xml serialisation).
This file is used to replay the recorded event and analyse it.

the context:

My application display a set of complex data (we called this a VIEW)
each view contains 1 object and 2 collections of objects (400 items each with about 20 properties).

The application record a state of the data at fixed interval (the time between 2 record may vary from 5 sec to 60 min), all records are stored in a VIEWS collection and we save this collection of VIEW to a trace file (using xmlserializer and serialise my object collection to a text writer)

All works well but when the file reach about 20 Mo, the file save can take more time than the time between 2 record event (file are stored on the network and sometime we have lags) and we wants to optimize this.

So my question is :

Is there a way to append an object to a file without have to load and write the entire file or a way to optimize the serialisation to file?

We have to use xml serialisation because the file must be readable by human & by code (deserialisation).

Answer 1

How about using a database? Those things are made to solve problems like these 🙂 If you want to do it manually, you will have to declare your own protocol. A primitive working implementation:

[Serializable]
public class Test
{
    public string Name { get; set; }
}

private static string Serialize(Test instance)
{
    XmlSerializer serializer = new XmlSerializer(typeof(Test));
    using (MemoryStream stream = new MemoryStream())
    {
        serializer.Serialize(stream, instance);
        return Convert.ToBase64String(stream.ToArray());
    }
}
private static Test Deserialize(string instance)
{
    XmlSerializer serializer = new XmlSerializer(typeof(Test));
    byte[] instanceBytes = Convert.FromBase64String(instance);

    using (MemoryStream stream = new MemoryStream(instanceBytes))
    {
        return serializer.Deserialize(stream) as Test;
    }
}



public static void Main(string[] args)
{
    using (MemoryStream stream = new MemoryStream())
    {
        // write some test objects
        StreamWriter writer = new StreamWriter(stream);
        writer.WriteLine(Serialize(new Test() { Name = "test1" }));
        writer.WriteLine(Serialize(new Test() { Name = "test2" }));
        writer.WriteLine(Serialize(new Test() { Name = "test3" }));
        writer.Flush();

        // reset
        stream.Position = 0;

        // read them back again
        StreamReader reader = new StreamReader(stream);
        Test test1 = Deserialize(reader.ReadLine());
        Test test2 = Deserialize(reader.ReadLine());
        Test test3 = Deserialize(reader.ReadLine());
    }
}