Question

We serialize/deserialize XML using XStream… and just got an OutOfMemory exception.

Firstly I don’t understand why we’re getting the error as we have 500MB allocated to the server.

Question is – what changes should we make to stay out of trouble? We want to ensure this implementation scales.

Currently we have ~60K objects, each ~50 bytes. We load the 60K POJO’s in memory, and serialize them to a String which we send to a web service using HttpClient. When receiving, we get the entire String, then convert to POJO’s. The XML/object hierarchy is like:

<root>
    <meta>
       <date>10/10/2009</date>
       <type>abc</type>
    </meta>

    <data>
        <field>x</field>
    </data>

    [thousands of <data>]
</root>

I gather the best approach is to not store the POJO’s in memory and not write the contents to a single String. Instead we should write the individual <data> POJO’s to a stream. XStream supports this but seems like the <meta> element wouldn’t be supported. Data would need to be in form:

<root> 
    <data>
        <field>x</field>
    </data>

    [thousands of <data>]
</root>

So what approach is easiest to stream the entire tree?

Answer 1

You definitely want to avoid serializing your POJOs into a humongous String and then writing that String out. Use the XStream APIs to serialize the POJOs directly to your OutputStream. I ran into the same situation earlier this year when I found that I was generating 200-300Mb XML documents and getting OutOfMemoryErrors. It was very easy to make the switch.

And ditto of course for the reading side. Don’t read the XML into a String and ask XStream to deserialize from that String: deserialize directly from the InputStream.

You mention a second issue regarding not being able to serialize the <meta> element and the <data> elements. I don’t think this is an XStream problem or limitation as I routinely serialize much more complex structures on the order of:

<myobject>
    <item>foo</item>
    <anotheritem>foo</anotheritem>
    <alist>
        <alistitem>
            <value1>v1</value1>
            <value2>v2</value2>
            <value3>v3</value3>
            ...
        </alistitem>
        ...
        <alistitem>
            <value1>v1</value1>
            <value2>v2</value2>
            <value3>v3</value3>
            ...
        </alistitem>
    </alist>
    <anotherlist>
        <anotherlistitem>
            <valA>A</valA>
            <valB>B</valB>
            <valC>C</valC>
            ...
        </anotherlistitem>
        ...
    </anotherlist>
</myobject>

I’ve successfully serialized and deserialized nested lists too.