Question

I have a Java program that is generating XML data which happens to be very large. I am using the typical parsers in javax.xml and the org.w3c.dom objects. The DOM in memory is becoming so large that it occasionally runs out of memory. Now, the reason it reaches the memory limit is because the program is generating all of the data at once and then transforming it to a String and then to a file. There are hundreds of elements being generated at once that could be made one at a time and buffered to a file rather than slowly filling the RAM. I’ve been looking into libraries out there but I’m a bit confused on what is good. What method should I be using to buffer this DOM document to a file rather than generating all at once?

Answer 1

Yes Marty, I also suggest you to use the SAX. It consumes less memory.
Here’s a simple sample code for SAX implementation.

public class SaxReader{
    public void readXML() throws Exception{
        ResultHandler oHandler                      = new ResultHandler();
        CreateParser parser                         = new CreateParser(oHandler);    
        String responseXML = "Place your XML as String HERE";
        InputSource input                                   = new InputSource(new StringReader(responseXml)); 
        input.setEncoding("UTF-8");     
        parser.parse(input);

    }
}


class ResultHandler extends DefaultHandler {
    public void startDocument (){
        System.out.println("Start document");
    }

    public void endDocument (){
        System.out.println("End document");
    }

    public void startElement(String namespaceURI, String localName, String qName, Attributes attribs)throws SAXException {
        strValue = "";
        if(null!=qName && qName.equals("ID")){
            System.out.println("Starting element <ID> : "+strValue);
        }
    }

    public void characters(char[] ch, int start, int length)throws SAXException {
        strValue += new String(ch, start, length);
        System.out.println(">>>" + strValue);//Reading the values.
    }

    public void endElement(String namespaceURI, String localName, String qName) throws SAXException {
        if(null!=qName && qName.equals("ID")){
            System.out.println("Ending element </ID> : "+strValue);
        }
    }
}