Question

I have a large XML file (many MBs) that I cannot afford to download as a whole.

<doc>
   <element>...</element>
   <element>...</element>
   ...
   <element>...</element>
</doc>

I need to read this XML from the web (or better – stream it) and then take the first N elements from its body to have them processed with an XSLT template.

What library/technique do you recommend for such a task? I am writing in Java.

Kindest regards,
Q.

Answer 1

You can use STAX to read the elements and redirect a subset of them to a different stream:

final XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
final XMLOutputFactory xmlOutputFactory = XMLOutputFactory.newInstance();
final XMLEventReader xmlReader = xmlInputFactory.createXMLEventReader(yourInputStream);
final XMLEventWriter xmlWriter = xmlOutputFactory .createXMLEventWriter(yourOutputStream); //The place where the resulting partial XML will go

while (xmlReader.hasNext()) {
    XMLEvent event = xmlReader.nextEvent();

    if (event ... some validation) {
        xmlWriter.add(event); //Forward it to xmlWriter
    }

    if (we have read enough elements) {
        break;
    }
}
xmlReader.close();
xmlWriter.flush();
xmlWriter.close();

Addition:

To write the closing tag, you should create a close tag event and pass it to same XMLEventWriter object whenever you need. For that you need an instance of XMLEventFactory. Here’s example of writing a closing tag:

XMLEventFactory eventFactory = XMLEventFactory.newInstance();
XMLEvent closeTagEvent = eventFactory.createEndElement("prefix", "http://namespace.com", "elementName");
//This will create a closing tag event for a previously opened <prefix:elementName xmlns:prefix="http://namespace.com">

xmlWriter.add(closeTagEvent);

xmlWriter.flush();
xmlWriter.close();

P.S. Sorry for confusion with one of my comments suggesting to use XMLStreamWriter. I had never used XMLEventWriter so did not know how to use it properly until now.