Question

There appears to be a memory leak when using the standard Java library (1.6.0_27) for evaluating XPath expressions.

See below for some code to reproduct this problem:

public class XpathTest {

    public static void main(String[] args) throws Exception {
        DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
        docFactory.setNamespaceAware(true);
        DocumentBuilder builder = docFactory.newDocumentBuilder();
        Document doc = builder.parse("test.xml");

        XPathFactory factory = XPathFactory.newInstance();
        XPath xpath = factory.newXPath();
        XPathExpression expr = xpath.compile("//Product");

        Object result = expr.evaluate(doc, XPathConstants.NODESET);
        NodeList nodes = (NodeList) result;
        for (int i = 0; i < nodes.getLength(); i++) {
            Node node = nodes.item(i);
            System.out.println(node.getAttributes().getNamedItem("id"));

            XPathExpression testExpr = xpath.compile("Test");
            Object testResult = testExpr.evaluate(node, XPathConstants.NODE);
            Node test = (Node) testResult;
            System.out.println(test.getTextContent());
        }
        System.out.println(nodes.getLength());
    }
}

An example XML file is given below:

<Products>
  <Product id='ID0'>
    <Test>0</Test>
  </Product>
  <Product id='ID1'>
    <Test>1</Test>
  </Product>
  <Product id='ID2'>
    <Test>2</Test>
  </Product>
  <Product id='ID3'>
    <Test>3</Test>
  </Product>
  ...
</Products>

When I run this example using the NetBeans profiler it appears that the allocations for the com.sun.org.apache.xpath.internal.objects.XObject class keeps increasing, even after garbage collection.

Am I using the XPath library in an incorrect way? Is this a bug in the Java libraries? Are there are potential workarounds?

Answer 1

There is no “memory leak” in this case. Memory leak are defined as instances where an application cannot reclaim memory. In this case there is no leak, as all XObject (and XObject[]) instances can be reclaimed at some point in time.

A memory profiler snapshot obtained from VisualVM yields the following observations:

• All XObject (and XObject[]) instances are created when the XPathExpression.evaluate method is invoked.
• XObject instances are reclaimed when they are no longer reachable from a GC root. In your case, the GC roots are the result and testResult local variables which are local to the stack of the main thread.

Based on the above, I suppose that your application is experiencing or likely to experience a memory exhaustion as opposed to a memory leak. This is true when you have a large number of XObject/XObject[] instances from an XPath expression evaluation, that haven’t been reclaimed by the garbage collector because

• they are either still reachable from a GC root,
• or the garbage collector hasn’t come around to reclaiming them yet.

The only solution to the first is to retain objects around in memory for the duration that they are required. You do not seem to be violating that in your code, but your code could certainly be made more efficient – you are retaining the result of the first XPath expression, to be used by the second, when certainly it could be performed more efficiently. //Product/Test can be used to retrieve the Test nodes, and also obtain the parent Product Nodes’ id values are shown in the following snippet (which evaluates only one XPath expression instead of two):

expr = xpath.compile("//Product/Test");
nodes = (NodeList) expr.evaluate(doc, XPathConstants.NODESET);
for (int i = 0; i < nodes.getLength(); i++)
{
    Node node = nodes.item(i);
    System.out.println(node.getParentNode().getAttributes().getNamedItem("id"));
    System.out.println(node.getTextContent());
}
System.out.println(nodes.getLength());

As far as the second observation is concerned, you ought to obtain GC logs (using the verbose:gc JVM startup flag). You could then decide to resize the young generation, if you have too many shortlived objects being created, as there is the possible likelihood that reachable objects will be moved to the tenured generation resulting in the likelihood that a major collection will be required to reclaim objects that are actually shortlived by nature. In an ideal scenario (considering your posted code), a young gen collection cycle should be done every few iterations of the for loop, as the XObject instances that are local to the loop, should be reclaimed as soon as the block’s local variables go out of scope.