Home/ Questions/Q 7410573
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-29T06:15:16+00:00

My project has a reporting module that gathers data from the database in the

  • 0

My project has a reporting module that gathers data from the database in the form of XML and runs an XSLT on it to generate the user’s desired format of report. Options at this point are HTML and CSV.

We use Java and Xalan to do all interaction with the data.

The bad part is that one of these reports that the user can request is 143MB (about 430,000 records) for just the XML portion. When this is transformed into HTML, I run out of heap space with a maximum of 4096G reserved for heap. This is unacceptable.

It seems that the problem is simply too much data, but I can’t help but think there is a better way to deal with this than limiting the customer and not being able to meet functional requirements.

I am glad to give more information as needed, but I cannot disclose too much about the project as I’m sure most of you understand. Also, the answer is yes; I need all of the data at the same time: I cannot paginate it.

Thanks

EDIT

All the transformation classes I am using are in the javax.xml.transform package. The implementation looks like this:

final Transformer transformer = 
  TransformerFactory.newInstance().newTransformer(
    new StreamSource(new StringReader(xsl)));
final StringWriter outWriter = new StringWriter();
transformer.transform(
  new StreamSource(new StringReader(xml)), new StreamResult(outWriter));
return outWriter.toString();

If possible, I would like to leave the XSLT the way it is. The StreamSource method of doing things should allow me to GC some of the data as it is processed, but I’m not sure what limitations on XSLT (functions, etc) this might require for it to do proper cleanup. If someone could point me at a resource detailing those limitations, it would be very helpful.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    We are able to improve this by doing two things.

    1. We take the XML source and destination format and make them files in temp. This keeps the initial creation and storage out of RAM, since the data is coming from a database and being written back to the DB as well. A handle to the data is all that’s necessary.

    2. Use the Saxonica transformer. This allows for a couple things including SAX-style transformations and the use of XSLT 2.0, which the Xalan parser does not.

    • 0