Question

I am writing a program for formatting 100s of MB String data (nearing a gig) into xml == And I am required to return it as a response to an HTTP (GET) request .

I am using a StringWriter/XmlWriter to build an XML of the records in a loop and returning the

using (StringWriter writer = new StringWriter())
using (writer = XmlWriter.Create(writer, settings)) //where settings are the xml props

writer.ToString() 

during testing I saw a few –out of memory exceptions– and quite clueless on how to find a solution? do you guys have any suggestions for a memory optimized delivery of the response?

is there a memory efficient way of encoding the data? or maybe chunking the data —
I just can not think of how to return it without building the whole thing into one HUGE string object

thanks

—
a few clarifications —
this is an asp .net webservices app over a gigabit ethernet link as josh noted. I am not very familiar with it so still a bit of a learning curve.

I am using XMLWriter to create the XML and create a string out of it using String

some stats —
response xml size = about 385 megs (my data size will grow very quickly to way more than this)

string object size as calculated by a memory profiler = peaked at 605MB

and thanks to everyone who responded…

Answer 1

Can’t you just stream the response to the client? XmlWriter doesn’t require its underlying stream to be buffered in memory. If it’s ASP.NET you can use the Response.OutputStream or if it’s WCF, you can use response streaming.