Home/ Questions/Q 381031
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-12T15:02:45+00:00

I need advice. I have application that imports 10,000 rows containing name & address

  • 0

I need advice. I have application that imports 10,000 rows containing name & address from a text file into XElements that are subsequently added to a synchronized queue. When the import is complete the app spawns worker threads that process the XElements by deenqueuing them, making a database call, inserting the database output into the request document and inserting the processed document into an output queue. When all requests have been processed the output queue is written to disk as an XML doc.

I used XElements for the requests because I needed the flexibility to add fields to the request during processing. i.e. Depending on the job type the app might require that it add phone number, date of birth or email address to a request based on a name/address match against a public record database.

My questions is; The XElements seems to use quite a bit of memory and I know there is a lot of parsing as the document makes its way through the processing methods. I’m considering replacing the XElements with a Dictionary object but I’m skeptical the gain will be worth the effort. In essence it will accomplish the same thing.

Thoughts?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    So you’re not actually using any XML as such? You’re just using XElement as a collection of name/value pairs? If so, I’d definitely use a dictionary. I would expect your code to potentially come out cleaner as well.

    On the other hand if you’re genuinely using XML, you probably want to stick with XElement.

    Do you actually have a problem? You say it’s using up quite a bit of memory – do you have enough memory? Could you buy more memory? That would almost certainly be cheaper than spending even a few hours refactoring, if it’s just for the sake of saving memory. (It sounds like this app is only run on one box – I could be wrong. The more widely deployed it is, the more it probably makes sense to spend some time optimising it.)

    EDIT: Okay, so buying more memory isn’t really viable. Even so, do you actually have a problem? What’s the impact of this perhaps using more memory than it needs? What’s it really costing you?

    • 0