Question

I am trying to better understand when I should and should not use Iterators. To me, whenever I have a potentially large amount of data to iterate through, I write an Iterator for it. If it also lends itself to the Iterator interface, then it seems like a win.

I was reading a little bit that there is a lot of overhead with using an Iterator.

A good example of where I used an Iterator was to iterate through a bunch of SQL scripts to execute one query at a time, reading it in, then executing it.

Is there another performance trade off I should be aware of? Before I used iterators, I would read the entire String of SQL commands to execute into an ArrayList, and the iterate through that. If the import is rather large (like for geolocation data, then the server tends to get bogged down).

Walter

Answer 1

I think your question is when you should ‘stream’ input rather than load it all into memory and the process it. It’s not really a question of using Iterator or not I think.

“It depends,” of course, though in your given example it sounds like streaming the input rather than loading it all into memory is a clear win, so iterate indeed.

The benefit of loading into memory is usually that the code is simpler, and maybe you get some benefit from loading large chunks into memory at once rather than reading bits at a time. The benefit of “streaming” is that you limit your memory requirements, and, gain performance associated with that.

As a very crude rule of thumb, I wouldn’t load anything like this into memory unless I were sure it was under 100K or so.