Question

here is my problem : at some point in my Java program, I get a (very) big List of Events from a database using the SimpleJdbcTemplate class from Spring.

List<Event> events = 
            this.simpleJdbcTemplate.query(myQuery,
            myMapper(), 
            new Object[] {
                    filter.getFirst(),
                    filter.getSecond(),
                    filter.getThird()}
            );

The problem is that the list may contain something like 600,000 Events … Therefore using a lot of memory (and also taking time to be processed).

However I don’t really need to retrieve all the Events at once. Actually I would like to be able to iterate over the list, read only a few events (linked to a specific KEY_ID – the sql query myQuery is ordered by KEY_ID), process them and finally get back iterating, letting the garbage collector get rid of the previous and already processed Events so that I never exceed a certain amount of memory.

Is there a nice way to do so using the Spring library (or any library)?

Cheers,
Vakimshaar.

Answer 1

If I understand correctly, you would like to iterate over the result set, but are not interested in building the full list of results.

Just use the query method with a ResultSetExtractor as argument. The ResultSetExtractor can use your mapper to transform the current row into an Event. Put every event into a list until you reach a different KEY_ID or the end of the result set, then proceed with your list of events and clear the list.