Question

By all means I usually debug my stuff for my own, and I also got an alternativ (bloated but working) solution for that, but I can’t seem to figure out the logical error in my following batch-code:

private int records = 0;
private Query q;

public void BatchProcessor(String className)
        throws Exception {
    int pageSize = 1000;
    boolean done = false;
    List<Test> resultList = null;

    while (!done) {

        q.setFirstResult(records);
        q.setMaxResults(records+pageSize);
        System.out.println("records: "+records);            

        if (records % pageSize == 0) {
            em.clear();
        }

        resultList = q.getResultList();
        process(resultList);
    }
}

private void process(List<Test> list) {
    Iterator<Test> itAmount = list.iterator();
    while (itAmount.hasNext()) {
        Test dto = itAmount.next();
        records=records+1;
    }
}

The output is as follows:

records: 0
records: 1000
records: 3000
records: 7000
records: 15000
records: 31000
…

This is driving me crazy. Each loop in the while() it seems to start from “0” again as the number of q.setFirstResult() and then the lists grows larger. The list grows larger each iteration until an “Out of Memory” Exception occurs. I also don’t get why it doesn’t print out at 20000 records, 40000, 50000 etc.
I completely fail to see the fault in this code 🙁 Please, does anyone have a hint?

Answer 1

You should do not do q.setMaxResults(records+pageSize); You should only do q.setMaxResults(pageSize);

If as your records increases you are increasing MaxResults

Java doc

Query setMaxResults(int maxResult)

Set the maximum number of results to retrieve.

The list is increasing because of your records+pageSize value

records= 0 MaxResults = 1000;
records= 1000 MaxResults = 2000;

It should be always like

records= 0 MaxResults = 1000;
records= 1000 MaxResults = 1000;
records= 2000 MaxResults = 1000;