Question

Problem I have now is that the stored procedure called by the jdbcTemplate returns large amount of records, million records, which make our java query method very slow.

My java query method does the paging of the results so it is very slow. How can I improve this without altering the DB stored procedure. That is, to query for specific rows only so the java method would not have the burden of processing million records to do the paging.

public PageModel<Map<String, String>> query(String sql, final int offset, final int limit) throws ReportException {
    try {
        return jdbcTemplate.query(sql, new ResultSetExtractor<PageModel<Map<String, String>>>() {
            @Override
            public PageModel<Map<String, String>> extractData(ResultSet rs)
                    throws SQLException, DataAccessException {
                long startTime = System.nanoTime();
                PageModel<Map<String, String>> pageModel  = new PageModel<Map<String,String>>();
                List<Map<String, String>> list = new ArrayList<Map<String,String>>();
                int rows = 0;
                // skip rows
                for (int i=0; i<offset&&rs.next(); i++) {
                    rows++;
                }
                // get rows
                for (int i=0; i<limit&&rs.next(); i++) {
                    Map<String, String> map = new HashMap<String, String>();
                    ResultSetMetaData metadata = rs.getMetaData();
                    int count = metadata.getColumnCount();
                    for (int j=1; j<=count; j++) {
                        map.put(metadata.getColumnName(j), rs.getString(j));
                    }
                    list.add(map);
                    rows++;
                }
                // iterate remaining rows to get total rows
                while (rs.next())
                    rows++;
                pageModel.setOffset(offset);
                pageModel.setLimit(limit);
                pageModel.setData(list);
                pageModel.setTotal(rows);
                long endTime = System.nanoTime();
                long duration = endTime - startTime;
                System.out.println("Query took: " + duration);
                return pageModel;
            }
        });
    } catch (DataAccessException e) {
        throw new ReportException(e);
    }
}

Answer 1

How can I improve this without altering the DB stored procedure.

There is no any solution in your case, only altering SP