Question

I need to fetch 2 million records from the oracle database(1 year data).
When I am retrieving it as list, it is taking ‘n’ minutes and it hangs.

sqlMap.queryForList("getResultSet", parameterMap);

So, I tried implementing IBatis "RowHandler" interface and I overrided the "handleRow(Object obj)" and I am able to get the result (One row at a time).

But I need to get ‘n’ rows at a time where n >= 1000. So I added fetchSize="1000" and resultSetType="FORWARD_ONLY" attribute to my select statement.

E.g:

<select id='getResultSet' parameterClass='java.util.Map' fetchSize="1000" resultSetType="FORWARD_ONLY">  

But still I am getting only one row at a time in the "handleRow(Object obj)" method.

@Override
public void handleRow(Object queryResult) {     

        if(queryResult != null) {

            try {

                tempResultMap = (ClassName) queryResult;
                resultList.add(tempResultMap);
                System.out.println("List Size -> " + reportList.size());
            } catch (Exception e) {
                e.printStackTrace();
            }       
        }   
    }  

When the method is called during query execution, "List Size ->" is always incrementing by one. But I am expecting increment rate of 1000 (As I have given fetchSize = “1000”)…

When I googled out, there is a property available(Driver.useCursorFetch) which can be used along with "fetchSize" and "resultSetType".
Reference : http://www.yaldex.com/mysql_manual/ch23s04.html or Ibatis queryWithRowHandler() still seems to fetch all rows.
But I think it is only for MySQL Database.
What is the equivalent property(Driver.useCursorFetch) for Oracle 11g database.
I need some configuration like below.

<bean id="sourceName" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />                         
    <property name="url" value="jdbc:oracle:thin:@host:port:sid" />
    <property name="username" value="$uname" />
    <property name="password" value="$pwd" />
    <!-- 
    Some thing like this 
    <property name="configName(Instead of Driver.useCursorFetch)" value="true/false" />
    -->

</bean>

Thanks in advance.

Answer 1

handleRow only gives you a single row. Using fetchSize, you should be able to delete your handleRow method, if I understand your scenario correctly.

If you really do need to handle each rown individually, you can make a stateful (non-singleton) RowHandler with a private list and implements the Observer pattern, notifying listeners each time you’ve hit 1000 records.