Question

The following code sample is inside a for loop that runs about 2 million times.

List<String> parameters = new LinkedList<String>();
stmt2 = null;
rs2= null;

//This is line 472
stmt2 = con.prepareStatement("select NAME from TABLE_NAME where FIELD="+ strId);
rs2 = stmt2.executeQuery();

while (rs2.next()) {
    parameters.add(rs2.getString("NAME"));
}

stack trace:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at com.mysql.jdbc.PreparedStatement.<init>(PreparedStatement.java:437)
    at com.mysql.jdbc.Connection.clientPrepareStatement(Connection.java:2185)
    at com.mysql.jdbc.Connection.prepareStatement(Connection.java:4782)
    at com.mysql.jdbc.Connection.prepareStatement(Connection.java:4687)
    at consistencyCheck.ConsistencyCheck.parameterCheck(ConsistencyCheck.java:472)
    at consistencyCheck.ConsistencyCheck.performConsistencyCheck(ConsistencyCheck.java:316)
    at consistencyCheck.ConsistencyCheck.main(ConsistencyCheck.java:198)

Please let me know if more information is required.

Thank you.

Thanks everyone for the answers. I will accept BalusC’s answer since he answered first. Unfortunately I cant upvote any other answers due to not enough reputation 🙁

Just a side note to all who suggested to increase memory heap. Increasing the memory heap is something you should never do unless you are 100 % sure that is the only solution to your problem. For example in my problem increasing the heap might ‘solve’ the problem, but the underlying blunder still remains.

Answer 1

Based on the comments, you seem to be creating the Statement and ResultSet inside the loop but never closing them. You need to close them in the loop as well. This will free up internal resources.

Also, you are not really taking benefit of the DB cache of the prepared statement. Right now you are string-concatenating the parameter in the SQL string which causes 2M String objects being created instead of 1 String object. Better prepare the statement before the loop.

try {
    // ...
    statement = connection.prepareStatement("select NAME from TABLE_NAME where FIELD=?");

    for ( /* 2M times? */ ) {
        statement.setInt(1, id);

        try {
            resultSet = statement.executeQuery();
            // ...
        } finally {
            if (resultSet != null) try { resultSet.close(); } catch (SQLException ignore) {}
        }
    }
} finally {
    if (statement != null) try { statement.close(); } catch (SQLException ignore) {}
}

Alternatively, you can also consider to use an IN clause instead. E.g.

WHERE field IN (1,2,3,4,5);

This is however trickier with placeholders. See also: What is the best approach using JDBC for parameterizing an IN clause?

Or as a completely different alternative, if necessary with help of a more experienced DB admin / SQL ninja, rewrite the entire thing so that you get exactly the results you need with only one SQL query. Ask if necessary a separate question about that on sql here on SO.