Question

Hey, im trying to wirte about 600000 Tokens into my MySQL Database Table. The Engine I’m using is InnoDB. The update process is taking forever :(. So my best guess is that I’m totally missing something in my code and that what I’m doing is just plain stupid.

Perhaps someone has a spontaneous idea about what seems to eat my performance:

Here is my code:

public void writeTokens(Collection<Token> tokens){

    try{
        PreparedStatement updateToken = dbConnection.prepareStatement("UPDATE tokens SET `idTag`=?, `Value`=?, `Count`=?, `Frequency`=? WHERE `idToken`=?;");

        for (Token token : tokens) {

            updateToken.setInt(1, 0);
            updateToken.setString(2, token.getWord());
            updateToken.setInt(3, token.getCount());
            updateToken.setInt(4, token.getFrequency());
            updateToken.setInt(5, token.getNounID());

            updateToken.executeUpdate();
        }
    }catch (SQLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

Thanks a lot!

Answer 1

I don’t have a Java-specific answer for you, but wrap the whole shebang in a transaction. If you don’t, then MySQL (when writing against InnoDB) starts and commits a new transaction per update statement.

Just execute START TRANSACTION before you start your calls, and execute COMMIT after all your updates/inserts are done. I also think that MySQL defers index updates until the end of the transaction, as well, which should help improve performance considerably if you’re updating indexed fields.