Question

I am developing an application in java for processing data and uploading it to a MySQL database. With this application, I iterate through all of the files that need processed in a directory. With each row of the file, I store the data from that row into a variety of variables and do some transformations to get all of the data ready for uploading to my database. Then I plan to upload that row’s data to the database with a PreparedStatement and INSERT query.

I am unfamiliar with the best practices for my situation. I’m in the process of teaching this to myself so I’m looking for a little guidance to make sure I do it correctly the first time.

Would it be inappropriate simply to open a connection at the beginning of the application’s runtime (ie. open it once)? For each insert query I would create and close the PreparedStatement, but I would simply leave the connection open until I finished processing all of the files.

I’ve read about connection pools elsewhere but I can’t figure out if it is applicable in my situation. I know its expensive to open and close connections frequently, but shouldn’t I be able to just open it once and run all of my queries under that connection?

If you would recommend using a connection pool, what service would you recommend for my situation. Preferably it will be easy to get a handle on quickly. My project is somewhat time sensitive. Thanks.

Answer 1

If you are developing a desktop Java application, you won’t be using connection pools. Connection pools are managed by Java Application Servers(Tomcat, JBoss, Glassfish) and will not be available in a desktop Java application.

Keeping a connection open is plausible if you are going to be doing just one update.

Also, a batch update would be a great idea for you:

String connectionURL = "jdbc:mysql://localhost:3306/database";
Connection con = null;
PreparedStatement stmt = null;

try {
    Class.forName("com.mysql.jdbc.Driver");
    con = DriverManager.getConnection(connectionURL);
    stmt = con.prepareStatement(statement);

    for (Object o : list) {    // this is a list of your Java entity class
        stmt.setString(1, "foo");    // this is to update the parameters in the PreparedStatement
        stmt.setString(2, "bar");

         stmt.addBatch();     //  this adds the PreparedStatement to the batch of statements to execute
    }

    stmt.executeBatch();

} catch (SQLException e) {
    e.printStackTrace();
} catch (Exception e) {
    e.printStackTrace();
} finally {
    if (stmt != null) {
        stmt.close();
    }
    if (con != null) {
        con.close();
    }
}

The idea of a batch update is to have one prepared statement, where you can simply change the parameters, and add the “new” statement to a batch of statements. Then, you can execute them all at one go when you invoke stmt.executeBatch()

I would advise you however, to execute statements probably 30 at a time if you have many such statements. Otherwise, you would be stranded if the program crashes/fails.