Question

I am trying to retrieve the data from the table and convert each row into CSV format like

s12, james, 24, 1232, Salaried

The below code does the job, but takes a long time, with tables of rows exceeding 1,00,000.

Please advise on optimizing technique:

 while(rset1.next()!=false) {
                         sr=sr+"\n";
                        for(int j=1;j<=rsMetaData.getColumnCount();j++)
                        {
                            if(j< 5)
                            {
                         sr=sr+rset1.getString(j).toString()+",";
                            }
                            else
                          sr=sr+rset1.getString(j).toString();
                        }

                       }

/SR

Answer 1

Two approaches, in order of preference:

Stream the output

PrintWriter csvOut = ... // Construct a write from an outputstream, say to a file
while (rs.next())
    csvOut.println(...) // Write a single line

(note that you should ensure that your Writer / OutputStream is buffered, although many are by default)

Use a StringBuilder

StringBuilder sb = new StringBuilder();
while (rs.next())
    sb.append(...) // Write a single line

The idea here is that appending Strings in a loop is a bad idea. Imagine that you have a string. In Java, Strings are immutable. That means that to append to a string you have to copy the entire string and then write more to the end. Since you are appending things a little bit at a time, you will have many many copies of the string which aren’t really useful.

If you’re writing to a File, it’s most efficient just to write directly out with a stream or a Writer. Otherwise you can use the StringBuilder which is tuned to be much more efficient for appending many small strings together.