Question

I have a Java program that reads some text from a file, line by line, and writes new text to an output file. But not all the text I write to my BufferedWriter appears in the output file after the program has finished. Why is that?

The details: the program takes a CSV text document and converts it into SQL commands to insert the data into a table. The text file has more than 10000 lines which look similar to following:

2007,10,9,1,1,1006134,19423882

The program seems to work fine except it just stops in the file randomly half way through creating a new SQL statement having printed it into the SQL file. It looks something like:

insert into nyccrash values (2007, 1, 2, 1, 4, 1033092, 259916);
insert into nyccrash values (2007, 1, 1, 1, 1, 1020246, 197687);
insert into nyccrash values (2007, 10, 9, 1

This happens after about 10000 lines but several hundred lines before the end of the file. Where the break happens is between a 1 and a ,. However, the characters doesn’t seem important because if I change the 1 to a 42 the last thing written to the new file is 4, which is cutting off the 2 from that integer. So it seems like the reader or writer must just be dying after writing/reading a certain amount.

My Java code is as follows:

import java.io.*;

public class InsertCrashData
{
    public static void main (String args[])
    {
        try
        {   
            //Open the input file.
            FileReader istream = new FileReader("nyccrash.txt");
            BufferedReader in = new BufferedReader(istream);
            //Open the output file.
            FileWriter ostream = new FileWriter("nyccrash.sql");
            BufferedWriter out = new BufferedWriter(ostream);
            String line, sqlstr;

            sqlstr = "CREATE TABLE nyccrash (crash_year integer, accident_type integer, collision_type integer, weather_condition integer, light_condition integer, x_coordinate integer, y_coordinate integer);\n\n"; 
            out.write(sqlstr);

            while((line = in.readLine())!= null)
            {
                String[] esa = line.split(",");
                sqlstr = "insert into nyccrash values ("+esa[0]+", "+esa[1]+", "+esa[2]+", "+esa[3]+", "+esa[4]+", "+esa[5]+", "+esa[6]+");\n";
                out.write(sqlstr);
            }
        }
        catch(Exception e)
        {
            System.out.println(e);
        }
    }
}

Answer 1

You need to close your OutputStream which will flush the remainder of your data:

out.close();

The default buffer size for BufferedWriter is 8192 characters, large enough to easily hold hundreds of lines of unwritten data.