Question

public void GrabData() throws IOException
{
    try {
        BufferedReader br = new BufferedReader(new FileReader("data/500.txt"));
        String line = "";
        int lineCounter = 0;
        int TokenCounter = 1;
        arrayList = new ArrayList < String > ();

        while ((line = br.readLine()) != null) {

            //lineCounter++;
            StringTokenizer tk = new StringTokenizer(line, ",");

            System.out.println(line);

            while (tk.hasMoreTokens()) {
                arrayList.add(tk.nextToken());
                System.out.println("check");
                TokenCounter++;

                if (TokenCounter > 12) {
                    er = new DataRecord(arrayList);
                    DR.add(er);
                    arrayList.clear();
                    System.out.println("check2");

                    TokenCounter = 1;
                }

            }
        }
    } catch (FileNotFoundException ex) {
        Logger.getLogger(Driver.class.getName()).log(Level.SEVERE, null, ex);
    }
}

Hello , I am using a tokenizer to read the contents of a line and store it into an araylist. Here the GrabData class does that job.

The only problem is that the company name ( which is the third column in every line ) is in quotes and has a comma in it. I have included one line for your example. The tokenizer depends on the comma to separate the line into different tokens. But the company name throws it off i guess. If it weren’t for the comma in the company column , everything goes as normal.

Example:-
Essie,Vaill,”Litronic , Industries”,14225 Hancock Dr,Anchorage,Anchorage,AK,99515,907-345-0962,907-345-1215,[e-mail],http://www.essievaill.com

Any ideas?

Answer 1

While StringTokenizer might not natively handle this for you, a couple lines of code will do it… probably not the most efficient, but should get the idea across…

while(tk.hasMoreTokens()) {
    String token = tk.nextToken();

    /* If the item is encapsulated in quotes, loop through all tokens to 
     * find closing quote 
     */
    if( token.startsWIth("\"") ){
        while( tk.hasMoreTokens() && ! tk.endsWith("\"") ) {
            // append our token with the next one.  Don't forget to retain commas!
            token += "," + tk.nextToken();
        }

        if( !token.endsWith("\"") ) {
            // open quote found but no close quote.  Error out.
            throw new BadFormatException("Incomplete string:" + token);
        }

        // remove leading and trailing quotes
        token = token.subString(1, token.length()-1);
    }
}