Question

I’ve a big txt file (1.5 Mb) as private resource in my Android Application.
The file is structured like this:

"A1|A2|A3|A4#B1|B2|B3|B4#C1|C2|C3|C4#..."

where A1, A2, A3, B1… are alphanumerical strings. I need to create an object for each group of strings, something like this:

MyObject objA = new MyObject("A1", "A2", "A3", "A4");
MyObject objB = new MyObject("B1", "B2", "B3", "B4");
...

I’ve developed a reader class in order to read this file… but it looks a bit slow.
Here it is:

public class TextFileReader {

    private static Charset charset = Charset.defaultCharset();
    private InputStream stream;
    private InputStreamReader reader;
    private StringBuffer buffer;

    public TextFileReader(Context c, String s) {
        try {
            this.stream = c.openFileInput(s);
            this.reader = new InputStreamReader(this.stream, charset);
            this.buffer = new StringBuffer();
        } catch (FileNotFoundException e) {
        }
    }

    public String readNextSubstring(char div) throws IOException {
        buffer.delete(0, buffer.length());
        int i;
        char c;
        while ((i = reader.read()) > -1) {
            c = (char) i;
            if (c == div) break;
            buffer.append(c);
        }
        return buffer.toString();
    }
}

I use TextFileReader in this way:

TextFileReader reader = new TextFileReader(context, "big_file.txt");

//First group
String group = reader.readNextSubstring('#');
String[] info = group.split("\\|");
MyObject objA = new MyObject(info[0], info[1], info[2], info[3]);

//Second group
group = reader.readNextSubstring('#');
info = group.split("\\|");
MyObject objB = new MyObject(info[0], info[1], info[2], info[3]);

//Obviously this is done in a while loop :)
...

What do you think about this method? Could it be structured in a more efficient way?
I don’t care about space, I care about time (the emulator takes a LOT of time in order to read the file)

Answer 1

I can find one simple tweak that can speed you up simply by improving how much time you spend on doing this: group.split("\\|");

The way java does it is that for each time you run this line, it will compile a new regex.

If you do this:

Pattern pattern = Pattern.compile("\\|");
TextFileReader reader = new TextFileReader(context, "big_file.txt");

while (reader.hasMoreData) 
{
  String group = reader.readNextSubstring('#');
  String[] info = pattern.split(group);
  MyObject objA = new MyObject(info[0], info[1], info[2], info[3]);
  ...
}

This will reduce the time it takes to compile the pattern – which can be quite considerable for your amount of data.

Also, 1.5MB is not that much, perhaps loading it all to memory using a BufferedReader will probably improve your performance even more.