Question

I have an Android app that allows the user to record data (such as accelerometer, latitude, longitude, etc.). There are a total of 9 of these fields, and the user can record for up to 10 minutes (3000 records per field). Therefore, a total of 27,000 data points can be collected. The user can also take pictures and videos to upload the SD Card as well.

When the user is done collecting data (or the 10 minutes are up), the data is stored in a String, which is later uploaded to the SD Card as a .csv file. The problem though is that it takes forever for the data to append to the string, due to the massive amount of garbage collecting (it appears to be approximately 5 collects or so per second!) The appending starts off fast, but seems to be slowing down as more and more as data is added.

This is the loop that causes the lag:

           for( i = 0 ; i < len2 ; i++ ) {
                data += accelX[i] + ", " + accelY[i] + ", " + accelZ[i] + 
                ", " + accelT[i] + ", " + latitu[i] + ", " + 
                longit[i] + ", " + orient[i] + ", " + 
                magneX[i] + ", " + magneY[i] + ", " + magneZ[i] + 
                ", " + millis[i] + "\n";

                partialProg = 100.0 * ( (double)(i+1) / (double)(len2));
                dia.setProgress((int) partialProg);

            }

data is just a String, and nothing is being newed, so I’m unsure of why GC is being called so often. My question is: what is the problem here, and/or how can I make this more efficient?

Answer 1

You could use StringBuilder to concatenate the data:

StringBuilder sb = new StringBuilder();
for( i = 0 ; i < len2 ; i++ ) {
    sb.append(accelX[i]).append(", ");
    sb.append(accelY[i]).append(", ");
    sb.append(accelZ[i]).append(", ");
    sb.append(accelT[i]).append(", ");
    sb.append(latitu[i]).append(", ");
    sb.append(longit[i]).append(", ");
    sb.append(orient[i]).append(", ");
    sb.append(magneX[i]).append(", ");
    sb.append(magneY[i]).append(", ");
    sb.append(magneZ[i]).append(", ");
    sb.append(millis[i]).append("\n");
}

StringBuilder is inherently faster for building long strings.

It also avoids allocating so many String objects into the heap; since each += operator is creating a new String object rather than modifying the last one. This in turn leads to a large number of GC calls to clean up all the redundent String objects. See also: http://chaoticjava.com/posts/stringbuilder-vs-string/

As Marcelo points out; you may also find dealing with large amounts of data in memory may become problematic on low-spec Android devices, at which point you should consider appending the contents of your StringBuilder to a temporary file every X number of iterations to keep the footprint low. At the end of the process you can stream the file to whatever the destination is planned to be, or read segments of it back to memory on demand.