Question

I have a cataloging application in Android for engineering industry. Over 300 thousand design images have been uploaded into the Android device. Over 200 new images are being synchronized daily to the device.

There is a table called CatalogDetail wherein the details of the designs are stored. It has around 20 columns for storing different attributes and two image paths, one for thumbnail and other for large image. The average size of thumbnail image is around 20KB and that of large image is around 40KB. And all these images are stored in the same folder (not in the database). The overall size of 300 thousand images is around 6GB. As per various search conditions, the thumbnail images and short descriptions are shown in the grid in the application.

If I’ve 5000 images in the folder, the application works properly and as the size of the data increases, the application became slower and slower. After loading 300 thousand images now, when I load 250 images (end user want at least 250 images to be loaded at a stretch) into the grid, loading to grid is comparatively slower, but when I try to scroll, it is taking quite a long time (around 10 – 15 seconds), which is not desirable.

I’ve profiled my SQLs and have found out that SQLs are NOT taking considerable time (less than 1 second to get data). Finally I’ve figured out that, the problem is with loading images from the folder. The images are stored in the internal memory. The application is working on Samsung Galaxy Tab 750 10.1 inch with Android 3.1 (Honeycomb) OS, RAM 1GB and 16GB internal memory. The application is developed using SDK 10.
Also one of the basic requirements from the end user is that the design images are very proprietary and they should be copied from the device. So the image folder is hidden and is not accessible outside the application.

Kindly let me know how the performance can be improved.
1. If images are index can the performance be improved? Is it possible to index (Is there any software, app, etc, available?
2. If I save 1000 images in each of the folders and if I create 300 folders and access from the images from them, can the performance be improved? But this should be the last option, since I’ve to re-load all images and modify the image location in my table.
3. Can threading help? (Sincerely I don’t know threading!)

Kindly suggest me how I can improve the performance.
Thanks for your help.

The search functionality will execute this method:

private void ShowData() {
    try {
        Cursor countc = dal.getCursor("Select count(*) From Product " + whereCondition);
        SQL = "Select ID as _id, Name, Code, ProductWeight, '" + fileLoc + "' || ImageLoc as ImageLoc, ReleaseDate From CatalogDetail " + whereCondition;
        Cursor c = dal.getCursor(SQL + " Order By " + orderByField + " " + OrderSeq);

        ProductGridView.setAdapter(new CatalogueListAdapter(this, c));
    } catch (Exception e) {
        e.printStackTrace();
    }
}  

This is how the catalogs are loaded to the grid:

public class CatalogueListAdapter extends SimpleCursorAdapter {
    private Context context;
    private Cursor c;

    public CatalogueListAdapter(Context context, Cursor c) {
        super(context, R.layout.thumbnail, c, new String[] {}, new int[] {});
        this.context = context;
        this.c = c;
    }
    @Override
    public View getView(int position, View inView, ViewGroup parent) {
        View v = inView;
        if (v == null) {
            LayoutInflater inflater = (LayoutInflater) context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
            v = inflater.inflate(R.layout.thumbnail, null);
        }
        c.moveToPosition(position);

        ImageView ivThumb = (ImageView) v.findViewById(R.id.thumbnail);
        String imageLoc = this.c.getString(this.c.getColumnIndex("ImageLoc"));
        if (imageLoc != null) 
            ivThumb.setImageURI(Uri.parse(imageLoc));
        return v;
    }
}

Answer 1

One thing that comes to my mind spontaneously would be to simply distribute the pictures into subfolders based on their names, or, if that is not possible, giving them names suitable for that.

Usually, an open() call requires to scan through a special file called a directory. If there are many entries, this linear seach can become very slow.

If you will, my suggestion is a bit like “indexing”, although I like the “hybrid” approach here. Just because Bubble Sort is always fastest for only a few items because of less overhead, similar rules apply for directories, too.

So if the images can be distributed by their file names or hash values of their file names, then use a subdirectory structure to place them. An evenly distribution must, of course, be the primary goal here, so choose the names or hash function wisely.

To suimmarize:

1. Yes and no; see text above. 😉
2. Yes, and I’d prefer it, because an additional index alone won’t help if my assumption is correct that the problem is really with open().
3. Again, if my assumption is right, then threading won’t help.