Question

I have a problem related to Memory Leak. In my app I have to read 2MB data from a Video file, and the method related to same always called when the Activity’s onCreate method called then the same statement which allocated 2MB byte array in the code, returns OutofMemory Exception frequently after 10 to 15 attempts because heap memory exceeds. The code is explained below (it is the part of my whole code):

//Reading DRM video from sdcard
                File file = new File("/sdcard/TvAnyTime/watch/"+IDValue+".mp4");
                   try {
                          is = new FileInputStream(file);
                   } catch (FileNotFoundException e2) {
                          e2.printStackTrace();
                   }
                   //reading 2^21 bytes
                   fileData = new byte[2097152];
                   int read = 0;
                   while(read != fileData.length) {
                      try {
                    read += is.read(fileData, read, fileData.length - read);
                } catch (IOException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
                   }

                   //The finalHashPattern is then xored across the video file 2^10 times
                   for(int i=0;i<2097152;i+=2048)
                   {
                    byte[] res = new byte[2048];
                    bytesafterXor = new byte[2048];

                    for(int j=0;j<2048;j++)
                    {

                        res[j] = fileData[i+j];
                        bytesafterXor[j] = (byte)(res[j]^finalhash[j]);
                        finaldatafile[i+j] = bytesafterXor[j];
                    }

                    finalHashafterXor.add(bytesafterXor);

                   }

The statement fileData = new byte[2097152]; is responsible for the OutOfMemory Exception because it is allocated every time when onCreate is called. Can we prevent the same by allocating each time a large memory? can we read it in chunks of data? Please suggest me the right solution regarding the same.

Thanks in advance.

Answer 1

Ok, since my edit to Kingamajick’s answer “mysteriously” disappeared while waiting for peer review, here is how to do file (or streams in general) chunked reading right:

fileData = new byte[2048];
try {

  int c = 0;
  while( (c = is.read(fileData)) >= 0 ) {

    // Process all bytes from fileData[0] through fileData[c-1], e.g.:   
    for(int i = 0; i < c; i++) {
      // do something with fileData[i]
      // ...
    }
  }

} catch (IOException e) {
  // TODO Auto-generated catch block
  e.printStackTrace();
}

Note that for files, the buffer (fileData above) will most likely always be completely filled by the read operation until there are not enough bytes left to read in the file. When reading from, for instance, network streams this is usually not the case as data is not always instantly available and when it becomes available it is likely just as much data as arrived in the last network packet. However, the above approach works for all streams.

Edit:

From your comment I take that you do not want to process the whole file but only the first 2mb. In this case you can modify the above approach a bit, like:

fileData = new byte[2048];

int leftToRead = 2048*1024; // Total amount of bytes you want to read.

try {

  int c = 0;

  // How many bytes may we read at once?
  int maxRead = Math.min( fileData.length, leftToRead );

  while( (leftToRead > 0) && (c = is.read(fileData, 0, maxRead)) >= 0 ) {

    // Process all bytes from fileData[0] through fileData[c-1], e.g.:   
    for(int i = 0; i < c; i++) {
      // do something with fileData[i]
      // ...
    }

    // We read c bytes, so we may have some bytes left:
    leftToRead -= c;

    // How many bytes may we read at once?
    maxRead = Math.min( fileData.length, leftToRead );

  }

  // Optionally:
  if ( leftToRead > 0 ) {
    System.out.println("Premature end of file.");
  }

} catch (IOException e) {
  // TODO Auto-generated catch block
  e.printStackTrace();
}

When processing the file byte-by-byte, as you do in your code excerpt, you can basically freely choose the size of the fileData buffer. A smaller buffer does not provide any real benefit, because it may cause more read operations on the underlying file system. Something in the range from 1kb to 64kb is usually a good size.