Question

I am currently writing a Java application to retrieve BLOB type data from the database and I use a query to get all the data and put them in a List of Map<String, Object> where the columns are stored. When I need to use the data I iterate the list to get the information.

However I got an OutOfMemoryError when I tried to get the list of rows more than a couple times. Do I need to release the memory in the codes? My code is as follows:

ByteArrayInputStream binaryStream = null;
OutputStream out = null;
try {   
   List<Map<String, Object>> result =
    jdbcOperations.query(
        sql,
        new Object[] {id},
        new RowMapper(){
          public Object mapRow(ResultSet rs, int i) throws SQLException {
            DefaultLobHandler lobHandler = new DefaultLobHandler();
            Map<String, Object> results = new HashMap<String, Object>();
            String fileName = rs.getString(ORIGINAL_FILE_NAME);
            if (!StringUtils.isBlank(fileName)) {
              results.put(ORIGINAL_FILE_NAME, fileName);
            }
            byte[] blobBytes = lobHandler.getBlobAsBytes(rs, "AttachedFile"); 
            results.put(BLOB, blobBytes);

            int entityID = rs.getInt(ENTITY_ID); 
            results.put(ENTITY_ID, entityID);
            return results;
          }
        }
    );

  int count = 0;
  for (Iterator<Map<String, Object>> iterator = result.iterator(); 
      iterator.hasNext();) 
  {
    count++;
    Map<String, Object> row = iterator.next();
    byte[] attachment = (byte[])row.get(BLOB);
    final int entityID = (Integer)row.get(ENTITY_ID);
    if( attachment != null) {
      final String originalFilename = (String)row.get(ORIGINAL_FILE_NAME);
      String stripFilename;
      if (originalFilename.contains(":\\")) {
        stripFilename = StringUtils.substringAfter(originalFilename, ":\\");
      }
      else {
        stripFilename = originalFilename;
      }
      String filename = pathName + entityID + "\\"+ stripFilename;

      boolean exist = (new File(filename)).exists();

      iterator.remove(); // release the resource

      if (!exist) {
        binaryStream = new ByteArrayInputStream(attachment);
        InputStream extractedStream = null;
        try {
          extractedStream = decompress(binaryStream);
          final byte[] buf = IOUtils.toByteArray(extractedStream);
          out = FileUtils.openOutputStream(new File(filename));
          IOUtils.write(buf, out);
        }
        finally {
          IOUtils.closeQuietly(extractedStream);
        }
      }
      else {
        continue;
      }
    }
  }
}
catch (FileNotFoundException e) {
  e.printStackTrace();
}
catch (IOException e) {
  e.printStackTrace();
}
finally {
  IOUtils.closeQuietly(out);
  IOUtils.closeQuietly(binaryStream);
}

Answer 1

Consider reorganizing your code so that you don’t keep all the blobs in memory at once. Instead of putting them all in a results map, output each one as you retrieve it.

The advice about expanding your memory settings is good also.