Question

I am currently trying to solve a problem on one of the online programming contests. The limit for the program is 64 megabyte in this contest.

I wrote a program in Java that has a section of fields in the class declararation that works like this:

private int[] sizes = new int[1024]; // 4096 bytes
private boolean[][] compat = new boolean[1024][1024]; // 128 kb
private boolean[][] compat2 = new boolean[1024][1024]; // 128 kb

private long[][][] dp = new long[29000][51][2]; // About 3*8 = 24 megabytes
private int [][] masks  = new int[29000][2]; // About 240 kb
private int avail = 0; 
private int avail2 = 0;
private int[] positions = new int[500000]; // About 2 megabytes
private int[][] ranges = new int[29000][2]; // About 240 kb
private int[][] maskToPos = new int[1024][1024]; // About 4 megabytes
private int[][][] init = new int[29000][51][2]; // About 3*4 = 12 megabytes

Now, the class has only a main procedure and some cycles inside it, without any additional arrays declared (just some variable to iterate through the cycles). However, then I tried to run this code on my local machine with the key -Xmx64m, I’ve got an OutOfMemoryError. It only managed to execute with the key -Xmx128m.

I also tried to lauch in on the online server, it gave the same error, and also gave additional info that my program used about 148460 kb.

But why so much? As far as I can calculate from the fragment above, it only should use about 40 megabytes. Is there something wrong with this calculation in the comments?

Answer 1

These two are the biggest killers:

private long[][][] dp = new long[29000][51][2]; // About 3*8 = 24 megabytes
private int[][][] init = new int[29000][51][2]; // About 3*4 = 12 megabytes

Looking at the second, for example… that isn’t 12 megabytes. You’ve got 29000 int[][] objects, each of which contains references to 51 int[] objects, each of which contains 2 integers.

Assuming a 32-bit reference size and 16 bytes of overhead for the array itself (length + common object overhead) that means the int[][] objects are each of size 51 * 4 + 16 = 220 bytes, and then the int[] objects are each of size 24 bytes. But you’ve got 29000 * 51 of those 24-byte objects – which is 35MB just in itself… Then there’s the 29000 int[][] objects, which is another 6MB… (Then there’s the top level array itself, but that’s only about 120K.)

Basically, you need to remember that Java doesn’t have multi-dimensional arrays: it has arrays of arrays, and each array is an object, with separate overhead. I suggest you may want to use:

private int[] init = new int[29000 * 51 * 2];

instead, and work out appropriate offsets yourself. (Ditto for dp, which is even worse as it’s long values rather than int values, making each of the 29000 * 51 arrays take at least 32 bytes rather than 24.)

Even just reversing the order in which you treat the dimensions would help:

private long[][][] dp = new long[2][51][29000];
private int[][][] init = new int[2][51][29000];

Now for each of these variables, there’s one top-level array-of-arrays-of-arrays, 2 arrays-of-arrays, and 102 arrays of long or int. That corresponds to a lot less overhead.

Your other calculations are incorrect too, but I think these two arrays-of-arrays-of-arrays are the worst.