Question

Ok, here’s the deal. I have an array (inputted from a 400 MB file) where if I run the sort() command on it, comp runs out of memory. The input file isn’t the problem, so I’ve decided to break the initial array into smaller arrays that I can perform the sort on. I can break the initial array into arrays of size 100k, which my code does.

(for the purposes of this testing, I’ve shrank the file from 400 MB to 40 MB)

I run my break array code, and on first iteration, i have an array of 100k as a reference in my @arrayList. My code is just this:

push @arrayList, \@sorted;        #@sorted is the sorted version of the 100k array
$temp = @arrayList;               #returns 1, which it should 
@arrayTemp2 = @{$arrayList[0]};
$temp = @arrayTemp2;              #returns 100k, which it should
@arrayTemp2 = @{$arrayList[1]};
$temp = @arrayTemp2;              #returns 0 since it is uninitialized

On the next loop in the for loop, the sorted array is the rest of the initial array, only 23k. Same code runs again, with these results:

push @arrayList, \@sorted;        #@sorted is the sorted version of the 23k array
$temp = @arrayList;               #returns 2, which it should 
@arrayTemp2 = @{$arrayList[0]};
$temp = @arrayTemp2;              #returns 23301, which is wrong
@arrayTemp2 = @{$arrayList[1]};
$temp = @arrayTemp2;              #returns 23301, which is right.

I’ve tried using every different way I can think of to fix this, and I just have no ideas left. Any help?

Thanks

Answer 1

Depending on how you are scoping it, you might be overwriting your memory when you update @sorted. See if this gives you better results:

push @arrayList, [ @sorted ];    # an array reference to a *copy* of @sorted