Question

I am trying to figure out why certain arrays I am saving as .rda’s seem to eat up more memory than others of equal size. Below are two objects, x and y,of the same size, type, and dimension. When I save each of these, one is 41 Mb and the other is 6Mb. Can anyone think of a reason why this could happen?

> dim(x)
[1]    71    14 10000
> dim(y)
[1]    71    14 10000 
> class(x)
[1] "array"
> class(y)
[1] "array"  
> object.size(y)
79520208 bytes
> object.size(x)
79520208 bytes

Answer 1

If you save using either the save or saveRDS commands, the default is to use compression. If you have different content in the vectors, they’ll compress differently…

Try save with compress=FALSE and compare again…

In the example below there is almost a 700x difference in file size:

set.seed(42)
x <- runif(1e6)  # random values should not compress well...
y <- rep(0, 1e6) # zeroes should compress very well...
object.size(x) # 8000040 bytes
object.size(y) # 8000040 bytes

save('x', file='x.rds')
save('y', file='y.rds')
file.info(c('x.rds', 'y.rds'))$size
#[1] 5316773    7838

save('x', file='x.rds', compress=FALSE)
save('y', file='y.rds', compress=FALSE)
file.info(c('x.rds', 'y.rds'))$size
#[1] 8000048 8000048