Question

I am trying to optimize the following code.

dim <- c(10000,100)

m <- matrix(sample(0:10, prod(dim), replace = TRUE), nrow = dim[1], ncol = dim[2])

system.time({

  output <- matrix(0, nrow = dim[1], ncol = dim[2])

  for (i in 1:dim[1]){        
    output[i,1] <- m[i,1]       
    for (j in 2:dim[2]){          
      output[i,j] <- output[i, j-1] * 0.5 + m[i,j]          
    }        
  }
})

Conceptually, it is quite similar to a simple cumulative sum:

system.time({

  output <- matrix(0, nrow = dim[1], ncol = dim[2])

  for (i in 1:dim[1]){       
      output[i,] <- cumsum(m[i,])        
  }
})

The problem is, the first part of the code is about 100 times slower. Is there any way to build a customized version of cumsum() that would do the trick ?

Answer 1

Your case is exactly the same as generating a AR(1) model with coefficient 0.5. You can use the filter function to generate the data. filter also support higher order recursion, convolution or mixture of them(think about the ARMA model). You may have a look ofconvolve for other convolutions. Also, you could compiler your code to speed up the loop. In my code, complied loop and uncompiled loop code is about 111 and 162 times slower than filter respectively.

library(compiler)
library(rbenchmark)

CustomCumsum<-function(x,alpha){
out<-x[1]
for(i in 2:length(x))
    out[i] <- out[i-1]*alpha+x[i]
out
}

compiledCustomCumsum<-cmpfun(CustomCumsum)

FilterCustomCumsum<-function(x,alpha) as.numeric(filter(x,alpha, method = "recursive"))

x<-rnorm(1000)
# Test whether they are the same
identical(compiledCustomCumsum(x,0.5) , FilterCustomCumsum(x,0.5) )

benchmark(
CustomCumsum=CustomCumsum(x,0.5),compiledCustomCumsum=compiledCustomCumsum(x,0.5),          FilterCustomCumsum=FilterCustomCumsum(x,0.5)
)

output:

                  test replications elapsed relative user.self sys.self user.child sys.child
2 compiledCustomCumsum          100    8.89  111.125      8.78     0.01         NA        NA
1         CustomCumsum          100   13.02  162.750     11.84     0.50         NA        NA
3   FilterCustomCumsum          100    0.08    1.000      0.08     0.00         NA        NA