Question

i have an xts object which is about 2 mil rows. i am trying to use rollapply with width=10000. rollapply seems to run out of memory and i am having trouble figuring out why? (i tried to look at the source by typing > rollapply but only this shows up:

function (data, ...) 
 UseMethod("rollapply")
 <environment: namespace:zoo>

does rollapply create 2 million – 10,000 copies of each chunk and then pass them in one by one? this would certainly cause it to run out of memory but i don’t think the creators of zoo would have done that? if that is the case, then does anyone know a more clever way to do my rolling apply.. my current plan is to simply do this:

m <- vector()
for (i in 10001:nrow(my_xts)) {
   m <- c(m, my_fun(my_xts[i-10000:i]))

}

sure there is a better way?

Answer 1

It looks like the memory consumption is occurring during the mapply call in zoo:::rollapply.zoo. There is an xts rollapply method but it is not exported, so it will not be called via method dispatch. That said, you can call it explicitly:

my_xts <- .xts(1:1e5*1,1:1e5)
m <- xts:::rollapply.xts(my_xts, width=10000, FUN=sum)

Use methods(foo) to see what methods are available whenever you encounter:

function(...)
  UseMethod("foo")