Question

Possible Duplicate:
Speed up the loop operation in R

I have a few questions regarding loops. I know that R works faster with vectorized calculations, and I would like to change the below code to take advantage of this. Looking into some other answers on the forum the sapply function seems to be able to replace the inside for loop, but I am generating a vector of zeros so there is an error. Tao remains 1000, and I think this is creating the problem.

My primary concern is speed, as I need to create a loop around the entire algorithm and plot in different V and n sizes for some further analysis.

Thanks for your help

Alternative loop

tao = 1000
L = (tao - 1)   
n = 10      
V = 5               
I = 10000                       
V_s = matrix(rnorm(I), I, 1)
V_b = matrix(rnorm(I), I, 1)

signal <- matrix(0, L, 1)  

for( j in (n:L)){

    sapply(((j-n+1):j),function (tao) signal[j] = signal[j] + abs(V_s[tao] - V_b[tao]))

    signal[j] = (signal[j] / (n * V) )

} 

Original loop

tao = 1000
L = (tao - 1)   
n = 10      
V = 5               
I = 10000                       
V_s = matrix(rnorm(I), I, 1)
V_b = matrix(rnorm(I), I, 1)

signal <- matrix(0, L, 1)  

for( j in (n:L)){

    for( tao in ((j-n+1):j))    {

        signal[j] = (signal[j] + abs(V_s[tao] - V_b[tao]))

    }
        signal[j] = (signal[j] / (n * V) )

}

Answer 1

Using filters, you can do your computation even without any loop (and sapply is nothing more than a hidden loop).

absdif <- abs(V_s - V_b)
signal <- filter(absdif[1:L], rep(1/(n*V), n), sides=1)
signal[is.na(signal)] <- 0

Understanding what is happening in the second line is not trivial when your not used to filters, though. Let’s have a closer look:

First we compute the absolute differences of V_s and V_b, which you loop uses frequently. Then comes the filter.
Your computation is nothing more than summing up the the n past values at each time value j. Thus, we have something like

signal[j] <- sum(absdif[j-n+1:j])

That is exactly what convolution filters do – summing up some values – in the general form by multiplication with some weight. As weight we choose 1/(n*V), for all values, which corresponds to the normalization you do in your outer loop. The last argument, sides=1 simply tells the filter to take values only from the past (sides=2 would mean sum(absdif[(j-n/2):(j+n/2)])).

The last line just fills up the NA values at the beginning (where the filter does not have enough data to compute the sum – this equals to skipping the first n values).

Finally, some timing:

Your full-loop solution:

   User      System       total 
  0.037       0.000       0.037 

The solution of juba:

   User      System       total 
  0.007       0.000       0.008 

The solution using filters:

   User      System       total 
  0.000       0.000       0.001 

Note that the concept of filters is really well researched and can be done incredibly fast.

Edit:
As noted in ?filter, R does not use Fast fourier transform with the standard filter command. Usually, FFT is the most efficient way to implement convolutions. However, even this can be done by replacing the filter command with

signal <- convolve(absdif[1:L], rep(1/(n*V), n), type='filter')

Note that now the first n entries are stripped of instead of set to NA. The result is the same, however. Timing this time is of no use – the total time is below the three digit output of system.time… However, note the following remark in the R help of filter:

convolve(, type=”filter”) uses the FFT for computations and so may be faster for long filters on univariate series, but it does not return a time series (and so the time alignment is unclear), nor does it handle missing values. filter is faster for a filter of length 100 on a series of length 1000, for example