Question

I have a simulation function that is very applicable to parallel processing. I have managed to make it run multicore (up to 32 cores on EC2 cc2.8xlarge).

Now i am wondering if I can also use Redis to make it run in multiple instances multiple cores in parallel.

The core of my simulation function is essentially

library(doMC)
registerDoMC(cores=32)
simresults<-foreach (i=1:nrow(input.data), .combine=c) %dopar% {
    simulation function (input.data$1[i],input.data$2[i],input.data$3[i])
}

simresults is then a vector of the results i gather. The simulation runs a couple of 10^7 times.

Is this possible with doRedis or any other tool (segue?)

Answer 1

Yes it is and there are several examples floating around — see for example the rredis Manual vignette of the rredis package.

There are of course other options besides redis and multicore.