Question

I have an R function which is essentially acting as a wrapper to a set of C functions – the R code calls the C code through .C("..."). This C code could be parallelized and compiled using some MPI implementation. However, having never used MPI before, I have no idea if such MPI-ed code would even be callable from R in a way that would let MPI work?

Does anyone have any experience with this kind of thing? I’m guessing the R MPI libraries are pointless for my purpose, given all the work is done deep within the C code. This would ultimate be run on HPC cluster, if that makes any difference?

Can you use mpicc to create a shared objected, and if such a shared object was called from R would a parallel implementation run, or would you just get the serial version (or indeed, as I suspect, would it just crash)?

I may well be missing info needed to understand the problem, so will update accordingly.

Answer 1

You seem confused.

You could just use the existing Rmpi package to spawn parallel execution of several R instances (on your different nodes) and each of those can use your .C()-called code as well.

Maybe you want to work through some simple examples to get a better feel about what can or cannot be done?