Question

I’m having problems multiplying two matrices of dimensions 5000×1024.
I tried to do it through loops in a conventional manner, but it takes forever.
Is there any good library with implemented and optimized matrix operations, or any algorithm that does it without 3 loops?

Answer 1

Have you looked into using OpenCL? One of the examples in the Cloo (C# OpenCL library) distribution is a large 2D matrix multiplication.

Unlike CUDA, OpenCL kernels will run on your GPU (if available and supported) or the CPU. On the GPU you’ll see really, really, really dramatic speed increases. I mean, really dramatic, on the order of 10x-100x depending on how efficient your kernel is and how many cores your GPU has. (A Fermi-based NVidia card will have between 384-512, and the new 600’s have something like 1500.)

If you’re not interested in going that route – though anyone that’s doing numerically-intensive, easily parallelizable operations like this should be using the GPU – make sure you’re at least using C#’s built-in parallelization:

Parallel.For(
  0
  ,5000
  , (i) => { 
    for(var j=0;j<1024;j++)
    {
      result[i,j] = .....
    }
);

also, check out GPU.NET and Brahma. Brahma lets you build OpenCL kernels in C# with LINQ. Definitely lowers the learning curve.