Question

How can I improve the efficiency of standard matrix multiplication algorithm?

The main operation involved in this approach is: C[i][j]+=A[i][p]*B[p][j]

What can be done to improve the efficiency of the algorithm?

Answer 1

You might want to have a look at using a BLAS (Basic Linear Algebra Subroutine) library, specifically Intel offer their MKL here, AMD have their ACML here and there’s also the (opensource) Goto BLAS here.

The (dense) matrix-matrix multiply kernel will be a ?GEMM call, where the ? indicates the floating point type. For example DGEMM will call the double routine.

Unless you’re extremely confident you know what you’re doing with low-level optimisations, these libraries will probably offer better performance than something you can code by hand.

If you do want to have a go at coding this yourself then you may want to consider the following:

1. Use “vector” instructions. SSE, SSE2..4 instructions are widely supported, some newer CPU‘s will also support AVX instructions.
2. Nested loop unrolling to maximise the ratio of floating point operations to load/store operations.
3. Block-wise algorithms to ensure effective cache use.
4. Multi-threading.

This reference might give you an idea of the current state of things:

High-performance implementation of the level-3 BLAS – K Goto.

Hope this helps.