Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Does Divide & Conquer Matrix Multiplication perform the same amount of additions/subtractions as the Classical Matrix multiplication?
I know that they do specifically for Multiplications as they both share the same O(n^3) complexity…
but when I try to count them in the program i’m making, the additions/subtracts are coming to a different numbers, and I’m not sure if this is correct.
If anyone knows for sure please let me know, thanks.
Let’s assume square matrices.
If you count the number of additions (there are no subtractions) in classic matrix multiplication, you get N^3 additions. There are N^2 elements, and each element is the dot-product of a row and column consisting of N-1 additions, so almost exactly N^3 additions.
To count the number of additions in divide-and-conquer matrix multiplication, let’s see how it works:
Split up NxN matrix into four (N/2)x(N/2) matrices, then treat it as a 2×2 matrix and perform block multiplication recursively. For example multiplying two 8×8 matrices:
The new matrix will be:
Each multiplication of [N/2xN/2]*[N/2xN/2] is a subproblem of size N/2. We must do 8 of these subproblems. This gives us a recurrence from the above:
That is, if we pay the price of N^2 additions, we are allowed to break down the problem of size N into 8 subproblems of size N/2.
We can solve using the Master Theorem (or more general Akra-Bazzi Theorem), or by inspection:
Using the Master Theorem,
additions[N] = O(N^(log_2(8))) = O(N^3)Why would we do this since it’s the same order of growth? We wouldn’t. It turns out that in order to get better asymptotic complexity, you don’t want to do this, you want to use an algebraic trick called Strassen’s method. See http://www.cs.berkeley.edu/~jordan/courses/170-fall05/notes/dc.pdf on page 4. Our new recurrence relation comes from counting the number of multiplications and additions as shown on that page. There are 18 additions of [N/2xN/2] matrices required to form an NxN matrix.
As we see, we have to do one fewer subproblem, but at the cost of doing more work in combining. The master theorem says that
additions[N] = O(N^(log_2(7))) ~= O(N^2.807).So asymptotically, there will be FEWER additions, but only asymptotically. The real story is revealed when we simulate both recurrence relations:
Results:
As we can see, with respect to additions only, Strassen outperforms the traditional normal matrix-multiplication with respect to number of additions, but only once your matrices exceed the size of roughly 30000×30000.
(Also note that the naive divide-and-conquer multiplication performs asymptotically the same, in terms of additions, as traditional matrix multiplication. However it still performs “worse” by initially a factor of 3, but as the matrix size increases, asymptotically worse by a factor of exactly 2 . Of course this tells us nothing about the true complexity which involves multiplications, but if it did, we might still want to use it if we had a parallel algorithm which could take advantage of the different computation structure.)