Question

I have the equation A * x = b

sizes of A is matrix sized n x m, x is m x 1 and b is n x 1.
A has more rows than columns (n < m).

My unknown is A and since n != m, A does not have an inverse.
My knowns are the two vectors x and b. Basically, I want to find the A that makes (A * x – b) close to zero.

Least squares seems appropriate, but I am not sure on how to proceed since it doesn’t seem to follow my text book nor the wikipedia entry; usually the matrix A is known. If OLS isn’t appropriate, what would be? Singluar Value Decomposition? Again, pointers please, my linear-algebra is rusty.

Would like to be able to implement this (python/C). Pointers to good readable code?

Answer 1

There are infinitely many solutions to this, and nothing as sophisticated as a singular value decomposition will be necessary.

If n = m, then A could be simply a diagonal matrix, but you have n > m. So we cannot be so simple.

We can view this as the problem of finding n independent vectors (the rows of A) such that when you form their dot product with x, they yield the corresponding element of b. Thus A*x=b (the solution will be exact) is equivalent to finding rows of A such that if A_i is the i’th row of A, then dot(A_i,x) = b(i).

Viewed in this way, the solution is obvious! As obviously, it shows why there is no “solution”, but infinitely many such solutions, all of which are pretty trivial.

So for example, pick ANY element of x that is non-zero. Say it is the kth element. Then create A_i as the vector that is ALL zero, except for the corresponding chosen non-zero element of x.

A_i(k) = b(i)/x(k)

All other elements of A_i are zero. So now, form the dot product. See that A*x = b is solved exactly, by a matrix X which has only one non-zero column, and that column a rather trivial one.