Question

I’m trying to make a simple linear regression function but continue to encounter a

numpy.linalg.linalg.LinAlgError: Singular matrix error

Existing function (with debug prints):

def makeLLS(inputData, targetData):
    print "In makeLLS:"
    print "    Shape inputData:",inputData.shape
    print "    Shape targetData:",targetData.shape
    term1 = np.dot(inputData.T, inputData)
    term2 = np.dot(inputData.T, targetData)
    print "    Shape term1:",term1.shape
    print "    Shape term2:",term2.shape
    #print term1
    #print term2
    result = np.linalg.solve(term1, term2)
    return result

The output to the console with my test data is:

In makeLLS:
    Shape trainInput1: (773, 10)
    Shape trainTargetData: (773, 1)
    Shape term1: (10, 10)
    Shape term2: (10, 1)

Then it errors on the linalg.solve line. This is a textbook linear regression function and I can’t seem to figure out why it’s failing.

What is the singular matrix error?

Answer 1

As explained in the other answer linalg.solve expects a full rank matrix. This is because it tries to solve a matrix equation rather than do linear regression which should work for all ranks.

There are a few methods for linear regression. The simplest one I would suggest is the standard least squares method. Just use numpy.linalg.lstsq instead. The documentation including an example is here.