Question

The problem i am trying to understand is easy but i cant seem to get the correct result in matlab. The actual problem is that i want to get the weight vectors of a 2 hidden layer input RBF using just the plain distance as a function, i.e. no Baysian or Gaussian function as my φ. I will use the function with 2 centres let’s say 0,0 and 1,1. So this will give me a Matrix φ of:

[0 sqrt(2) ; 1 1; 1 1; sqrt(2) 0] *[w1; w2] = [0;1;1;0] As defined my the XOR function.

When i apply the pseudoinverse of the Φ in matlab * [0;1;1;0] though i get [0.33 ; 0.33] which is not the correct value which would allow me to get the correct output values [0;1;1;0].

i.e. .33 * sqrt(2) != 0 .

Can someone explain to me why this is the case?

Answer 1

I’ll take a swag at this. The matrix, I’ll call A, A = [0 sqrt(2) ; 1 1; 1 1; sqrt(2) 0] has full column rank, but not full row rank, i.e. rank(A) = 2. Then you essentially solve the system Ax = b, where x is your weighting vector. You could also just do x = A\b in Matlab, which is supposedly a much more accurate answer. I get the same answer as you. This is a very rough explanation, when your system can not be solved for a certain solution vector, it means that there exists no such vector x that can be solved for Ax = b. What Matlab does is try to estimate the answer as close as possible. I’m guessing you used pinv, if you look at the Matlab help it says:

If A has more rows than columns and is not of full rank, then the overdetermined least squares problem

minimize norm(A*x-b)

does not have a unique solution. Two of the infinitely many solutions are

x = pinv(A)*b 

and

y = A\b 

So, this appears to be your problem. I would recommend looking at your φ matrix if possible to come up with a more robust system. Hope this is useful.