Question

I’d like to find the principal components of a data matrix X in Matlab by solving the optimization problem min||X-XBB’||, where the norm is the Frobenius norm, and B is an orthonormal matrix. I’m wondering if anyone could tell me how to do that. Ideally, I’d like to be able to do this using the optimization toolbox. I know how to find the principal components using other methods. My goal is to understand how to set up and solve an optimization problem which has a matrix as the answer. I’d very much appreciate any suggestions or comments.

Thanks!
MJ

Answer 1

The thing about Optimization is that there are different methods to solve a problem, some of which can require extensive computation.

Your solution, given the constraints for B, is to use fmincon. Start by creating a file for the non-linear constraints:

function [c,ceq] = nonLinCon(x)
c = 0;
ceq = norm((x'*x - eye (size(x))),'fro'); %this checks to see if B is orthonormal.

then call the routine:

B = fmincon(@(B) norm(X - X*B*B','fro'),B0,[],[],[],[],[],[],@nonLinCon)

with B0 being a good guess on what the answer will be.

Also, you need to understand that this algorithms tries to find a local minimum, which may not be the solution you ultimately want. For instance:

X = randn(1,2)
fmincon(@(B) norm(X - X*B*B','fro'),rand(2),[],[],[],[],[],[],@nonLinCon)
ans =
    0.4904    0.8719
    0.8708   -0.4909
fmincon(@(B) norm(X - X*B*B','fro'),rand(2),[],[],[],[],[],[],@nonLinCon)
ans =
    0.9864   -0.1646
    0.1646    0.9864

So be careful, when using these methods, and try to select a good starting point