Question

I have an assignment to implement MoG with EM in matlab. The assignment:

My code atm;

clear
clc

load('data2')

%% INITIALIZE
K = 20
pi =  0.01:((1-0.01)/K):1;
for k=1:20
    sigma{k} = eye(2);
    mu(k,:) = [rand(1),rand(1)];
end


%% Posterior over the laten variables
addition = 0;

for k =1:20 
    addition = addition + (pi(k)*mvnpdf(x,mu(k,:), sigma{k}));
end
test = 0;
for k =1:20 
   gamma{k} =  (pi(k)*mvnpdf(x,mu(k), sigma{k})) ./ addition;  
end

data has 1000 rows and 2 columns (so 1000 datapoints). My question is now how do I calculate the responsibilities. When I try to calculate the covariance matrix I get a 1×1000 matrix. While I believe the covariance matrix should be 2×2.

Answer 1

Unfortunately, I don’t speak Matlab, so I can’t really see where your code is incorrect, but I can answer generally (and maybe someone who knows Matlab can see if your code can be salvaged). Each datapoint has a gamma associated with it, which is the expectation of an indicator variable for each component in the mixture. Calculating them is pretty simple: for the i-th datapoint and the k-th component, gamma_ik is just the density of the k-th component at the i-th point, multiplied by the k-th mixture coefficient (the prior probability that the point came from the k-th component, which is pi in your assignment), normalised by this quantity computed over all k. Thus for each datapoint, you have a vector of responsibilities (of length k) with a sum of one.