Question

I’m trying to make my own implementation of the FastICA algorithm based on the paper here: http://www.cs.helsinki.fi/u/ahyvarin/papers/NN00new.pdf.

I need some help w/ the math though.

In the middle of page 14 there is an equation that looks somewhat like

w+ = E{ xg(w^Tx) } – E{ g[prime]( w^T x)} w

What does the E mean? Back from my probability days I recall that it is the “expected value” of a random variable but it doesn’t make sense to me what the expected value of a vector is.

Thanks,

mj

Answer 1

ICA is interesting stuff. I used it some in my graduate research, but I didn’t dig in too much under the hood; I just downloaded the FastICA implementation for MatLab and used that.

Anyway, you are correct that E{…} denotes expected value. The elements of the vector x represent the individual signals. Strictly speaking, x is a time series and should be written x(t), but the convention in ICA is to treat x instead as a random variable. In that context, of course, the idea of expected value makes sense. For example E{x} would just be the mean value of x (taken to be zero in ICA as the signals have been centered).

The authors of the paper you linked also have a book on ICA. It’s outrageously expensive on Amazon, but if you can find a copy at, say, a nearby university library, it might be worth a look. It’s been several years, but I remember it as being as gentle an introduction as one could hope for given the mathematics.