Question

I’m presently working with data in text files. I need to use algorithm called principal component analysis so I have counted the words in text file which occurred more than one time in text file for eg

relation occured times
help occured 6 times
between OCCURED 3 TIMES
Analysis occurred 4 times
component occured 5 times
present occurred 6 times

So by taking count of above distinct words i need to form matrix of m x n. I am using C#.

Answer 1

Several points:

1. PCA is simple. However, you really need understand it before using it. This is a pity as it is not a black box tool, like a clustering algorithm.

2. PCA is performed on the covariance matrix(that is X’*X, where each row of X is a text document). You can see that you cannot store a # of words by # of words matrix in memory. So for text data, you cannot directly use PCA. You need to use SVD and this technique is called latent semantic analysis. PCA and SVD are the same when the data are centered. In practice, data centering is not applied to text data as centering causes sparse into dense.

3. Both PCA and SVD is easy, several lines of Matlab code. Only several lines of C# code if you have a linear algebra library for eigen-decomposition or SVD. The hard part as I noted is that you need to understand them.

4. A more popular method to analyze text documents is probabilistic latent semantic analysis. Which is easy to understand and easy to code without using any matrix decompositions. Of course, you still need to learn some math.