Home/ Questions/Q 8325391
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-09T00:25:19+00:00

I am trying to use Weka for feature selection using PCA algorithm. My original

  • 0

I am trying to use Weka for feature selection using PCA algorithm.

My original feature space contains ~9000 attributes, in 2700 samples.

I tried to reduce dimensionality of the data using the following code:

AttributeSelection selector = new AttributeSelection();
PrincipalComponents pca = new PrincipalComponents();
Ranker ranker = new Ranker();
selector.setEvaluator(pca);
selector.setSearch(ranker);
Instances instances = SamplesManager.asWekaInstances(trainSet);
try { 
    selector.SelectAttributes(instances);
    return SamplesManager.asSamplesList(selector.reduceDimensionality(instances));
} catch (Exception e ) {
            ...
}

However, It did not finish to run within 12 hours. It is stuck in the method selector.SelectAttributes(instances);.

My questions are:
Is so long computation time expected for weka’s PCA? Or am I using PCA wrongly?

If the long run time is expected:

How can I tune the PCA algorithm to run much faster? Can you suggest an alternative? (+ example code how to use it)?

If it is not:

What am I doing wrong? How should I invoke PCA using weka and get my reduced dimensionality?

Update: The comments confirms my suspicion that it is taking much more time than expected.

I’d like to know: How can I get PCA in java – using weka or an alternative library.

Added a bounty for this one.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    After deepening in the WEKA code, the bottle neck is creating the covariance matrix, and then calculating the eigenvectors for this matrix. Even trying to switch to sparsed matrix implementation (I used COLT‘s SparseDoubleMatrix2D) did not help.

    The solution I came up with was first reduce the dimensionality using a first fast method (I used information gain ranker, and filtering based on document frequencey), and then use PCA on the reduced dimensionality to reduce it farther.

    The code is more complex, but it essentially comes down to this:

    Ranker ranker = new Ranker();
InfoGainAttributeEval ig = new InfoGainAttributeEval();
Instances instances = SamplesManager.asWekaInstances(trainSet);
ig.buildEvaluator(instances);
firstAttributes = ranker.search(ig,instances);
candidates = Arrays.copyOfRange(firstAttributes, 0, FIRST_SIZE_REDUCTION);
instances = reduceDimenstions(instances, candidates)
PrincipalComponents pca = new PrincipalComponents();
pca.setVarianceCovered(var);
ranker = new Ranker();
ranker.setNumToSelect(numFeatures);
selection = new AttributeSelection();
selection.setEvaluator(pca);
selection.setSearch(ranker);
selection.SelectAttributes(instances );
instances = selection.reduceDimensionality(wekaInstances);

    However, this method scored worse then using a greedy information gain and a ranker, when I cross-validated for estimated accuracy.

    • 0