I am trying to learn a dataset using Svmlight. I always see the following warning during the optimization process.
WARNING: Relaxing epsilon on KT-Conditions (0.010000).
.....................................................
WARNING: Relaxing epsilon on KT-Conditions (0.100000).
.....................................................
WARNING: Relaxing epsilon on KT-Conditions (1.000000).
.....................................................
WARNING: Relaxing epsilon on KT-Conditions (10.00000).
....................................................
Is it important and what does it mean? Currently training is not successful. Can it be related with this warning? Thanks.
The KT Conditions are part of the optimization problem that is at the core of SVM. Without getting into the theory in too much detail there are a series of conditions that when you have the optimal solution will be true. However, going through enough iterations to make the KT conditions exactly true takes longer than just making them true to within espilon (a very tiny number) of being true. Essentially rather than saying A must equal B, we say A must equal B within + or – epsilon. Research has found that this works just as well in practice as taking the extra time to get the conditions exactly right.
SVM Light is first trying with a very tiny epsilon, when it can’t solve that problem, it is relaxing epsilon by making it bigger to try to find one, but clearly if the problem is not solvable it tries a few times before deciding it isn’t solvable.
So I would say the warnings are because your problem is not solvable, rather than the other way around.