Question

I am trying out Liblinear for linear SVM classification on some 2D points (I am using a simple python gui to add points for 2 classes and then draw the line that separates the classes), but even though I am using the bias option (-B 1) for training, I get a bias very close to zero (the separating line almost passes through the origin).

I also tried simply training the 2-point set:

-1 1:10 2:30
+1 1:10 2:80

but I still get a very small bias (a line passing through the origin instead of a horizontal line in the XY plane as I guess it should be). Here is my output vector w:

0.2003362041634111, 
-0.03465897160331861, 
0.0200336204163411 

What am I doing wrong?

Answer 1

I’m not sure you are doing anything wrong.

From the liblinear FAQ:

Q: Is LIBLINEAR gives the same result as LIBSVM with linear kernel?

They should be very similar. However, sometimes the difference may not
be small. Note that LIBLINEAR does not use the bias term b by default.
If you observe very different results, try to set -B 1 for LIBLINEAR.
This will add the bias term to the loss function as well as the
regularization term (w^Tw + b^2). Then, results should be closer.

This says that liblinear tries to make the bias term as small as possible. If it can set it to zero and still get good training-set accuracy then it will.

There isn’t a particularly good reason to imagine that regularising the bias will provide a better classifier, so many other learning systems it does not enter into the regularisation term. However in ‘real-world’ problems with very high dimensionality it’s also very likely that the data are separable without needing a bias term, so regularising it does no harm and can be easier to implement.