Question

What is the difference between a Bayesian network and a Naive Bayes classifier? I noticed one is just implemented in Matlab as classify the other has an entire net toolbox.

If you could explain in your answer which one is more likely to provide a better accuracy as well I would be grateful (not a pre-requisite).

Answer 1

Short answer, if you’re only interested in solving a prediction task: use Naive Bayes.

A Bayesian network (has a good wikipedia page) models relationships between features in a very general way. If you know what these relationships are, or have enough data to derive them, then it may be appropriate to use a Bayesian network.

A Naive Bayes classifier is a simple model that describes particular class of Bayesian network – where all of the features are class-conditionally independent. Because of this, there are certain problems that Naive Bayes cannot solve (example below). However, its simplicity also makes it easier to apply, and it requires less data to get a good result in many cases.

Example: XOR

You have a learning problem with binary features x1 and x2 and a target variable y = x1 XOR x2.

In a Naive Bayes classifier, x1 and x2 must be treated independently – so you would compute things like “The probability that y = 1 given that x1 = 1” – hopefully you can see that this isn’t helpful, because x1 = 1 doesn’t make y = 1 any more or less likely. Since a Bayesian network does not assume independence, it would be able to solve such a problem.