Question

Assuming I have a haar cascade xml file. Is there a way to render the file into an image to create the “perfect specimen”?

So if the xml was for face detection, it will show an image of the face that would get the highest score?

If not possible, is there a way to image the “principal components”?

btw, I am using the Python version of openCV.

Answer 1

As far as I know, this would be difficult. The classification scheme is complex, and I don’t think there’s an easy way to “invert” it, to get the “perfect specimen,” or even something like the principal components. (More on principal components below.) Let me try to quickly explain how the classification is done, so that it’s clearer why these things are hard; then I’ll mention a few other approaches you could try.

How the cascade works (approximately)

The idea of the cascade is to make classification faster by rejecting as many non-faces as quickly as possible. (This is important since most things in an image will be non-faces.) You do this by having a cascade of classifiers. The first classifier is trained so that it rejects, say, 90% of the things that are not faces, but keeps at least 99.99% of faces. This is still poor performance — you’ll get tons of false positives if you just use this classifier. However, you can perform this classification pretty quickly, so you can quickly get rid of lots of non-faces.

OK, so what’s in one of these classifiers? Well, each classifier is a boosted set of weak classifiers. What that means is, for every image you pass to the classifier, it passes it on to each of N very simple classifiers, and each of the simple classifiers votes “true” or “false.” Each of the simple (weak) classifiers is weighted, so the weighted vote gives the answer for the strong classifier.

OK — so what is a weak classifier? I think in OpenCV, the weak classifiers are decision trees of height 2 (see OpenCV docs, starting page ~368). So each weak classifier compares the input image against a simple pattern; if the image is similar enough to the pattern, it’s passed off to be compared with another pattern; and if it’s sufficiently dissimilar, it’ll be passed on to be compared with a third pattern.

Why this makes things hard, and some things you could try

So as you can probably see, it’s not really possible to make a “perfect specimen,” since there are many nonlinearities in the decision process — a “perfect specimen” for one of the weak classifiers may be a terrible specimen for most of the others, meaning it would not, overall, make for a good face. Same problem with mapping important components, but you might be able to get somewhere by choosing the most important (that is, highest-weighted) weak classifiers in the first (or the final) stage, and then plotting the filters that are used by those stages.

Another approach you could try is one that I saw in an art project, but can’t find a link to, sadly. An artist generated random combinations of shapes, and saw how well they triggered a face detector. He would randomly add more shapes, and keep configurations of shapes that did a better job of triggering face detection. Through this hill-climbing strategy, he was able to generate images that looked like faces to the algorithm. A more nuanced version of this approach — searching stimulus space for things that are classified as faces — may help with finding a “perfect specimen,” so to speak.

Hope this helps…good luck!