Question

I’m building an application that will take an image of a single person’s whole body and will produce a “mugshot” for that person.

Mugshot meaning an image of the person’s whole face, neck, hair and ears at the same general size of another mugshot.

Currently I’m using
http://askernest.com/archive/2008/05/03/face-detection-in-c.aspx
to implement OpenCV and I’m using

harrcascade_frontalface_default.xml  
harrcascade_frontalface_alt.xml  
harrcascade_frontalface_alt2.xml  
harrcascade_frontalface_alt_tree.xml

as my cascades.

I use all of the cascades because a single one will not detect all my faces. After I get all of the faces detected by all of the cascades I find my average square and use that for my final guess of how tall and wide the mugshot should be.

My problem is 3 parts.

• My current process is rather slow. How can I speed up the detection process?
  Edit: I’m finding that the processing time is directly related to photo size. Reducing the size of the photos may prove to be helpful.

• A single cascade will not detect all the faces I come across so I’m using all of them. This of course produces many varied squares and a few false positives. What method can I use to identify false positives and leave them out of the average square calculation? ex.
  Edit : I’m implementing an average of values within standard deviation. Will post code soon.

• I’m not exactly sure of the best way find the mugshot given the square coordinates of the face. Where can I find face to mugshot ratios?
  Edit : Solved this one. Assuming all my heads are ratios of their faces.

  static public Rectangle GetMugshotRectangle(Rectangle rFace)
  {
      int y2, x2, w2, h2;
  
      //adjust as neccessary
      double heightRatio = 2;
  
      y2 = Convert.ToInt32(rFace.Y - rFace.Height * (heightRatio - 1.0) / 2.0);
      h2 = Convert.ToInt32(rFace.Height * heightRatio);
      //height to width ratio is 1.25 : 1 in mugshots
      w2 = Convert.ToInt32(h2 * 4 / 5);
      x2 = Convert.ToInt32((rFace.X + rFace.Width / 2) - w2 / 2);
  
      return new Rectangle(x2, y2, w2, h2);
  }
  

  
  I just need to get rid of those false positives.

Ok make that 4 issues.

• Our camera that we will be using is currently out of commission so I don’t have a method of capturing images at the moment. Where can I find full body images of people that isn’t pure pron like google’s image search for full body images?
  Edit : “Person standing” Makes a good search 🙂

Answer 1

Ok, I figured it out but the project is on ice for the moment.
I do not have the source to paste as the VM takes forever to load up.
If someone is really interested, let me know and I’ll post.
If you see something that looks like it could be done better let me know.

The steps I took were as follows.

1. Load the image, if greater than 500 pixels in either height or width make a new image of a version scaled down to a maximum of 500px height or width. Save the scale.
2. Run the C# OpenCV implementation at Ask Ernest on all of the frontal face Harrcasscades at Harrtraining.
3. For some reason the rectangles produced by Ask Ernest need to be scaled up 1.3X
4. With the generated rectangles use standard deviation to remove false positives. I did this for each corner of the rectangle finding the distance each rectangle corner was off from the average corner location. I made a note of any rectangle who had a corner out of whack and removed it from the list of candidate rectangles.
5. Get the average rectangle from the remaining rectangles and apply “GetMugshotRectangle” from above in the question.
6. Scale the mugshot rectangle back up using the scale we saved in step 1.
7. Cut out the mugshot from the original image and save to it’s new location.

Done!