Question

I have a segmentated image, looks like this:

// I am not allowed to post pictures since I am a new member, so only the link:

// Turns out I cannot post two hyperlinks either, so I am only going to post the link of the map file.

Edit: I believe now I can post images:

alt text http://img4.imageshack.us/img4/917/test0seg.jpg

I also have a map file that clearly shows the segments:

alt text http://img4.imageshack.us/img4/6904/test0map.jpg

Now, what I need to do is to create a binary image file that consists of just the deer in the center painted white, rest of the image painted black.

What methods do you suggest for merging segments?

I have something like this in mind:

1. Calculate color averages for each segment.
2. Compare them and merge the most similar segments.

If I do that, I end up with 3 segments: Floor (white part), wall (black and light grey part combined) and the object (grey part).

At this points what can be done to correctly obtain the object?

Note that object does not have to be in the center, it may even be partly off-screen.

(I also thought about calculating the area each segment takes and labeling the smallest area as the object; but there may be times when objects cover most of the image, so it may not produce correct results.)

I’d really appreciate any help. Thanks in advance.

Answer 1

This is a somewhat difficult question, since “the object” is a subjective term. Clearly, you want the most interesting object, so we just have to decide what an interesting object looks like. This will have to be something statistical.

Let’s assume that, like in your images, your object of interest is one of a small number of segments. We will just compute a score for every segment, and call the highest-scoring one the object.

I would just play around with adding together various score functions. Some good ones might be:

• distance or squared distance of a segment’s center to the exact center of the image (this would find your example object.)
• number of pixels of the segment that are on the border of the image (this would find your example object, since the bad clusters have large borders)
• number of Canny edges inside the segment divided by number of pixels in the segment, if you think your object will be “more interesting” than the background
• number of SIFT keypoints divided by number of pixels, same rationale as previous
• break RGB space into a relatively small number of bins, like 512 (so, break each of R,G,B into 8 rough levels, like 0-31, 32-63, etc. if you are working with 8-bit images) and look at properties of each segment considered as a distribution over this space. An interesting object might have a higher-entropy distribution, or a lower-entropy one, depending on your context.