Question

For college, we have been given an assignment where, given an image, we have to identify the “figures”, their color, and the amount of “pixel-groups” inside them. Let me explain:

The image above has one figure (in the image there can be multiple figures, but let us forget about that for now).

• The background color of the canvas is the pixel at 0,0 (in this case, yellow)
• The border color of the figure is black (it can be any color other than the canvas’ background color).
• The figure’s background color is white (it can also be the same as the canvas’ background color).
• A figure can only have one background color.
• There are two pixel groups in the figure. One is a pool of blue pixels, and the other is a pool of red with some green inside. As you can see, it doesn’t matter the color of the pixel group’s pixels (it is just different than the figure’s background color). What matters is the fact that they’re in contact (even diagonally). So despite having two different colors, such group is considered as just one anyway.
• As you can see, the border can be as irregular as you wish. It only has, however, one color.
• It is known that a pixel group will not touch the border.
• I was told that a pixel group’s colors can be any except the figure’s background color. I assume that then it can be the same as the figure’s border color (black).

We have been given a class capable of taking images and converting them to a matrix (each element being an integer representing the color of the pixel).

And that’s it. I’m doing it with Java.

WHAT HAVE I DONE SO FAR

• Iterate through each pixel in the matrix
• If I find a pixel that is different from the background color, I will assume it belongs to the border of the figure. I will call this pixel initialPixel from now on.
• Note that the initialPixel in the image I provided is that black pixel in the top-left corner of the figure. I made a sharp cut there purposefully to illustrate it.
• My mission now is to find the background color of the figure (in this case white).

But I’m having quite a great deal of trouble to find such background color (white). This is the closest method I did, which worked for some cases – but not with this image:

• Since I know the color of the border, I could find the first different color that is to the south of the initialPixel. Did sound like a good idea – it did work sometimes, but it would not work with the image provided: it will return yellow in this case, since initialPixel is quite away from the figure’s contents.

Assuming I did find the figure’s background color (white), my next task would be to realize that there exist two pixel groups within the figure. This one seems easier:

• Since I now know the figure’s background color (white), I can try iterating through each pixel within the figure and, if I find one that does not belong to the border and is not part of the figure’s background, I can already tell there is one pixel group. I can begin a recursive function to find all pixels related to such group and “flag” them so that in the future iterations I can completely ignore such pixels.

WHAT I NEED

Yes, my problem is about how to find the figure’s background color (keep in mind it can be the same as the whole image’s background color – for now it is yellow, but it can be white as well) based on what I described before.

I don’t need any code – I’m just having trouble thinking a proper algorithm for such. The fact that the border can have such weird irregular lines is killing me.

Or even better: have I been doing it wrong all along? Maybe I shouldn’t have focused so much on that initialPixel at all. Maybe a different kind of initial method would have worked? Are there any documents/examples about topics like this? I realize there is a lot of research on “computer vision” and such, but I can’t find much about this particular problem.

SOME CODE

My function to retrieve a vector with all the figures:
*Note: Figure is just a class that contains some values like the background color and the number of elements.

public Figure[] getFiguresFromImage(Image image) {
    Figure[] tempFigures = new Figure[100];
    int numberOfFigures = 0;
    matrixOfImage = image.getMatrix();
    int imageBackgroundColor = matrixOfImage[0][0];
    int pixel = 0;

    for (int y = 0; y < matrixOfImage.length; ++y) {
        for (int x = 0; x < matrixOfImage[0].length; ++x) {
            pixel = matrixOfImage[y][x];
            if (!exploredPixels[y][x]) {
                // This pixel has not been evaluated yet
                if (pixel != imageBackgroundColor ) {
                    // This pixel is different than the background color
                    // Since it is a new pixel, I assume it is the initial pixel of a new figure
                    // Get the figure based on the initial pixel found
                    tempFigures[numberOfFigures] = retrieveFigure(y,x);
                    ++numberOfFigures;
                }
            }
        }   
    }

    // ** Do some work here after getting my figures **

    return null;
}

Then, clearly, the function retrieveFigure(y,x) is what I am being unable to do.

Notes:

• For learning purposes, I should not be using any external libraries.

Answer 1

A good way to solve this problem is to treat the image as a graph, where there is one node (‘component’ in this answer) for each color filled area.

Here is one way to implement this approach:

1. Mark all pixels as unvisited.

2. For each pixel, if the pixel is unvisited, perform the flood fill algorithm on it. During the flood fill mark each connected pixel as visited.

   Now you should have a list of solid color areas in your image (or ‘components’), so you just have to figure out how they are connected to each other:

3. Find the component that has pixels adjacent to the background color component – this is your figure border. Note that you can find the background color component by finding the component with the 0,0 pixel.

4. Now find the components with pixels adjacent to the newly found ‘figure border’ component. There will be two such components – pick the one that isn’t the background (ie that doesn’t have the 0,0 pixel). This is your figure background.

5. To find the pixel groups, simply count the number of components with pixels adjacent to the figure background component (ignoring of course the figure border component)

Advantages of this approach:

• runs in O(# pixels) time.
• easy to understand and implement.
• doesn’t assume the background color and figure background color are different.

---

To make sure you understand how iterating through the components and their neighbors might work, here’s an example pseudocode implementation for step 5:

List<Component> allComponents; // created in step 2
Component background; // found in step 3 (this is the component with the 0,0 pixel)
Component figureBorder; // found in step 4
List<Component> pixelGroups = new List<Component>(); // list of pixel groups

for each Component c in allComponents:
    if c == background:
        continue;
    for each Pixel pixel in c.pixelList:
        for each Pixel neighbor in pixel.neighbors:
            if neighbor.getComponent() == figureBorder:
                c.isPixelGroup = true;

int numPixelGroups = 0;
for each Component c in allComponents:
    if (c.isPixelGroup)
        numPixelGroups++;