Question

I have two images(small and big). Big one contains a small one. Like if the small one is a photo and a big one is a page from the photo album.

How do I get coordinates of that small image in the big one using PHP? And also I need to know the size of that image in big one…so just a(x,y) coordinate of any angle and sizes of sides of that presentation of the small image…

(x,y, width, height)

I’ve already asked the question like that and got a brilliant answer (here) but I’ve forgot to mention over there that the size of a small image could be different from the the size of that image in the big image…

And also if it is possible to deal with a presentation of that small image in the big image can have something covering one of its angles… Like in this example:

Small image:

Big image:

Small image always has just a rectangular shape.

Answer 1

Alright, this answer does not perfectly answer the question, but it should give you a good start! I know I repeat myself in the code, but my goal was simply to get something working so you can build on it, this isn’t production code!

Preconditions

Starting with the large picture:

We need to find as best as possible the position of this other picture:

I decided to break the process into many substeps, which you could improve or remove depending on what you want the code to do.

For testing purposes, I did test my algorithm on different input images so you’ll see a variable defining what file to load…

We start with:

function microtime_float()
{
    list($usec, $sec) = explode(" ", microtime());
    return ((float)$usec + (float)$sec);
}

$time_start = microtime_float();

$largeFilename = "large.jpg";

$small = imagecreatefromjpeg("small.jpg");
$large = imagecreatefromjpeg($largeFilename);

and

imagedestroy($small);
imagedestroy($large);

$time_end = microtime_float();
echo "in " . ($time_end - $time_start) . " seconds\n";

To have a good idea on our performances. Luckily, most of the algorithm was pretty fast so I didn’t have to optimize more.

Background Detection

I started by detecting the background color. I assumed that the background color would be the color most present in the picture. To do this, I only counted how many references of each color I could find in the large picture, sort it with decending values and took the first one as the background color (should allow the code to be adaptable if you changed the source pictures)

function FindBackgroundColor($image)
{
    // assume that the color that's present the most is the background color
    $colorRefcount = array();

    $width = imagesx($image);
    $height = imagesy($image);

    for($x = 0; $x < $width; ++$x)
    {
        for($y = 0; $y < $height; ++$y)
        {
            $color = imagecolorat($image, $x, $y);
            if(isset($colorRefcount[$color]))
                $colorRefcount[$color] = $colorRefcount[$color] + 1;
            else
                $colorRefcount[$color] = 1;
        }
    }

    arsort($colorRefcount);
    reset($colorRefcount);

    return key($colorRefcount);
}
$background = FindBackgroundColor($large); // Should be white

Partitionning

My first step was to try to find all the regions where non background pixels were. With a little padding, I was able to group regions into bigger regions (so that a paragraph would be a single region instead of multiple individual letters). I started with a padding of 5 and got good enough results so I stuck with it.

This is broken into multiple function calls, so here we go:

function FindRegions($image, $backgroundColor, $padding)
{
    // Find all regions within image where colors are != backgroundColor, including a padding so that adjacent regions are merged together
    $width = imagesx($image);
    $height = imagesy($image);

    $regions = array();

    for($x = 0; $x < $width; ++$x)
    {
        for($y = 0; $y < $height; ++$y)
        {
            $color = imagecolorat($image, $x, $y);

            if($color == $backgroundColor)
            {
                continue;
            }

            if(IsInsideRegions($regions, $x, $y))
            {
                continue;
            }

            $region = ExpandRegionFrom($image, $x, $y, $backgroundColor, $padding);
            array_push($regions, $region);
        }
    }

    return $regions;
}

$regions = FindRegions($large, $background, 5);

Here, we iterate on every pixel of the picture, if its background color, we discard it, otherwise, we check if its position is already present in a region we found, if that’s the case, we skip it too. Now, if we didn’t skip the pixel, it means that it’s a colored pixel that should be part of a region, so we start ExpandRegionFrom this pixel.

The code to check if we’re inside a region is pretty simple:

function IsInsideRegions($regions, $x, $y)
{
    foreach($regions as $region)
    {
        if(($region["left"] <= $x && $region["right"] >= $x) && 
           ($region["bottom"] <= $y && $region["top"] >= $y))
        {
            return true;
        }
    }
    return false;
}

Now, the expanding code will try to grow the region in each direction and will do so as long as it found new pixels to add to the region:

function ExpandRegionFrom($image, $x, $y, $backgroundColor, $padding)
{
    $width = imagesx($image);
    $height = imagesy($image);

    $left = $x;
    $bottom = $y;
    $right = $x + 1;
    $top = $y + 1;

    $expanded = false;

    do
    {
        $expanded = false;

        $newLeft = ShouldExpandLeft($image, $backgroundColor, $left, $bottom, $top, $padding);
        if($newLeft != $left)
        {
            $left = $newLeft;
            $expanded = true;
        }

        $newRight = ShouldExpandRight($image, $backgroundColor, $right, $bottom, $top, $width, $padding);
        if($newRight != $right)
        {
            $right = $newRight;
            $expanded = true;
        }

        $newTop = ShouldExpandTop($image, $backgroundColor, $top, $left, $right, $height, $padding);
        if($newTop != $top)
        {
            $top = $newTop;
            $expanded = true;
        }

        $newBottom = ShouldExpandBottom($image, $backgroundColor, $bottom, $left, $right, $padding);
        if($newBottom != $bottom)
        {
            $bottom = $newBottom;
            $expanded = true;
        }
    }
    while($expanded == true);

    $region = array();
    $region["left"] = $left;
    $region["bottom"] = $bottom;
    $region["right"] = $right;
    $region["top"] = $top;

    return $region;
}

The ShouldExpand methods could have been written in a cleaner fashion, but I went for something fast to prototype with:

function ShouldExpandLeft($image, $background, $left, $bottom, $top, $padding)
{
    // Find the farthest pixel that is not $background starting at $left - $padding closing in to $left
    for($x = max(0, $left - $padding); $x < $left; ++$x)
    {
        for($y = $bottom; $y <= $top; ++$y)
        {
            $pixelColor = imagecolorat($image, $x, $y);

            if($pixelColor != $background) 
            {
                return $x;
            }
        }
    }

    return $left;
}

function ShouldExpandRight($image, $background, $right, $bottom, $top, $width, $padding)
{
    // Find the farthest pixel that is not $background starting at $right + $padding closing in to $right
    $from = min($width - 1, $right + $padding);
    $to = $right;
    for($x = $from; $x > $to; --$x)
    {
        for($y = $bottom; $y <= $top; ++$y)
        {
            $pixelColor = imagecolorat($image, $x, $y);

            if($pixelColor != $background) 
            {
                return $x;
            }
        }
    }

    return $right;
}

function ShouldExpandTop($image, $background, $top, $left, $right, $height, $padding)
{
    // Find the farthest pixel that is not $background starting at $top + $padding closing in to $top
    for($x = $left; $x <= $right; ++$x)
    {
        for($y = min($height - 1, $top + $padding); $y > $top; --$y)
        {
            $pixelColor = imagecolorat($image, $x, $y);

            if($pixelColor != $background)
            {
                return $y;
            }
        }
    }

    return $top;
}

function ShouldExpandBottom($image, $background, $bottom, $left, $right, $padding)
{
    // Find the farthest pixel that is not $background starting at $bottom - $padding closing in to $bottom
    for($x = $left; $x <= $right; ++$x)
    {
        for($y = max(0, $bottom - $padding); $y < $bottom; ++$y)
        {
            $pixelColor = imagecolorat($image, $x, $y);

            if($pixelColor != $background)
            {
                return $y;
            }
        }
    }

    return $bottom;
}

Now, to see if the algorithm was succesful, I added some debug code.

Debug Rendering

I created a second image to store debug info and store it on disk so I could later see my progress.

Using the following code:

$large2 = imagecreatefromjpeg($largeFilename);
$red = imagecolorallocate($large2, 255, 0, 0);
$green = imagecolorallocate($large2, 0, 255, 0);
$blue = imagecolorallocate($large2, 0, 0, 255);

function DrawRegions($image, $regions, $color)
{
    foreach($regions as $region)
    {
        imagerectangle($image, $region["left"], $region["bottom"], $region["right"], $region["top"], $color);
    }
}

DrawRegions($large2, $regions, $red);

imagejpeg($large2, "regions.jpg");

I could validate that my partitioning code was doing a decent job:

Aspect Ratio

I decided to filter out some regions based on aspect ratio (the ratio between the width and the height). Other filtering could be applied such as average pixel color or something, but the aspect ratio check was very fast so I used it.

I simply defined a “window” where regions would be kept, if their aspect ration was between a minimum and maximum value;

$smallAspectRatio = imagesx($small) / imagesy($small);

function PruneOutWrongAspectRatio($regions, $minAspectRatio, $maxAspectRatio)
{
    $result = array();
    foreach($regions as $region)
    {   
        $aspectRatio = ($region["right"] - $region["left"]) / ($region["top"] - $region["bottom"]);
        if($aspectRatio >= $minAspectRatio && $aspectRatio <= $maxAspectRatio)
        {
            array_push($result, $region);
        }
    }

    return $result;
}

$filterOnAspectRatio = true;

if($filterOnAspectRatio == true)
{
    $regions = PruneOutWrongAspectRatio($regions, $smallAspectRatio - 0.1 * $smallAspectRatio, $smallAspectRatio + 0.1 * $smallAspectRatio);
    DrawRegions($large2, $regions, $blue);
}

imagejpeg($large2, "aspectratio.jpg");

By adding the DrawRegions call, I now paint in blue the regions that are still in the list as potential positions:

As you can see, only 4 position remains!

Finding the Corners

We’re almost done! Now, what I’m doing is looking at the colors in the four corners from the small picture, and try to find the best matching pixel in the corners of the remaining regions. This code has the most potential to fail so if you have to invest time in improving the solution, this code would be a good candidate.

function FindCorners($large, $small, $regions)
{
    $result = array();

    $bottomLeftColor = imagecolorat($small, 0, 0);
    $blColors = GetColorComponents($bottomLeftColor);
    $bottomRightColor = imagecolorat($small, imagesx($small) - 1, 0);
    $brColors = GetColorComponents($bottomRightColor);
    $topLeftColor = imagecolorat($small, 0, imagesy($small) - 1);
    $tlColors = GetColorComponents($topLeftColor);
    $topRightColor = imagecolorat($small, imagesx($small) - 1, imagesy($small) - 1);
    $trColors = GetColorComponents($topRightColor);

    foreach($regions as $region)
    {
        $bottomLeft = null;
        $bottomRight = null;
        $topLeft = null;
        $topRight = null;

        $regionWidth = $region["right"] - $region["left"];
        $regionHeight = $region["top"] - $region["bottom"];

        $maxRadius = min($regionWidth, $regionHeight);

        $topLeft = RadialFindColor($large, $tlColors, $region["left"], $region["top"], 1, -1, $maxRadius);
        $topRight = RadialFindColor($large, $trColors, $region["right"], $region["top"], -1, -1, $maxRadius);
        $bottomLeft = RadialFindColor($large, $blColors, $region["left"], $region["bottom"], 1, 1, $maxRadius);
        $bottomRight = RadialFindColor($large, $brColors, $region["right"], $region["bottom"], -1, 1, $maxRadius);

        if($bottomLeft["found"] && $topRight["found"] && $topLeft["found"] && $bottomRight["found"])
        {
            $left = min($bottomLeft["x"], $topLeft["x"]);
            $right = max($bottomRight["x"], $topRight["x"]);
            $bottom = min($bottomLeft["y"], $bottomRight["y"]);
            $top = max($topLeft["y"], $topRight["y"]);
            array_push($result, array("left" => $left, "right" => $right, "bottom" => $bottom, "top" => $top));
        }
    }

    return $result;
}

$closeOnCorners = true;
if($closeOnCorners == true)
{
    $regions = FindCorners($large, $small, $regions);
    DrawRegions($large2, $regions, $green);
}

I tried to find the matching color by increasing “radially” (its basically squares) from the corners until I find a matching pixel (within a tolerance):

function GetColorComponents($color)
{
    return array("red" => $color & 0xFF, "green" => ($color >> 8) & 0xFF, "blue" => ($color >> 16) & 0xFF);
}

function GetDistance($color, $r, $g, $b)
{
    $colors = GetColorComponents($color);

    return (abs($r - $colors["red"]) + abs($g - $colors["green"]) + abs($b - $colors["blue"]));
}

function RadialFindColor($large, $color, $startx, $starty, $xIncrement, $yIncrement, $maxRadius)
{
    $result = array("x" => -1, "y" => -1, "found" => false);
    $treshold = 40;
    for($r = 1; $r <= $maxRadius; ++$r)
    {
        $closest = array("x" => -1, "y" => -1, "distance" => 1000);
        for($i = 0; $i <= $r; ++$i)
        {
            $x = $startx + $i * $xIncrement;
            $y = $starty + $r * $yIncrement;

            $pixelColor = imagecolorat($large, $x, $y);

            $distance = GetDistance($pixelColor, $color["red"], $color["green"], $color["blue"]);
            if($distance < $treshold && $distance < $closest["distance"])
            {
                $closest["x"] = $x;
                $closest["y"] = $y;
                $closest["distance"] = $distance;
                break;
            }
        }

        for($i = 0; $i < $r; ++$i)
        {   
            $x = $startx + $r * $xIncrement;
            $y = $starty + $i * $yIncrement;

            $pixelColor = imagecolorat($large, $x, $y);

            $distance = GetDistance($pixelColor, $color["red"], $color["green"], $color["blue"]);
            if($distance < $treshold && $distance < $closest["distance"])
            {
                $closest["x"] = $x;
                $closest["y"] = $y;
                $closest["distance"] = $distance;

                break;
            }
        }

        if($closest["distance"] != 1000)
        {
            $result["x"] = $closest["x"];
            $result["y"] = $closest["y"];
            $result["found"] = true;
            return $result;
        }
    }

    return $result;
}

As you can see, I’m no PHP expert, I didn’t know there was a built in function to get the rgb channels, oops!

Final Call

So now that the algorithm ran, let’s see what it found using the following code:

foreach($regions as $region)
{
    echo "Potentially between " . $region["left"] . "," . $region["bottom"] . " and " . $region["right"] . "," . $region["top"] . "\n";
}

imagejpeg($large2, "final.jpg");

imagedestroy($large2);

The output (which is pretty close to the real solution):

Potentially between 108,380 and 867,827
in 7.9796848297119 seconds

Giving this picture (the rectangle between 108,380 and 867,827 is drawn in green)

Hope this helps!