Question

Hi have already function solution but one issue:

            // The screenshot will be stored in this bitmap.
            Bitmap capture = new Bitmap(rec.Width, rec.Height, PixelFormat.Format24bppRgb);
            using (Graphics g = Graphics.FromImage(capture))
            {
                g.CopyFromScreen(rec.Location, new System.Drawing.Point(0, 0), rec.Size);
            }

            MCvSURFParams surfParam = new MCvSURFParams(500, false);
            SURFDetector surfDetector = new SURFDetector(surfParam);

            // Template image 
            Image<Gray, Byte> modelImage = new Image<Gray, byte>("template.jpg");
            // Extract features from the object image
            ImageFeature[] modelFeatures = surfDetector.DetectFeatures(modelImage, null);

            // Prepare current frame
            Image<Gray, Byte> observedImage = new Image<Gray, byte>(capture);
            ImageFeature[] imageFeatures = surfDetector.DetectFeatures(observedImage, null);


            // Create a SURF Tracker using k-d Tree
            Features2DTracker tracker = new Features2DTracker(modelFeatures);

            Features2DTracker.MatchedImageFeature[] matchedFeatures = tracker.MatchFeature(imageFeatures, 2);
            matchedFeatures = Features2DTracker.VoteForUniqueness(matchedFeatures, 0.8);
            matchedFeatures = Features2DTracker.VoteForSizeAndOrientation(matchedFeatures, 1.5, 20);
            HomographyMatrix homography = Features2DTracker.GetHomographyMatrixFromMatchedFeatures(matchedFeatures);

            // Merge the object image and the observed image into one image for display
            Image<Gray, Byte> res = modelImage.ConcateVertical(observedImage);

            #region draw lines between the matched features

            foreach (Features2DTracker.MatchedImageFeature matchedFeature in matchedFeatures)
            {
                PointF p = matchedFeature.ObservedFeature.KeyPoint.Point;
                p.Y += modelImage.Height;
                res.Draw(new LineSegment2DF(matchedFeature.SimilarFeatures[0].Feature.KeyPoint.Point, p), new Gray(0), 1);
            }

            #endregion

            #region draw the project region on the image

            if (homography != null)
            {
                // draw a rectangle along the projected model
                Rectangle rect = modelImage.ROI;
                PointF[] pts = new PointF[] { 
                    new PointF(rect.Left, rect.Bottom),
                    new PointF(rect.Right, rect.Bottom),
                    new PointF(rect.Right, rect.Top),
                    new PointF(rect.Left, rect.Top)
                };

                homography.ProjectPoints(pts);

                for (int i = 0; i < pts.Length; i++)
                    pts[i].Y += modelImage.Height;

                res.DrawPolyline(Array.ConvertAll<PointF, Point>(pts, Point.Round), true, new Gray(255.0), 2);
            }

            #endregion

            pictureBoxScreen.Image = res.ToBitmap();

the result is:

And my problem is that, function homography.ProjectPoints(pts);
Get only first occurrence of pattern (white rectangle in pic above)

How i can Project all occurrence of template, respectively how I can get occurrence of template rectangle in image

Answer 1

I face a problem similar to yours in my master thesis. Basically you have two options:

1. Use a clustering such as Hierarchical k-means or a point density one such as DBSCAN (it depends on two parameters but you can make it threshold free in bidimensional R^2 space)
2. Use a multiple robust model fitting estimation techniques such as JLinkage. In this more advanced technique you clusters points that share an homography instead of cluster points that close to each other in euclidean space.

Once you partition your matches in “clusters” you can estimate homographies between matches belonging to correspondant clusters.