Question

I have a planar marker where I have run the SIFT algorithm to extract features. Then I run a detector to find this marker in the scene and extract features again. I match the points and extract the homography from the matched pairs with OpenCV using findHomography().

Now I want to project the 2D points detected in the marker with the computed homography to compare the positions with the 3D points measured from the scene and calculate the reprojection error. I am confused with the pixel coordinates, centimeters, calibration matrices, I don’t know which conversions should I do first.

Does anybody know a link on this or can explain the method?

Answer 1

If you call the homography matrix H, the camera matrix K (needed to convert to pixels) would be something like this, depending on your resolution.

Mat K= Mat::zeros(3,3,CV_32FC1);
K.at<float>(0,0)=500.0f;        
K.at<float>(0,2)=320.0f;      // width/2    
K.at<float>(1,1)=500.0f;    
K.at<float>(1,2)=240.0f;      // height/2 
K.at<float>(2,2)=1.0f;

If your marker points are vector points of 2D:

vector<Point2f> marker_point; //containing coordinates in centimeters

then the projection would be like this, with the result a 3D point in pixel coordinates.

Mat point(3,1,CV_32FC1);        
point.at<float>(0) = marker_point.x;
point.at<float>(1) = marker_point.y;
point.at<float>(2) = 1.0f;
point = H* point;
point = point/point.at<float>(2);       //Normalize
point = K* point;                   //to pixels