Question

I’m using calcOpticalFlowPyrLK, findHomography, warpPerspective to do video stabilizing.

The result is stabilized, which is great, but it’s also distorted. On top/bottom you can see the edges as warpPerspective did the transformation, sometimes the middle is crushed.

I understand this is part of what it does, but I was wondering what can I do to eliminate as much of these ugly distortions?

Worst case scenario can I tell it to only convert on 2D similar to Phase Correlation?

Update: link to an example image: https://i.stack.imgur.com/Tg1IK.png

Update2, the code:

calcOpticalFlowPyrLK(baseGray, gray, points[0], points[1], status, err, winSize, 3, termcrit, 0, 0.001);
lastHomography = findHomography(points[0], points[1], CV_RANSAC, 3); 
warpPerspective(image, newImage, lastHomography, image.size(), WARP_INVERSE_MAP, BORDER_TRANSPARENT);

Answer 1

The artifacts in your provided image look like they are the result of nearest neighbor interpolation.

The default interpolation method according to the documentation should be a linear interpolation. But by using the WARP_INVERSE_MAP flag without explicitly specifying an interpolation method opencv seems to use the nearest neighbor method instead.

So the solution to you problem is to also explicitly specify a interpolation method.

warpPerspective(image, newImage, lastHomography, image.size(), INTER_LINEAR | WARP_INVERSE_MAP, BORDER_TRANSPARENT);