Question

I’m using calcOpticalFlowPyrLK to detect points in an 300×400 image. I’m feeding this data to findHomography and warpPerspective and doing video stabilization. On an iPhone it’s currently doing this at 500ms and I was wondering if I could bring this down.

TermCriteria termcrit(CV_TERMCRIT_ITER|CV_TERMCRIT_EPS,20,0.03);
cv::Size winSize(31,31);
[..]
calcOpticalFlowPyrLK(baseGray, gray, points[0], points[1], status, err, winSize, 3, termcrit, 0, 0.001);

What would be good parameters (eg win size) while still doing a good job for video stabilization?

Answer 1

For time consuming algorithm in Image Processing, I usually first try to reduce my image resolution (let s say by a 2 a factor) and check if I still get the same kind of results.

The optical flow is well known to be a high consuming technique

cvResize could be you solution

In a lot of cases, the results of your algorithm will be roughly the same, and in your case you might get as low as 100 ms.

Concerning the winsize, the documentation says :

winSize – Size of the search window at each pyramid level.

This is a search window, so you want to reduce it to get faster. Try with a 15 pixels window.

EDIT :

Most of the time, I try to create several snippets of different resolutions for each input image I get.
This way, depending on the action to be performed, I can choose the resolution wisely, and increase my performance .

A concrete example :
I get a 640*480 input image.
I create a 320*240 version of this image.
For some reason, I have to calculate the optical flow on it, which I know is consuming. I will use the lower resolution version to calculate my coefficients.
As for histogram equalization, which is fact, I will choose the full resolution image as input to get a maximum of data.

Just avoid to calculate things with data from different resolution at the same time and you shoudn’t have problems.