Question

I’m trying to use cv::calcOpticalFlowPyrLK but sometimes an internal assertion in that function fails. The assertion is npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0. I’m using OpenCV 2.3.1. The source code for this function is available here.

It’s hard to wrap my head around their code, especially because of my limited experience with computer graphics and their lack of comments. Why is this assertion being triggered and what does it say about my problem?

Edit: I call the function as follows:

cv::calcOpticalFlowPyrLK(curBwFrame, prvFrame, features, newFeatures, trackingStatus, errors);

I found out that the features vector, which was obtained by calling cv::goodFeaturesToTrack(curBwFrame, features, 5, 0.2, 0.5, skinMask); with a non-empty mask that appears to be sufficiently big and a valid image, doesn’t contain any features. How can this happen?

curBwFrame

skinMask

I’m able to reproduce the problem using the following code snippet:

#include <vector>
#include <cassert>
#include <opencv2\opencv.hpp>
using std::vector;
using namespace cv;

int main() {
    vector<Point2f> features;
    cv::Mat curBwFrame = imread("curBwFrame.png");
    cv::cvtColor(curBwFrame, curBwFrame, CV_RGB2GRAY);
    imwrite("test.png", curBwFrame);

    cv::Mat skinMask = imread("skinMask.png");
    cv::cvtColor(skinMask, skinMask, CV_RGB2GRAY);
    imwrite("test.png", skinMask);

    cv::goodFeaturesToTrack(curBwFrame, features, 5, 0.2, 0.5, skinMask);
    assert(features.size() > 0);

    return 0;
}

Answer 1

The main problem are your parameters.In the OpenCV 2.3.2 documentation (no compatibility change between 2.3.1) this is the description of the method parameters:

void goodFeaturesToTrack(InputArray image, OutputArray corners, int maxCorners, double qualityLevel, double minDistance, InputArray mask=noArray(), int blockSize=3, bool useHarrisDetector=false, double k=0.04 )

Parameters:

• image – Input 8-bit or floating-point 32-bit, single-channel image.
• corners – Output vector of detected corners.
• maxCorners – Maximum number of corners to return. If there are more
  corners than are found, the strongest of them is returned.
• qualityLevel – Parameter characterizing the minimal accepted quality
  of image corners. The parameter value is multiplied by the best
  corner quality measure, which is the minimal eigenvalue (see
  cornerMinEigenVal() ) or the Harris function response (see
  cornerHarris() ). The corners with the quality measure less than the
  product are rejected. For example, if the best corner has the quality
  measure = 1500, and the qualityLevel=0.01 , then all the corners with
  the quality measure less than 15 are rejected.
• minDistance – Minimum possible Euclidean distance between the
  returned corners.
• mask – Optional region of interest. If the image is not empty (it
  needs to have the type CV_8UC1 and the same size as image ), it
  specifies the region in which the corners are detected.
• blockSize – Size of an average block for computing a derivative
  covariation matrix over each pixel neighborhood. See
  cornerEigenValsAndVecs() .
• useHarrisDetector – Parameter indicating whether to use a Harris
  detector (see cornerHarris()) or cornerMinEigenVal().
• k – Free parameter of the Harris detector.

I recommend you to play a little with qualityLevel and minDistance to suffice your needs.