Question

I’m trying to compute a differential image velocity invariants (e.g. curl, divergence, deformation, etc) from a video using OpenCV in Python. To do that, I need to compute the spatial derivatives in the x,y directions of the optical flow. Unfortunately, OpenCV only seems to supply the APIs for computing optical flow, not its derivative.

Are there any Python libraries out there for computing spatial derivatives of optical flow? I found this SO question that was somewhat similar Lucas Kanade Optical Flow, Direction Vector, and there is code the person wrote for computing spatial derivatives, but if at all possible I’d love a library rather than writing the code myself. Any suggestions would be appreciated!

Answer 1

This is the way I see it (I’ve worked with optical flow a little bit):

You want to compute the individual partial derivatives of the optical flow field; one for the x direction, and one for the y.

I’d attempt to solve the problem like so:

• Split your flow array/matrix into two matrices: x and y flow.
• For each of those, you could go the naive route and just do a simple difference: derivative = current_state - last_state. But this approach is very messy, as the derivative will be sensitive to the slightest bit of error.
• To counter that, you could approximate one chunk of your data points (maybe a whole row?) with a regression curve that is easily differentiable, like a polynomial.

The just differentiate that approximated curve and you’re good to go.

You could also just smooth individual matrices and do a naive difference, which should be much faster than approximating data points, but should be more tolerant to error.