I’m working on an augmented reality app for iPhone that involves a very processor-intensive object recognition algorithm (pushing the CPU at 100% it can get through maybe 5 frames per second), and in an effort to both save battery power and make the whole thing less “jittery” I’m trying to come up with a way to only run that object recognizer when the user is actually moving the camera around.
My first thought was to simply use the iPhone’s accelerometers / gyroscope, but in testing I found that very often people would move the iPhone at a consistent enough attitude and velocity that there wouldn’t be any way to tell that it was still in motion.
So that left the option of analyzing the actual video feed and detecting movement in that. I got OpenCV working and tried running their pyramidal Lucas-Kanade optical flow algorithm, which works well but seems to be almost as processor-intensive as my object recognizer – I can get it to an acceptable framerate if I lower the depth levels / downsample the image / track fewer points, but then accuracy suffers and it starts to miss some large movements and trigger on small hand-shaking-y ones.
So my question is, is there another optical flow algorithm that’s faster than Lucas-Kanade if I just want to detect the overall magnitude of camera movement? I don’t need to track individual objects, I don’t even need to know which direction the camera is moving, all I really need is a way to feed something two frames of video and have it tell me how far apart they are.
Why not use the combo of the accelerometer/gyro motion sensing, and a very low res image tracker? Each method seems to be confused by completely different user motions.