Question

I’m doing research into AR on the iPhone and am trying to figure out how people are getting each frame of video? I’m wanting to figure out AR using computer vision( OpenCV ). So basically I will have a pattern on a piece of paper that I will find using OpenCV and place a graphic on top of the pattern.

I know about the movie class UIImagePickerController, but am unsure how you would go about getting to each frame.

Can someone point me in the right direction?

Answer 1

UIImagePickerController is the means for displaying a camera view and taking single pictures with a camera-like front end. It’s not what you’re looking for.

Instead you need to look into AVFoundation, particularly the classes surrounding AVCaptureSession. You’ll want to acquire a meaningful AVCaptureDevice (which can be the front or back camera on the iPhone 4 and current iPod Touch), create an AVCaptureDeviceInput that references it and add that as an input to an AVCaptureSession. Then just create an AVCaptureVideoDataOutput and set it up with a meaningful delegate and a Grand Central Dispatch dispatch queue.

When you start the session going, you’ll receive delegate callbacks on the queue you created providing CMSampleBufferRefs, from which you can pull a CVImageBufferRef and hence the pixel data.