Question

I am using AVAssetWriter to save the live feed from the camera. This works well using this code

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer  fromConnection:(AVCaptureConnection *)connection{ 

 CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
 CMTime lastSampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);

 if(videoWriter.status != AVAssetWriterStatusWriting){
    [videoWriter startWriting];
    [videoWriter startSessionAtSourceTime:lastSampleTime];
 }

 if(adaptor.assetWriterInput.readyForMoreMediaData) [adaptor appendPixelBuffer:imageBuffer withPresentationTime:lastSampleTime];
 else NSLog(@"adaptor not ready",);
}

I am usually getting close to 30 fps (however not 60 fps on iPhone 4s as noted by others) and when timing [adaptor appendPixelBuffer] it only takes a few ms.

However, I don’t need the full frame, but I need high quality (low compression, key frame every frame) and I am going to read it back a process several times later. I therefore would like to crop the image before writing. Fortunately I only need a strip in the middle so I can do a simple memcpy of the buffer. To do this I am creating a CVPixelBufferRef that I am copying into and writing with the adaptor:

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer  fromConnection:(AVCaptureConnection *)connection{ 

 CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
 CMTime lastSampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);

 if(videoWriter.status != AVAssetWriterStatusWriting){
    [videoWriter startWriting];
    [videoWriter startSessionAtSourceTime:lastSampleTime];
 }

 CVPixelBufferLockBaseAddress(imageBuffer,0);
 size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
 size_t width = CVPixelBufferGetWidth(imageBuffer);
 size_t height = CVPixelBufferGetHeight(imageBuffer);
 void * buffIn = CVPixelBufferGetBaseAddress(imageBuffer);

 CVPixelBufferRef pxbuffer = NULL;
 CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, width, height, kCVPixelFormatType_32BGRA, nil, &pxbuffer);

 NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);
 CVPixelBufferLockBaseAddress(pxbuffer, 0);

  void *buffOut = CVPixelBufferGetBaseAddress(pxbuffer);
  NSParameterAssert(buffOut != NULL);

  //Copy the whole buffer while testing
  memcpy(buffOut, buffIn, width * height * 4); 
  //memcpy(buffOut, buffIn+sidecrop, width * 100 * 4); 

  if (adaptor.assetWriterInput.readyForMoreMediaData) [adaptor appendPixelBuffer:pxbuffer withPresentationTime:lastSampleTime];
  else NSLog(@"adaptor not ready");

   CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
   CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
}

This also works and the video looks OK. However it is very slow and the frame rate becomes unacceptable. And strangely, the big slowdown isn’t the copying but that the [adaptor appendPixelBuffer] step now takes 10-100 times longer than before. So I guess that it doesn’t like the pxbuffer I create, but I can see why. I am using kCVPixelFormatType_32BGRA when setting up both the video out and the adaptor.

Can anyone suggest a better way to do the copying/cropping? Can you do that directly on the ImageBuffer?

Answer 1

I found a solution. In iOS5 (I had missed the updates) you can set AVAssetWriter to crop your video (as also noted by Steve). Set AVVideoScalingModeKey to AVVideoScalingModeResizeAspectFill

videoWriter = [[AVAssetWriter alloc] initWithURL:filmurl 
                                        fileType:AVFileTypeQuickTimeMovie 
                                           error:&error];  
NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys: 
     AVVideoCodecH264, AVVideoCodecKey, 
     [NSNumber numberWithInt:1280], AVVideoWidthKey,  
     [NSNumber numberWithInt:200], AVVideoHeightKey,
     AVVideoScalingModeResizeAspectFill, AVVideoScalingModeKey,// This turns the
                                                               // scale into a crop
     nil]; 
videoWriterInput = [[AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo 
                                                       outputSettings:videoSettings] retain];