Question

UPDATE
– I have fixed some mistakes in the code below and the images are displayed on the other device, but I have another problem. While video capture is open, the “master” device sends data continuously, sometimes this capture appears on “slave” device and in a very short time, the image “blinks” to blank and repeat this all time for a short period. Any idea about this?

I’m working on a app that’s need to send live camera capture and live microphone capture to another device in network.

I have done the connection between devices using a TCP server and publish it with bonjour, this works like a charm.

The most important part is about to send and receive video and audio from “master” device and render it on “slave” device.

First, here a piece of code where the app get the camera sample buffer and transform in UIImage:

@implementation AVCaptureManager (AVCaptureVideoDataOutputSampleBufferDelegate)

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
  dispatch_sync(dispatch_get_main_queue(), ^{
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];

    UIImage *image = [self imageFromSampleBuffer:sampleBuffer];
    NSData *data = UIImageJPEGRepresentation(image, 0.2);

    [self.delegate didReceivedImage:image];
    [self.delegate didReceivedFrame:data];

    [pool drain];
  });
}


- (UIImage *) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer
{
  CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

  CVPixelBufferLockBaseAddress(imageBuffer, 0);

  size_t width = CVPixelBufferGetWidth(imageBuffer);
  size_t height = CVPixelBufferGetHeight(imageBuffer);
  size_t bytesPerRow = width * 4;

  CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();

  void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);

  CGContextRef context = CGBitmapContextCreate(
                                               baseAddress,
                                               width,
                                               height,
                                               8,
                                               bytesPerRow,
                                               colorSpace,
                                               kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder32Little
                                               );

  CGImageRef quartzImage = CGBitmapContextCreateImage(context);

  UIImage *image = [UIImage imageWithCGImage:quartzImage];
  CGImageRelease(quartzImage);
  CGColorSpaceRelease(colorSpace);
  CGContextRelease(context);
  CVPixelBufferUnlockBaseAddress(imageBuffer, 0);

  return image;
}

@end

The message “[self.delegate didReceivedImage:image];” is just to test the image capture on master device, and this image works on capture device.

The next is about how to I send it to network:

- (void) sendData:(NSData *)data
{
  if(_outputStream && [_outputStream hasSpaceAvailable])
  {
    NSInteger bytesWritten = [_outputStream write:[data bytes] maxLength:[data length]];

    if(bytesWritten < 0)
      NSLog(@"[ APP ] Failed to write message");

  }
}

Look I’m using RunLoop to write and read streams, I think this is better than open and closes streams constantly.

Next, I receive the “NSStreamEventHasBytesAvailable” event on the slave device, the piece of code where handle this is:

case NSStreamEventHasBytesAvailable:
      /*I can't to start a case without a expression, why not?*/
      NSLog(@"[ APP ] stream handleEvent NSStreamEventHasBytesAvailable");
      NSUInteger bytesRead;
      uint8_t buffer[BUFFER_SIZE];

      while ([_inputStream hasBytesAvailable])
      {
        bytesRead = [_inputStream read:buffer maxLength:BUFFER_SIZE];
        NSLog(@"[ APP ] bytes read:  %i", bytesRead);

        if(bytesRead)
          [data appendBytes:(const void *)buffer length:sizeof(buffer)];
      }


      [_client writeImageWithData:data];

      break;

The value of BUFFER_SIZE is 32768.
I think the while block is not necessary, but I use it because if I can’t read all available bytes at first iteration, I can read in the next.

So, this is the point, the stream comes correctly but the image serialized on NSData seems be corrupted, in the next, I just send data to client…

[_client writeImageWithData:data];

… and create a UIImage with data in client class simple like this…

[camPreview setImage:[UIImage imageWithData:data]];

In the camPreview (yes is a UIImageView), I have a image just to display the placeholder on the screen, when I get the imagem from network and pass to camPreview, the placeholder gets blank.

Other think is about the output, when I start the capture, first parts where I receive data, I get this message from system:

Error: ImageIO: JPEG Corrupt JPEG data: 28 extraneous bytes before marker 0xbf

Error: ImageIO: JPEG Unsupported marker type 0xbf

After some little time, I get this messages anymore.

The point is find the cause of the image not are displayed on the “slave” device.

Thanks.

Answer 1

I am not sure how often you are sending images, but even if it is not very often I think I would scan for the SOI and EOI markers in the JPEG data to insure you have all the data. Here is a post I quickly found