Question

I am developing an application that sends a stream of JPEG images (MJPEG stream) to a remote PC. I am having trouble changing the preview frame rate in the camera parameters. It seems that no matter what rate I set, the camera only gives me a frame at 15 fps. At first I thought it was due to image compression and then transmitting the data. But I created a debug instance where I store the first compressed JPEG image and send that image every time the onPreviewFrame function is called. That results in a reliable 15fps, but I have it set at 30fps. Anyone have any ideas of why the fps is so inconsistent? When I set it up to compress every preview frame and send out over wifi I get a rate of 5-40 FPS and it jumps all over the place.

Here is of my code for my Preview (sets up camera in full screen)

public class Preview extends SurfaceView implements PreviewCallback, SurfaceHolder.Callback {
SurfaceHolder mHolder;
Camera mCamera;
DatagramSocket udpSocket = null;
final int PORT = 1235;
final String IPAddress = "192.168.1.101";
int packetSize = 1024; // Specify max size of each UDP packet
int WIDTH = 640;
int HEIGHT = 480;   
int FPS = 30;       
int quality = 70;
/**
 * Initialize the Camera Preview Holder and create a udpSocket.
 * @param context
 */
Preview(Context context) {
    super(context);
    mHolder = getHolder();
    mHolder.addCallback(this);
    mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    try {
        udpSocket = new DatagramSocket();
    } catch (SocketException e) {
        Log.e(tag, "Error creating UDP Socket!");
    }
}
public void surfaceCreated(SurfaceHolder holder) {
    mCamera = Camera.open();
    try {
       mCamera.setPreviewDisplay(holder);
    } catch (IOException exception) {
        mCamera.release();
        mCamera = null;
        Log.e(tag, "Error settign Camera Preview Holder!");
    }
}
public void surfaceDestroyed(SurfaceHolder holder) {
    mCamera.setPreviewCallback(null);
    mCamera.stopPreview();
    udpSocket.close();
    mCamera.release();
    udpSocket = null;
    mCamera = null;
}
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
    Camera.Parameters parameters = mCamera.getParameters();
    parameters.setPreviewSize(WIDTH, HEIGHT);
    parameters.setPreviewFrameRate(FPS);
    parameters.setRotation(90);

    mCamera.setParameters(parameters);
    mCamera.setPreviewCallback(this);
    mCamera.startPreview();
}
/** Compresses the preview framer (data) into a JPEG image and send to a host at
 * the specified IPAddress and Port number for playback. Only sends frame if sendFrame
 * is true.
 * (non-Javadoc)
 * @see android.hardware.Camera.PreviewCallback#onPreviewFrame(byte[], android.hardware.Camera)
 * @see android.hardware.Camera
 */
public void onPreviewFrame(byte[] data, Camera camera) {
    final YuvImage imgPreview = new YuvImage(data, ImageFormat.NV21, WIDTH, HEIGHT, null);
    byte[] buffer;
    ByteArrayOutputStream jpegOutStream = new ByteArrayOutputStream();

    int offset = 0;
    int TotalLength, lengthLeft;
    DatagramPacket dgpout;
    // Compress image into JPEG
    imgPreview.compressToJpeg(new Rect(0, 0, imgPreview.getWidth(), imgPreview.getHeight()), quality, jpegOutStream);
    buffer = jpegOutStream.toByteArray();
    TotalLength = lengthLeft = buffer.length;
    // Send frame out. Split into packets of desired length.
    try {
        while(lengthLeft>packetSize){
            dgpout= new DatagramPacket(buffer, offset, packetSize);
            offset+=packetSize;
            lengthLeft-=packetSize;
            dgpout.setAddress(InetAddress.getByName(IPAddress));
            dgpout.setPort(PORT);
            udpSocket.send(dgpout);
        }
        if(lengthLeft>0){
            dgpout= new DatagramPacket(buffer,offset,lengthLeft);
            offset+=lengthLeft;
            lengthLeft-=lengthLeft;
            dgpout.setAddress(InetAddress.getByName(IPAddress));
            dgpout.setPort(PORT);
            udpSocket.send(dgpout);
        }
        Log.i(tag, "Sent Successfully: frame size="+TotalLength +"Buffer Size: "+udpSocket.getSendBufferSize());
    } catch (UnknownHostException e1) {
        Log.e(tag   , "UnknownHostException. Sending failed.");
    } catch (IOException e) {
        Log.e(tag   , "IOException. Sending failed.");
    }
}

}

And here is my debug sample for only sending the same JPEG over and over

if(sendFrame){
        sendFrame = false;
        final YuvImage imgPreview = new YuvImage(data, ImageFormat.NV21, WIDTH, HEIGHT, null);

        fixImage = data;
                byte[] buffer;
                ByteArrayOutputStream jpegOutStream = new ByteArrayOutputStream();

                int offset = 0;
                int TotalLength, lengthLeft;
                DatagramPacket dgpout;
                // Compress image into JPEG
                imgPreview.compressToJpeg(new Rect(0, 0, imgPreview.getWidth(), imgPreview.getHeight()), quality, jpegOutStream);
                buffer = jpegOutStream.toByteArray();

                fixJPEG = buffer;
                fixTotalLength = buffer.length;

    }
    int lengthLeft = 0;
    int offset = 0;
    DatagramPacket dgpout;
    if(fixImage==null || fixJPEG==null)
        return;
    /* Compress Optional */
    if(true){
        final YuvImage imgPreview = new YuvImage(fixImage, ImageFormat.NV21, WIDTH, HEIGHT, null);
        ByteArrayOutputStream jpegOutStream = new ByteArrayOutputStream();
        imgPreview.compressToJpeg(new Rect(0, 0, imgPreview.getWidth(), imgPreview.getHeight()), quality, jpegOutStream);
        fixJPEG = jpegOutStream.toByteArray();
    }
    fixTotalLength = lengthLeft = fixJPEG.length;
    // Send frame out. Split into packets of desired length.
    try {
        while(lengthLeft>packetSize){
            dgpout= new DatagramPacket(fixJPEG, offset, packetSize);
            offset+=packetSize;
            lengthLeft-=packetSize;
            dgpout.setAddress(InetAddress.getByName(IPAddress));
            dgpout.setPort(PORT);
            udpSocket.send(dgpout);
        }
        if(lengthLeft>0){
        dgpout= new DatagramPacket(fixJPEG,offset,lengthLeft);
            offset+=lengthLeft;
            lengthLeft-=lengthLeft;
            dgpout.setAddress(InetAddress.getByName(IPAddress));
            dgpout.setPort(PORT);
            udpSocket.send(dgpout);
        }
        Log.i(tag, "Sent Successfully: frame size="+fixTotalLength +"Buffer Size: "+udpSocket.getSendBufferSize());
    } catch (UnknownHostException e1) {
        Log.e(tag   , "UnknownHostException. Sending failed.");
    } catch (IOException e) {
        Log.e(tag   , "IOException. Sending failed.");
    }

Does anyone have any ideas for improving efficiency? Does this happen on all devices? I am running Android 2.2. Any ideas are appreciated.

*EDIT Added entire Preview class

Answer 1

It’s quite possible the Atrix is (legitimately) ignoring your request to change the preview frame rate.

In your surfaceChanged callback, try invoking the following and inspecting the value in the debugger:

List<Integer> rates = parameters.getSupportedPreviewFrameRates();

The Android documentation says:

Returns a list of supported preview frame rates. null if preview frame
rate setting is not supported.

Additionally, the documents for setPreviewFrameRate() says:

This is the target frame rate. The actual frame rate depends on the
driver.

So, it could be that the Atrix doesn’t support anything other than 15 fps for preview.

In any event, the proper way to use the preview frame rate is to first call getSupportedPreviewFrameRates(). If it returns a non-null result, find the closest number on the list to what you want and pass that to setPreviewFrameRate(). Otherwise you’ll have to live with whatever rate the device supports.