Question

I am using Opencv for some real-time video processing.

As a front-end I am using QT framework.

On my GUI, I have an input image window (mapped to a Label) and an output image window (mapped to another Label) and 3 push buttons. One to Start input video capture, the second to process the video (code not written yet), and third to Exit.

I am currently able to stream my video and display it on the Front-end. But this locks my GUI and am unable to Exit.

I tried using QTimers (using suggestions from this and the QT forum), but my GUI still remains locked.

Would appreciate if someone can point my in the right direction.

Below is the code:

mainwindow.h

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>   // for cvtColor
#include <iostream>
#include <QTimer>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();

private slots:
    void on_buttonCaptureVideo_clicked();

    void on_buttonExit_clicked();

public slots:
    virtual void doNextFrame() {repaint();}

private:
    Ui::MainWindow *ui;
    CvCapture *capture;          // OpenCV Video Capture Variable
    IplImage *frame;            // Variable to capture a frame of the input video
    cv::Mat source_image;     // Variable pointing to the same input frame
    cv::Mat dest_image;      // Variable to output a frame of the processed video
    QTimer *imageTimer;
};

#endif // MAINWINDOW_H

mainwindow.cpp

#include "mainwindow.h"
#include "ui_mainwindow.h"

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
}

MainWindow::~MainWindow()
{
    delete ui;
    cvReleaseImage(&frame);
    cvReleaseCapture(&capture);
}

void MainWindow::on_buttonCaptureVideo_clicked()
{
    // Set to 25 frames per second

    const int imagePeriod = 1000/25;   // ms

    imageTimer = new QTimer(this);

    imageTimer->setInterval(imagePeriod);

    connect(imageTimer, SIGNAL(timeout()), this, SLOT(doNextFrame()));

    // Use the default camera
    capture = cvCreateCameraCapture(-1);

    while(capture)
    {
    // Capture a frame
    frame = cvQueryFrame(capture);

    // Point to the same frame
    source_image = frame;

    // Resize Image
    cv::resize(source_image, source_image, cv::Size(128,128) , 0, 0);

    // Change to RGB format
    cv::cvtColor(source_image,source_image,CV_BGR2RGB);

    // Convert to QImage
    QImage qimg = QImage((const unsigned char*) source_image.data, source_image.cols, source_image.rows, QImage::Format_RGB888); // convert to QImage

    // Display on Input Label
    ui->labelInputVideo->setPixmap(QPixmap::fromImage(qimg));

    // Resize the label to fit the image
    ui->labelInputVideo->resize(ui->labelInputVideo->pixmap()->size());

    }
}

void MainWindow::on_buttonExit_clicked()
{

    connect(ui->buttonExit, SIGNAL(clicked()), qApp, SLOT(closeAllWindows()));
}

Answer 1

When you click your button, the

while(capture) { ... }

loop will run forever, as capture will never be set to NULL.
This means the code flow never leaves your loop and thus the main thread cannot process anything else, like e.g. repainting.

The QTimer will emit its timeout() signals, but they will be placed as Events in Qt’s Event Queue. As long as your on_buttonCaptureVideo_clicked() method is running, those Events will not be processed.

Here are my suggestions how to make it work:

---

This code:

// Set to 25 frames per second  
const int imagePeriod = 1000/25;   // ms        
imageTimer = new QTimer(this);        
imageTimer->setInterval(imagePeriod);        
connect(imageTimer, SIGNAL(timeout()), this, SLOT(doNextFrame()));   
// Use the default camera            
capture = cvCreateCameraCapture(-1);  

belongs into the constructor of MainWindow as you want to set that up one time only. There is no need to do it again when the user clicks the button the second, third, etc time.

The code which is within your while loop should go into the doNextFrame() slot (without the while construct).

Then your button will only do

imageTimer->start();

and then e.g. do

imageTimer->stop();

when it is clicked again.

Example code:

void MainWindow::on_buttonCaptureVideo_clicked()
{
    if( imageTimer->isActive() )
    {
        imageTimer->stop();
    }
    else
    {
        imageTimer->start();
    }
}

---

What will happen if you do that?

When you click the button, your on_buttonCaptureVideo_clicked() clicked slot will be called from the GUI thread, the timer will be started, and the method will return almost instantly.
Now the GUI thread is free and able to handle repaints etc.
From that point on, the timer will be sending timeout() signals every 40ms. Whenever the GUI thread is free, it will handle this signal and call your doNextFrame slot.
This slot will capture the next frame and return when it is done. When it is done, the GUI thread will be able to process other Events (e.g. repaint) again.
As soon as you click the button again, the timer will stop, and no new timeout() events will be sent. If you still see a couple of frames after the button has been clicked, this could mean that the timer events were sent faster than they could be processed.