Question

Does anybody know a good tutorial or documentation that deals with mixing OpenCV and OpenGL in C++, related with Computer Vision and 3D rendering?

I want to create virtual scenarios with objects and then find object's 3D poses using OpenCV, so I can compare with the known OpenGL position but I didn’t find much “quality” information so far. For example, I would need the intrinsic parameters of the opengl camera in order to use OpenCV for detecting objects in 3D world, etc.

Answer 1

There is nothing difficult in mixing OpenGL and OpenCV. These are just two libraries, one for graphics and the other for computer vision. There is no overlap so you can just include both in your project.

Then it should be a matter of rendering some rectangle or a box (or whatever you wish to locate using OpenCV) – there are plenty of tutorials for that, and then copying the rendered data using the glReadPixels() function to client memory and passing that to OpenCV.

There is no tutorial most likely because it is so simple …

You can just:

void onDisplay(void* param)
{
    glClearColor(0, 1, 0, 1);
    glClear(GL_COLOR_BUFFER_BIT); // Make a green window.
}

...

void MyClass::MyInit(bool useCustomInit)
{
    std::string winname = "MyWnd";
    if (useCustomInit) {
        namedWindow(winname, CV_WINDOW_NORMAL);
        resizeWindow(winname, 640, 480);
        auto handle = cvGetWindowHandle(winname.c_str());
        InitializeOpenGL(handle); // your custom OpenGL initialization.
    } else {
        namedWindow(winname, CV_WINDOW_NORMAL | CV_WINDOW_OPENGL);
        resizeWindow(winname, 640, 480);
    }
    void *param_value = reinterpret_cast<void*>(this); // Can e.g. do this or just use nullptr.
    setOpenGlDrawCallback(winname, onDisplay, param_value);

    // ...
}

If you’re happy with default OpenGL window, you can use the Init(false). If you want fancy bit depths and stencil buffers and whatnot, initialize OpenGL yourself using the OS’ window handle. There are really really many tutorials for doing that.