Question

I was here to ask, how can I convert an AVFrame to an opengl texture. Actually, I created a renderer the outputs me the audio (Audio is working) and the video, but the video is not outputing. Here is my code:

Texture creation:

glGenTextures(1,&_texture);
glBindTexture(GL_TEXTURE_2D,_texture);
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR ); 
glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );

Code Info: _texture variable is a GLuint that keeps the texture ID

Function that gets the AVFrame and convert it to OpenGL Texture:

int VideoGL::NextVideoFrame(){
// Get a packet from the queue
AVPacket *videopacket = this->DEQUEUE(VIDEO);
int frameFinished;
if(videopacket!=0){
    avcodec_decode_video2(_codec_context_video, _std_frame,&frameFinished,videopacket);

    if(frameFinished){

        sws_scale(sws_ctx, _std_frame->data, _std_frame->linesize, 0, _codec_context_video->height, _rgb_frame->data, _rgb_frame->linesize);

        if(_firstrendering){
        glBindTexture(GL_TEXTURE_2D,_texture);
        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, _codec_context_video->width,_codec_context_video->height,0,GL_RGB,GL_UNSIGNED_BYTE,_rgb_frame->data[0]);

        _firstrendering = false;

        }else{

            glActiveTexture(_texture);
            glBindTexture(GL_TEXTURE_2D,_texture);
            glTexSubImage2D(GL_TEXTURE_2D,0,0,0,_codec_context_video->width,_codec_context_video->height,GL_RGB,GL_UNSIGNED_BYTE,_rgb_frame->data[0]);

        }
        av_free_packet(videopacket);
        return 0;
    }else{

        av_free_packet(videopacket);
        return -1;
    }

}else{
    return -1;
}
return 0;
}

Code Information: There is a queue where a thread store the AVFrames, this function is frequently called to get the AVFrames, until it gets a NULL it stops to being called.

That’s actually not working. (I tried to look at some questions in stack overflow, it’s still not working)
Any example, or someone that helps me to correct any error there?

Additional Data: I tried to change the GL_RGB to GL_RGBA and started to play with the formats, anyway it crashes when I try GL_RGBA (Because the width and height are very big, anyway I tried to resize them). I have tried to change the sizes to Power Of 2, stills not working.

1 Edit:

Thread function:

DWORD WINAPI VideoGL::VidThread(LPVOID myparam){

VideoGL * instance = (VideoGL*) myparam;
instance->wave_audio->Start();

int quantity=0;

AVPacket packet;
while(av_read_frame(instance->_format_context,&packet) >= 0){
    if(packet.stream_index==instance->videoStream){
        instance->ENQUEUE(VIDEO,&packet);
    }
    if(packet.stream_index==instance->audioStream){
        instance->ENQUEUE(AUDIO,&packet);
    }
}

instance->ENQUEUE(AUDIO,NULL);
instance->ENQUEUE(VIDEO,NULL);

return 0;
}

Thread creation function:

CreateThread(NULL, 0, VidThread, this, NULL, NULL);

Where this refers to the class that contains the NextVideoFrame, and the _texture members.

Solved:

I followed some of the datenwolf tips, and now the video is displaying correctly with the audio/video:

Answer 1

OpenGL with multithreading is a bit tricky. There’s only one or no active OpenGL context per thread and a given OpenGL context can be active in only one thread at a time. Migrating a context between threads is possible, but you shouldn’t play hot OpenGL-context-potato between threads.

There are two possible ways:

• Use memory mapped Pixel Buffer Objects (map in one thread, update in the other)

• Use a secondary OpenGL context, sharing its textures with the other. Then bind one context to each thread (you can use the same drawable, i.e. HDC on Windows). Then you can update the texture in the one thread while the other draws it. Texture updates introduce synchronization points between threads, so you can’t accidently write into a texture while it’s being accessed for drawing.