Question

In my Mac application, I define a rectangular texture based on YUV 4:2:2 data from an attached camera. Using standard vertex and texture coordinates, I can draw this to a rectangular area on the screen without any problems.

However, I would like to use a GLSL fragment shader to process these image frames on the GPU, and am having trouble passing in the rectangular video texture as a uniform to the fragment shader. When I attempt to do so, the texture simply reads as black.

The shader program compiles, links, and passes validation. I am receiving the proper address for the uniform from the shader program. Other uniforms, such as floating point values, pass in correctly and the fragment shader responds to changes in these values. The fragment shader receives the correct texture coordinates. I’ve also sprinkled my code liberally with glGetError() and seen no errors anywhere.

The vertex shader is as follows:

void main()
{
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

    gl_FrontColor = gl_Color;
    gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;
}

and the fragment shader is as follows:

uniform sampler2D videoFrame;

void main()
{
    gl_FragColor = texture2D(videoFrame, gl_TexCoord[0].st);
}

This should simply display the texture on my rectangular geometry.

The relevant drawing code is as follows:

static const GLfloat squareVertices[] = {
    -1.0f, -1.0f,
    1.0f, -1.0f,
    -1.0f,  1.0f,
    1.0f,  1.0f,
};

const GLfloat textureVertices[] = {
    0.0, videoImageSize.height,
    videoImageSize.width, videoImageSize.height,
    0.0, 0.0,
    videoImageSize.width, 0.0
};

CGLSetCurrentContext(glContext);

if(!readyToDraw)
{
    [self initGL];
    readyToDraw = YES;
}


glViewport(0, 0, (GLfloat)self.bounds.size.width, (GLfloat)self.bounds.size.height);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();

glMatrixMode(GL_MODELVIEW);
glLoadIdentity(); 

glClearColor(0.5f, 0.5f, 0.5f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);

glEnable(GL_TEXTURE_2D);

glGenTextures(1, &textureName);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_RECTANGLE_EXT, textureName);

glTexImage2D(GL_TEXTURE_RECTANGLE_EXT, 0, GL_RGBA, videoImageSize.width, videoImageSize.height, 0, GL_YCBCR_422_APPLE, GL_UNSIGNED_SHORT_8_8_REV_APPLE, videoTexture);  

glUseProgram(filterProgram);    

glUniform1i(uniforms[UNIFORM_VIDEOFRAME], 0);   

glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glVertexPointer(2, GL_FLOAT, 0, squareVertices);
glTexCoordPointer(2, GL_FLOAT, 0, textureVertices);

glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

[super drawInCGLContext:glContext pixelFormat:pixelFormat forLayerTime:interval displayTime:timeStamp];

glDeleteTextures(1, &textureName);

This code resides within a CAOpenGLLayer, where the superclass’s -drawInCGLContext:pixelFormat:forLayerTime: displayTime: simply runs glFlush().

The uniform address is read using code like the following:

uniforms[UNIFORM_VIDEOFRAME] = glGetUniformLocation(filterProgram, "videoFrame");

As I said, if I comment out the glUseProgram() and glUniform1i() lines, this textured rectangle draws properly. Leaving them in leads to a black rectangle being drawn.

What could be preventing my texture uniform from being passed into my fragment shader?

Answer 1

Not sure about the GLSL version you’re using, but from 1.40 upwards there’s the type sampler2DRect specifically for accessing non-power-of-two textures. Might be what you’re looking for, however I don’t know how rectangular textures were handled before glsl 1.40.