Question

I have a code where I receive YUV channels and I’m drawing them using OpenGLES. Basically, I have a shader that combines them together.

I would like to add a sharpen filter to the result (using the following example: http://igortrindade.wordpress.com/2010/04/23/fun-with-opengl-and-shaders/)

I’m not sure how to run another shader on the actual result (since I would like to run it after my previous shader combined all channels to a single frame).

My current code looks like that:

            glUniform1i(texLum, 0);
            glUniform1i(texU, 1);
            glUniform1i(texV, 2);

            glEnableVertexAttribArray(positionLoc);
            glVertexAttribPointer(positionLoc, 
                                  4, 
                                  GL_FLOAT, 
                                  GL_FALSE, 
                                  0, 
                                  &vertices[0]);

            glEnableVertexAttribArray(texCoordLoc);
            glVertexAttribPointer(texCoordLoc, 
                                  2, 
                                  GL_FLOAT, 
                                  GL_FALSE, 
                                  0, 
                                  &texCoords[0]);


            glDrawElements(GL_TRIANGLES, sizeof(indices)/sizeof(indices[0]), GL_UNSIGNED_BYTE, &indices[0]);

I guess I need to add the new shader right before the last line (glDrawElements), but I’m not sure how to call it.

My shader looks like this:

    static char const *frag = 
    "uniform lowp sampler2D texLum; \n"
    "uniform lowp sampler2D texU; \n"
    "uniform lowp sampler2D texV; \n"
    "varying mediump vec2 texCoordAtFrag; \n"
    "void main() { \n"
    "   lowp float Y = texture2D(texLum, texCoordAtFrag).r; \n"
    "   lowp float U = texture2D(texU, texCoordAtFrag).r; \n"
    "   lowp float V = texture2D(texV, texCoordAtFrag).r; \n"
    "   lowp float R = 1.164 * (Y - 16.0 / 256.0) + 1.596 * (V - 0.5); \n"
    "   lowp float G = 1.164 * (Y - 16.0 / 256.0) - 0.813 * (V - 0.5) - 0.391 * (U - 0.5); \n"
    "   lowp float B = 1.164 * (Y - 16.0 / 256.0) + 2.018 * (U - 0.5); \n"
    "   gl_FragColor = vec4(R,G,B,1); \n"
    "}\r\n";

    static char const *vert = 
    "varying mediump vec2 texCoordAtFrag; \n"
    "attribute vec4 Position; \n"
    "attribute vec2 TexCoord; \n"
    "void main() { \n"
    "   texCoordAtFrag = TexCoord; \n"
    "   gl_Position = Position; \n"
    "}\r\n";

Where texLum,texU,texV are the textures holding the channels.

Answer 1

Sharpen is a convolution filter, so it reads nine input values to produce one output value. So if you have another shader that is supposed to occur before it and operates one pixel at a time, there’d be a decent argument for running them in two steps (YUV transformation first, sharpen second) so as to eliminate repeated calculations, even if it weren’t also the easiest way to combine shaders as closed boxes.

If you want to combine them live, break the YUV transformation into a separate function and have the sharpen filter call that function instead of texture2D. The GL shader compiler doesn’t have any sort of restriction on the number of source files you can link together to make a compiled program precisely so that you can follow the usual programming routes for function reuse.

If you’d prefer to run one then run the other then use an intermediate render-to-texture stage. Do the YUV transform, then switch buffers and use the output of that as the input to the sharpen.

In practice the former may actually be faster than the latter since a YUV transform is probably a fast operation (if it’s YUV to RGB then it’s one matrix multiplication, for example) whereas memory speed and the need to do quite a drastic change state can be quite expensive. You’d probably need to profile if performance is a concern.

EDIT: so, from your current main you could just adapt that to (typed here, as I go, please forgive errors):

"uniform lowp sampler2D texLum; \n"
"uniform lowp sampler2D texU; \n"
"uniform lowp sampler2D texV; \n"
"varying mediump vec2 texCoordAtFrag; \n"
"lowp vec4 yuvTexture2D(mediump vec2 coord) { \n"
"   lowp float Y = texture2D(texLum, coord).r; \n"
"   lowp float U = texture2D(texU, coord).r; \n"
"   lowp float V = texture2D(texV, coord).r; \n"
"   lowp float R = 1.164 * (Y - 16.0 / 256.0) + 1.596 * (V - 0.5); \n"
"   lowp float G = 1.164 * (Y - 16.0 / 256.0) - 0.813 * (V - 0.5) - 0.391 * (U - 0.5); \n"
"   lowp float B = 1.164 * (Y - 16.0 / 256.0) + 2.018 * (U - 0.5); \n"
"   return vec4(R,G,B,1.0); \n"
"}\r\n

And then in your sharpen filter you’d substitute the call to texture2D(<whatever>, coord) with a call to yuvTexture2D(coord), having either included the fragment in the source listing for the sharpen shader or linked it into the program. With respect to switching to using a matrix approach, I guess you’d want (I’m going to format it other than as a string constant, for ease of typing):

uniform lowp sampler2D texLum;
uniform lowp sampler2D texU;
uniform lowp sampler2D texV;
varying mediump vec2 texCoordAtFrag;

const mediump mat4 yuvToRgb = 
mat4(   1.164,  1.164,  1.164,  -0.07884, 
        2.018, -0.391,    0.0,  1.153216,
        0.0, -0.813,    1.596,  0.53866, 
        0.0,      0.0,    0.0,  1.0);

lowp vec4 yuvTexture2D(mediump vec2 coord) {
   lowp vec4 yuv = 
         vec4(
             texture2D(texLum, coord).r,
             texture2D(texU, coord).r,
             texture2D(texV, coord).r,
             1.0)
   return yuvToRgb * yuv;
}