Question

I am having trouble displaying a raw YUV file that it is in NV12 format.

I can display a selected frame, however, it is still mainly in black and white with certain shades of pink and green.

Here is how my output looks like

Anyways, here is how my program works. (This is done in cocoa/objective-c, but I need your expert advice on program algorithm, not on syntax.)

Prior to program execution, the YUV file is stored in a binary file named "test.yuv". The file is in NV12 format, meaning the Y plan is stored first, then the UV plan is interlaced. My file extraction has no problem because I did a lot testings on it.

During initiation, I create a lookup table that will convert binary/8 bites/a byte/chars into respected Y, U, V float values

For the Y plane this is my code

-(void)createLookupTableY //creates a lookup table for converting a single byte into a float between 0 and 1
{
    NSLog(@"YUVFrame: createLookupTableY");
    lookupTableY = new float [256];
    
    for (int i = 0; i < 256; i++)
    {
        lookupTableY[i] = (float) i /255;
        //NSLog(@"lookupTableY[%d]: %f",i,lookupTableY[i]);//prints out the value of each float
    }
}

The U Plane lookup table

-(void)createLookupTableU //creates a lookup table for converting a single byte into a float between 0 and 1
{
    NSLog(@"YUVFrame: createLookupTableU");
    lookupTableU = new float [256];
    
    for (int i = 0; i < 256; i++)
    {
        lookupTableU[i] =  -0.436 + (float) i / 255* (0.436*2);
        NSLog(@"lookupTableU[%d]: %f",i,lookupTableU[i]);//prints out the value of each float
    }
}

And the V look-up table

-(void)createLookupTableV //creates a lookup table for converting a single byte into a float between 0 and 1
{
    NSLog(@"YUVFrame: createLookupTableV");
    lookupTableV = new float [256];
    
    for (int i = 0; i < 256; i++)
    {
        lookupTableV[i] =  -0.615 + (float) i /255 * (0.615*2);
        NSLog(@"lookupTableV[%d]: %f",i,lookupTableV[i]);//prints out the value of each float
    }
}

after this point, I extract the Y & UV plan and store them into two buffers, yBuffer & uvBuffer

at this point, I attempt to convert the YUV data and stored it into a RGB buffer array called "frameImage"

-(void)sortAndConvert//sort the extracted frame data into an array of float
{
    NSLog(@"YUVFrame: sortAndConvert");
    int frameImageCounter = 0;
    int pixelCounter = 0;
    for (int y = 0; y < YUV_HEIGHT; y++)//traverse through the frame's height
    {
        for ( int x = 0; x < YUV_WIDTH; x++)//traverse through the frame's width
        {
            
            float Y = lookupTableY [yBuffer [y*YUV_WIDTH + x] ];
            float U = lookupTableU [uvBuffer [ ((y / 2) * (YUV_WIDTH / 2) + (x/2)) * 2  ] ]; 
            float V = lookupTableV [uvBuffer [  ((y / 2) * (YUV_WIDTH / 2) + (x/2)) * 2 + 1] ];
        
            float RFormula = Y + 1.13983f * V;
            float GFormula = Y - 0.39465f * U - 0.58060f * V;
            float BFormula = Y + 2.03211f * U;
            
             frameImage [frameImageCounter++] = [self clampValue:RFormula];
             frameImage [frameImageCounter++] = [self clampValue:GFormula];
             frameImage [frameImageCounter++] = [self clampValue:BFormula];

        }
    }

}

then I tried to draw the Image in OpenGL

-(void)drawFrame:(int )x
{
    
    GLuint texture;
    
    glGenTextures(1, &texture);
    glBindTexture(GL_TEXTURE_2D, texture);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, YUV_WIDTH, YUV_HEIGHT, 0, GL_RGB, GL_FLOAT, frameImage);
    
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);

    glEnable(GL_TEXTURE_2D);
    glBindTexture(GL_TEXTURE_2D, texture);
    glRotatef( 180.0f, 1.0f, 0.0f, 0.0f );

    glBegin( GL_QUADS );
    glTexCoord2d(0.0,0.0); glVertex2d(-1.0,-1.0);
    glTexCoord2d(1.0,0.0); glVertex2d(+1.0,-1.0);
    glTexCoord2d(1.0,1.0); glVertex2d(+1.0,+1.0);
    glTexCoord2d(0.0,1.0); glVertex2d(-1.0,+1.0);
    glEnd();
    
    glFlush();
}

so basically this my program in a nut shell. essentially i read the binary YUV files, stores all the data in a char array buffer. i then translate these values into their respected YUV float values.

This is where I think the error might be: in the Y lookup table I standardize the Y plane to [0,1], in the U plane I standardized the values between [-0.435,0.436], and in the V plane I standardized it bewteen [-0.615,0.615]. I did this because those are the YUV value ranges according to wikipedia.

And the YUV to RGB formula is the same formula from Wikipedia. I have also tried various other formulas, and this is the only formula that gives the rough outlook of the frame. Anyone might venture to guess to why my program is not correctly displaying the YUV frame data. I think it is something to do with my standardization technique, but it seems alright to me.

I have done a lot of testings, and I am 100% certain that the error is caused by by look up table. I don’t think my setting formulas are correct.

A note to everyone who is reading
this. For the longest time, my frame
was not displaying properly because I
was not able to extract the frame data
correctly.

When I first started to program, I was
under the impression that in a clip of
say 30 frames, all 30 Y planar datas
are first written in the data file,
followed then by UV plane datas.

What I found out through trial and
error was that for a YUV data file,
specifically NV12, the data file is
stored in the following fashion

Y(1) UV(1) Y(2) UV(2) … …

@nschmidt

I changed my code to what you suggested

float U = lookupTableU [uvBuffer [ (y * (YUV_WIDTH / 4) + (x/4)) * 2 ] ]
float V = lookupTableU [uvBuffer [ (y * (YUV_WIDTH / 4) + (x/4)) * 2 + 1] ]

and this is the result that i get

here is the print line from the console. i am print out the values for Y, U, V and
RGB value that are being translated and stored on in the frameImage array

YUV:[0.658824,-0.022227,-0.045824] RGBFinal:[0.606593,0.694201,0.613655]

YUV:[0.643137,-0.022227,-0.045824] RGBFinal:[0.590906,0.678514,0.597969]

YUV:[0.607843,-0.022227,-0.045824] RGBFinal:[0.555612,0.643220,0.562675]

YUV:[0.592157,-0.022227,-0.045824] RGBFinal:[0.539926,0.627534,0.546988]

YUV:[0.643137,0.025647,0.151941] RGBFinal:[0.816324,0.544799,0.695255]

YUV:[0.662745,0.025647,0.151941] RGBFinal:[0.835932,0.564406,0.714863]

YUV:[0.690196,0.025647,0.151941] RGBFinal:[0.863383,0.591857,0.742314]

Update July 13, 2009

The problem was finally solved thanks to the recommendation from nschmidt . It turns out that my YUV file was actually in YUV 4:1:1 format. I was originally told that it was in YUV NV12 format. Anyways, I would like to share my results with you.

Here is output

and my code for decode is as follows

        float Y = (float) yBuffer [y*YUV_WIDTH + x] ;
        float U = (float) uvBuffer [ ((y / 2) * (YUV_WIDTH  / 2) + (x/2))   ] ; 
        float V = (float) uvBuffer [  ((y / 2) * (YUV_WIDTH  / 2) + (x/2))  + UOffset] ;

        float RFormula = (1.164*(Y-16) + (1.596* (V - 128) ));
        float GFormula = ((1.164 * (Y - 16)) - (0.813 * ((V) - 128)) - (0.391 * ((U) - 128)));
        float BFormula = ((1.164 * (Y - 16)) + (2.018 * ((U) - 128)));


        frameImage [frameImageCounter] = (unsigned char)( (int)[self clampValue:RFormula]);
        frameImageCounter ++;
        frameImage [frameImageCounter] =  (unsigned char)((int)[self clampValue:GFormula]);
        frameImageCounter++;
        frameImage [frameImageCounter] = (unsigned char)((int) [self clampValue:BFormula]);
        frameImageCounter++;



GLuint texture;

glGenTextures(1, &texture);
glEnable(GL_TEXTURE_2D);

glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP_SGIS, GL_TRUE);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, YUV_WIDTH, YUV_HEIGHT, 0, GL_RGB, GL_UNSIGNED_BYTE, frameImage);

//glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_S,GL_CLAMP_TO_EDGE_SGIS);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_T,GL_CLAMP_TO_EDGE_SGIS);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_LINEAR);

glRotatef( 180.0f, 1.0f, 0.0f, 0.0f );

glBegin( GL_QUADS );
glTexCoord2d(0.0,0.0); glVertex2d(-1.0,-1.0);
glTexCoord2d(1.0,0.0); glVertex2d(+1.0,-1.0);
glTexCoord2d(1.0,1.0); glVertex2d(+1.0,+1.0);
glTexCoord2d(0.0,1.0); glVertex2d(-1.0,+1.0);
glEnd();

NSLog(@"YUVFrameView: drawRect complete");
glFlush();

essentially, I used NSData for the raw file extraction. stored in a char array buffer. For YUV to RGB conversion, I used the above formula, afterwards, I clamped the values to [0:255]. then I used a 2DTexture in OpenGL for display.

So if you are converting YUV to RGB in the future, use the formula from above. If are using the YUV to RGB conversion formula from the earlier post, then you need to display the texture in GL_Float from the values for RGB are clamped between [0:1]

Answer 1

Next try 🙂
I think you’re uv buffer is not interleaved. It looks like the U values come first and then the array of V values. Changing the lines to

unsigned int voffset = YUV_HEIGHT * YUV_WIDTH / 2;
float U = lookupTableU [uvBuffer [ y * (YUV_WIDTH / 2) + x/2] ]; 
float V = lookupTableV [uvBuffer [ voffset + y * (YUV_WIDTH / 2) + x/2] ];

might indicate if this is really the case.