Question

I’m using OpenTK in MonoTouch to render some textures in iOS, and some of the textures come up broken. This is a closeup of an iPad screenshot showing one correctly rendered texture (the top one) and two broken ones below:

I’m not doing anything weird. I’m loading the texture from a semitransparent PNG using CGImage->CGBitmapContext->GL.TexImage2D. I’m rendering each sprite with two triangles, and my fragment shader just reads the texel from the sampler with texture2D() and multiplies it by a uniform vec4 to color the texture.

The files themselves seem to be okay, and the Android port of the same application using Mono for Android, and the exact same binary resources renders them perfectly. As you can see, other transparent textures work fine.

If it helps, pretty much every texture is broken when I run the program in the simulator. Also this problem persists even if I rebuild the program.

Any ideas on how to figure out what is causing this problem?

Here’s my vertex shader:

attribute vec4 spritePosition;
attribute vec2 textureCoords;
uniform mat4 projectionMatrix;
uniform vec4 color;

varying vec4 colorVarying;
varying vec2 textureVarying;

void main()
{
    gl_Position = projectionMatrix * spritePosition;
    textureVarying = textureCoords;

    colorVarying = color;
}

Here’s my fragment shader:

varying lowp vec4 colorVarying;
varying lowp vec2 textureVarying;
uniform sampler2D spriteTexture;

void main()
{
    gl_FragColor = texture2D(spriteTexture, textureVarying) * colorVarying;
}

I’m loading the image like this:

using (var bitmap = UIImage.FromFile(resourcePath).CGImage)
{
    IntPtr pixels = Marshal.AllocHGlobal(bitmap.Width * bitmap.Height * 4);
    using (var context = new CGBitmapContext(pixels, bitmap.Width, bitmap.Height, 8, bitmap.Width * 4, bitmap.ColorSpace, CGImageAlphaInfo.PremultipliedLast))
    {
        context.DrawImage(new RectangleF(0, 0, bitmap.Width, bitmap.Height), bitmap);
        int[] textureNames = new int[1];
        GL.GenTextures(1, textureNames);
        GL.BindTexture(TextureTarget.Texture2D, textureNames[0]);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureMagFilter, (int)All.Linear);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureMinFilter, (int)All.Linear);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureWrapS, (int)All.ClampToEdge);
        GL.TexParameter(TextureTarget.Texture2D, TextureParameterName.TextureWrapT, (int)All.ClampToEdge);
        GL.TexImage2D(TextureTarget.Texture2D, 0, PixelInternalFormat.Rgba, bitmap.Width, bitmap.Height, 0, PixelFormat.Rgba, PixelType.UnsignedByte, pixels);
        CurrentResources.Add(resourceID, new ResourceData(resourcePath, resourceType, 0, new TextureEntry(textureNames[0], bitmap.Width, bitmap.Height)));
    }
}

and in my onRenderFrame, I have this:

GL.ClearColor(1.0f, 1.0f, 1.0f, 1.0f);
GL.Clear(ClearBufferMask.ColorBufferBit);
GL.Enable(EnableCap.Blend);
GL.BlendFunc(BlendingFactorSrc.SrcAlpha, BlendingFactorDest.OneMinusSrcAlpha);
GL.UseProgram(RenderingProgram);
GL.VertexAttribPointer((int)ShaderAttributes.SpritePosition, 2, VertexAttribPointerType.Float, false, 0, squareVertices);
GL.VertexAttribPointer((int)ShaderAttributes.TextureCoords, 2, VertexAttribPointerType.Float, false, 0, squareTextureCoords);
GL.EnableVertexAttribArray((int)ShaderAttributes.SpritePosition);
GL.EnableVertexAttribArray((int)ShaderAttributes.TextureCoords);
//...
GL.ActiveTexture(TextureUnit.Texture0);
GL.BindTexture(TextureTarget.Texture2D, textureEntry.TextureID);
GL.Uniform1(Uniforms[(int)ShaderUniforms.Texture], 0);
// ...    
GL.DrawArrays(BeginMode.TriangleStrip, 0, 4);

That triangle strip is made out of two triangles that make up the texture, with the vertex and texture coordinates set to where I want to show my sprite. projectionMatrix is a simple ortographic projection matrix.

As you can see, I’m not trying to do anything fancy here. This is all pretty standard code, and it works for some textures, so I think that in general the code is okay. I’m also doing pretty much the same thing in Mono for Android, and it works pretty well without any texture corruption.

Corrupted colors like that smell like uninitialized variables somewhere, and seeing it happen only on the transparent part leads me to believe that I’m having uninitialized alpha values somewhere. However, GL.Clear(ClearBufferMask.ColorBufferBit) should clear my alpha values, and even so, the background texture has an alpha value of 1, and with the current BlendFunc, should set the alpha for those pixels to 1. Afterwards, the transparent textures have alpha values ranging from 0 to 1, so they should blend properly. I see no uninitialized variables anywhere.

…or… this is all the fault of CGBitmapContext. Maybe by doing DrawImage, I’m not blitting the source image, but drawing it with blending instead, and the garbage data comes from when I did AllocGlobal. This doesn’t explain why it consistently happens with just these two textures though… (I’m tagging this as core-graphics so maybe one of the quartz people can help)

Let me know if you want to see some more code.

Answer 1

Okay, it is just as I had expected. The memory I get with Marshal.AllocHGlobal is not initialized to anything, and CGBitmapContext.DrawImage just renders the image on top of whatever is in the context, which is garbage.

So the way to fix this is simply to insert a context.ClearRect() call before I call context.DrawImage().

I don’t know why it worked fine with other (larger) textures, but maybe it is because in those cases, I’m requesting a large block of memory, so the iOS (or mono) memory manager gets a new zeroed block, while for the smaller textures, I’m reusing memory previously freed, which has not been zeroed.

It would be nice if your memory was allocated to something like 0xBAADF00D when using the debug heap, like LocalAlloc does in the Windows API.

Two other somewhat related things to remember:

1. In the code I posted, I’m not releasing the memory requested with AllocHGlobal. This is a bug. GL.TexImage2D copies the texture to VRAM, so it is safe to free it right there.

2. context.DrawImage is drawing the image into a new context (instead of reading the raw pixels from the image), and Core Graphics only works with premultiplied alpha (which I find idiotic). So the loaded texture will always be loaded with premultiplied alpha if I do it in this way. This means that I must also change the alpha blending function to GL.BlendFunc(BlendingFactorSrc.One, BlendingFactorDest.OneMinusSrcAlpha), and make sure that all crossfading code works over the entire RGBA, and not just the alpha value.