Question

Basically, I’m trying to make a layering system with OpenGL|es that has to support layer hierarchies (layers can have sublayers and so on). When a layer has a parent, it has to be clipped to its parent layer when it is drawn. When all layers are rectangular, it is fairly straight forward and I can just work out the intersections and crop out the out of bounds bits. However, each layer can also have a transform matrix and if we transform a layer inside another layer, it will no longer be rectangular meaning that I can’t just work out the intersections.

The illustration below shows the issue (the parent is in pink while the child is in light green):

For the above example, a glScissor call would be sufficient to set a rectangular bounding box but this will not do as the parent layer can also be transformed (so it will no longer be rectangular). Also, if a parent layer is transformed, the child’s transform matrix is multiplied by the parent’s before the child is rendered, therefore creating an illusion of the child being inside the parent even when it is transformed.

Answer 1

A few suggestions:

• Clip the geometry of the sublayers yourself (fast but you need to do the math).
• Render each layer into a texture (flexible but slow with many layers).
• Use the stencil buffer (fill rate intensive with many layers).