Question

In reference to the problem diskussed in article OpenGL clipping a new question arises.

I am implementing the clipping for a node based 2D scene graph. Whenever the clipping boxes are axis aligned I am using the glScissor like proposed in OpenGL clipping. I have sucessfully implemented node clipping of boxes that are axis aligned.

It turns out that each node has to intersect it’s clipping rectangle with the ones of it’s ancestors that use clipping. (That is necessary in case of siblings that are overlapping with the clipping box of an ancestor).

The mechanism of intersecting non axis aligned rectangles has to be implemented using the stencil buffer. I started implementing the proposed solution in OpenGL clipping but am having problems with chidrens having clip rects overlaping with theyr ancestors ones.

Right now stencil clipping works perfect with only one clipping box.
But in case of a child or grand child that intersects with an ancestor the algorith fails, because the intersection of the 2 involved rectangles would be needed here (like in the axis aligned version) as mask and not the full rectangle of the child.

I have thought out the following algorithm:

The topmost node writes starts with a counter value of 1 and draws it’s clipping rectangle filled with 1s into the stencil buffer and renders it’s children stencil-testing against 1. Each sibling that also has clipping turned on draws it’s bounding rectangle by adding 1 to the stencil buffer and then testing against 2 and so on. Now when a siblings clipping rect overlaps with an ancestors one,the region where they overlap will be filled with 2s, giving perfect clipping when testing against 2. This algorithm can be extended to a maximum of 255 nested clipping nodes.

Here comes my questions:

How do I perform rendering to the stencil buffer in a way that instead of 1s being writing,1s are added to the current stencil buffer value, whenever rendering is performed.

This is my code I use to prepare for rendering to the stencil buffer:

glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, ref, mask);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);

I am looking for a setup that will increase the stencil buffers current value by the value of ref instead of writing ref into the buffer.

Can someone help me out?

Answer 1

glStencilFunc(GL_ALWAYS, ref, mask); says that:

• the first argument says that the stencil comparison test always succeeds
• the second and third are hence ignored, but in principle would set a constant to compare the value coming from the stencil buffer to and which of the bits are used for comparison

glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE); has the effect that:

• when the stencil test fails, the new value replaces the old (per the first argument)
• when the depth test fails but the stencil test passes, the new value replaces the old (per the second argument)
• when the stencil and depth test both pass, the new value replaces the old (per the third argument)

So you’re setting things up so that everything you try to draw will pass the stencil test, and in any case its value will be copied into the stencil buffer whether it passes the test or not.

Probably what you want to do is to change your arguments to glStencilOp so that you perform a GL_INCR on pixels that pass. So in your stencil buffer you’ll end up with a ‘1’ anywhere touched by a single polygon, a ‘2’ anywhere that you drew two successive polygons, a ‘3’ anywhere you drew 3, etc.

When drawing the user-visible stuff you can then use something like glStencilFunc set to GL_GEQUAL and to compare incoming values to ‘1’, ‘2’, ‘3’ or whatever.

If you prefer not to keep track of how many levels deep you are, supposing you have one clip area drawn into the stencil, you can modify it to the next clip area by:

1. drawing all new geometry with GL_INCR; this’ll result in a stencil buffer where all pixels that were in both areas have a value of ‘2’ and all pixels that were in only one area have a value of ‘1’
2. draw a full screen polygon, to pass only with stencil fund GL_GREATER and reference value 0, with GL_DECR. Result is ‘0’ everywhere that was never painted or was painted only once, ‘1’ everywhere that was painted twice.