I'm using the stencil buffer for clipping things in my app, and it seems like it's only working on some video cards.
I ran the following code to try and diagnose the problem:
GL11.glEnable(GL11.GL_STENCIL_TEST);
boolean stencil = GL11.glIsEnabled(GL11.GL_STENCIL_TEST);
ByteBuffer b = ByteBuffer.allocateDirect(128);
IntBuffer i = b.asIntBuffer();
log.debug("Stencil test enabled: " + stencil);
GL11.glGetInteger(GL11.GL_STENCIL_BITS, i);
log.debug("Stencil bits: " + b.get());
Looks like on the cards where it doesn't work, the number of stencil bits is 0. Which would cetainly explain why it's not working, but I can't believe that these cards don't support a stencil buffer. One of them, for example is a relatively-high-end GeForce 7800GXT. It does, however, work on an older Radeon 9800XT.
Could anyone give me any pointers? Seems like I must be missing some step in enabling the stencil buffer... aside from glEnable :)
Just in case it's useful, here is the code I'm using to clip via the stencil buffer:
GL11.glClear(GL11.GL_STENCIL_BUFFER_BIT);
GL11.glEnable(GL11.GL_STENCIL_TEST);
// only draw to the stencil buffer
GL11.glStencilFunc(GL11.GL_NEVER, 1, 1);
// the 2nd and 3rd params are irrelevant since those conditions never happen
GL11.glStencilOp(GL11.GL_REPLACE, GL11.GL_KEEP, GL11.GL_KEEP);
rect.render(...); // rect is a sprite - ie a quad w/ a texture mapped to it
// only draw where the stencil has been drawn on by rect
GL11.glStencilFunc(GL11.GL_EQUAL, 1, 1);
// and don't touch the stencil
GL11.glStencilOp(GL11.GL_KEEP, GL11.GL_KEEP, GL11.GL_KEEP);
// draw anything that needs to be clipped
...
// turn of clipping
GL11.glDisable(GL11.GL_STENCIL_TEST);
Any help is appreciated!
If you're getting 0 stencil bits from a glGet that means your framebuffer was created without a stencil component. You need to manually request >0 stencil bits (usually 8, any other value is likely to not be supported) when you create your display. Something like:
Display.create( new PixelFormat(32, 0, 24, 8, 0) ); // 32 bit colour, 24 bit depth, 8 bit stencil
Thank you, will try it out when I get home tonight. Guess that would explain the difference between diff cards, too, maybe just a different default - I was just calling Display.create() w/ no arguments.