stencil buffer problem on some video cards

[avm1979]

I'm using the stencil buffer for clipping things in my app, and it seems like it's only working on some video cards.

I ran the following code to try and diagnose the problem:

GL11.glEnable(GL11.GL_STENCIL_TEST);
boolean stencil = GL11.glIsEnabled(GL11.GL_STENCIL_TEST);
ByteBuffer b = ByteBuffer.allocateDirect(128);
IntBuffer i = b.asIntBuffer();
log.debug("Stencil test enabled: " + stencil);
GL11.glGetInteger(GL11.GL_STENCIL_BITS, i);
log.debug("Stencil bits: " + b.get());

Looks like on the cards where it doesn't work, the number of stencil bits is 0.  Which would cetainly explain why it's not working, but I can't believe that these cards don't support a stencil buffer.  One of them, for example is a relatively-high-end GeForce 7800GXT.  It does, however, work on an older Radeon 9800XT.

Could anyone give me any pointers?  Seems like I must be missing some step in enabling the stencil buffer... aside from glEnable


Just in case it's useful, here is the code I'm using to clip via the stencil buffer:

GL11.glClear(GL11.GL_STENCIL_BUFFER_BIT);
GL11.glEnable(GL11.GL_STENCIL_TEST);

// only draw to the stencil buffer
GL11.glStencilFunc(GL11.GL_NEVER, 1, 1);
// the 2nd and 3rd params are irrelevant since those conditions never happen
GL11.glStencilOp(GL11.GL_REPLACE, GL11.GL_KEEP, GL11.GL_KEEP);      
       
rect.render(...);  // rect is a sprite - ie a quad w/ a texture mapped to it
      
// only draw where the stencil has been drawn on by rect
GL11.glStencilFunc(GL11.GL_EQUAL, 1, 1);
// and don't touch the stencil
GL11.glStencilOp(GL11.GL_KEEP, GL11.GL_KEEP, GL11.GL_KEEP);

// draw anything that needs to be clipped

...


// turn of clipping
GL11.glDisable(GL11.GL_STENCIL_TEST);


Any help is appreciated!

[Orangy Tang]

If you're getting 0 stencil bits from a glGet that means your framebuffer was created without a stencil component. You need to manually request >0 stencil bits (usually 8, any other value is likely to not be supported) when you create your display. Something like:

Display.create( new PixelFormat(32, 0, 24, 8, 0) ); // 32 bit colour, 24 bit depth, 8 bit stencil

[avm1979]

Thank you, will try it out when I get home tonight.  Guess that would explain the difference between diff cards, too, maybe just a different default - I was just calling Display.create() w/ no arguments.