So I'm trying to overlay a 2D box on top of the rest of my geometry. To save on rendering times, I want to draw the overlay first, drawing into the stencil as I do. Then when I draw the rest of it, I use the stencil test to see if the overlay has already been drawn there.
First of all I tried this:
void init() {
glEnable(GL_STENCIL_TEST);
glClearStencil(0);
glStencilFunc(GL_LESS, 1, 0xFFFFFFFF);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
}
void render() {
glClear(GL_STENCIL_BUFFER_BIT);
glPushAttrib(GL_STENCIL_BUFFER_BIT);
glStencilFunc(GL_ALWAYS, 1, 0xFFFFFFFF);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
drawOverlay();
glPopAttrib();
drawRestOfScene();
}
NB This is just the important stuff pertaining to the question.
However, if I do this then only the overlay gets drawn and nothing else. To get it to work as expected I must change the regular stencil function to GL_GREATER like so:
void init() {
glEnable(GL_STENCIL_TEST);
glClearStencil(0);
glStencilFunc(GL_GREATER, 1, 0xFFFFFFFF); //This line is the only change
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
}
void render() {
glClear(GL_STENCIL_BUFFER_BIT);
glPushAttrib(GL_STENCIL_BUFFER_BIT);
glStencilFunc(GL_ALWAYS, 1, 0xFFFFFFFF);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
drawOverlay();
glPopAttrib();
drawRestOfScene();
}
And that works as I would expect the first snippet to.
Whilst it is working, I would very much like to know why.
Thank you.
you have to carefully read the glstencilfunc operating functions. ref & mask is compared to stencil & mask . that's it, ref & mask equal to 1, so that it can never be less than < stencil & mask, which equals to "at most" 1, GL_LESS will always fail because of your ref equals to 1. (and not a hex value like the mask you chose)
edit : just put "1" in place of 0xF. for convenience.
Sorry, I don't really understand what you're saying.
To begin with stencil is cleared to 0, the reference value is 1 and the mask is 0xFFFFFFFF (Or 1 you're quite right). So:
(0 & 1) < (1 & 1)
0 < 1
true
Then when the overlay has been drawn in that pixel, the stencil value becomes 1, so:
(1 & 1) < (1 & 1)
1 < 1
false
Where am I going wrong here?
Scrap this, I'm officially an idiot. I just reread the glStencilFunc docs as you suggested.
Turns out the reference value gets compared to the stencil buffer value whereas I assumed (because I'm an idiot) that stencil buffer value got compared to the reference value.
Ie. Its (ref & mask) < (stencil & mask)
not as I assumed (stencil & mask) < (ref & mask)
Somehow that seemed more intuitive to me.
Thank you for your answer.
oh yeah..
About the hex values, which I haven't tried anyway, they ought to be used when letting multiple stencils overlap, so that a stencil value like 0x00F00F will return something greater than ref 0x00000F & mask 0xFFFFFF... Thereby GL_LESS would make sense.. ':)
Ah, I see what you're getting at.
Whilst I know what all the bitwise operations do, and how they're often used and stuff, they still confuse the hell out of me.