help with GL_SELECTION and picking

[riflemon]

Hi y'all, I'm having a problem with GL_SELECTION render mode. My code is:

       selection.flip();

       IntBuffer viewport = ByteBuffer.allocateDirect(4*4).order(ByteOrder.nativeOrder() ).asIntBuffer();
       GL.glGetInteger(GL_VIEWPORT, viewport);

       GL.glSelectBuffer(selection);
       GL.glRenderMode(GL_SELECT);
       GL.glInitNames();
       GL.glPushName(0);
       GL.glMatrixMode(GL_PROJECTION);
       GL.glPushMatrix();
       GL.glLoadIdentity();
       GLU.gluPickMatrix(dm.width / 2,
               dm.height / 2,
               5,
               5,
               viewport);
       setPerspective();
       renderScene();
       GL.glPopName();
       GL.glMatrixMode(GL_PROJECTION);
       GL.glPopMatrix();
       GL.glFlush();
       int hits = GL.glRenderMode(GL_RENDER);

       if (hits == -1) {
           badSelections[0]++;
           int t = GL.glGetError();
           System.out.println("t = " + t);

       }
       if (hits == 0)
           badSelections[1]++;

       if (badSelections[0] % 50 == 0 ||
           badSelections[1] % 50 == 0) {
           System.out.print("badSelections (0) " + badSelections[0]);
           System.out.println(" (1) " + badSelections[1]);
       }

       if (hits > 0)
           System.out.println("hits = " + hits);
       int idx = 0;
       for (int i = 0; i < hits; i++) {
           int names = selection.get(idx);
           System.out.println("Hit(" + i + ") - Names.length = " + names);
           idx++;
           float zMin = Float.intBitsToFloat(selection.get(idx++) );
           System.out.println("zMin = " + zMin);
           float zMax = Float.intBitsToFloat(selection.get(idx++) );
           System.out.println("zMax = " + zMax);
           idx += names;
       }


       selectionEnabled = false;

I've defined selection as a buffer with 8192 ints (32768 bytes).
The output from the line:

   

int hits = GL.glRenderMode(GL_RENDER);

is -1, which is weird... GL.glGetError() returns GL_NO_ERROR

I'm getting 0 if I comment out the line:
   

GL.glRenderMode(GL_SELECT);

which makes sense; I comment it out with the purpose of seeing if my gluPickMatrix code is working correctly, which it is. Of course, I'm passing in names into the NameStack as well.

It strikes me as odd that GL.glRenderMode(GL_RENDER) should ever return -1... is this hardcoded or something? Anybody have any clue on this?

Thanks in advance,

JP

[Matzon]

It strikes me as odd that GL.glRenderMode(GL_RENDER) should ever return -1... is this hardcoded or something? Anybody have any clue on this?

as for your first question:

/*
 * Class:     org_lwjgl_opengl_GL11
 * Method:    glRenderMode
 */
JNIEXPORT jint JNICALL Java_org_lwjgl_opengl_GL11_glRenderMode(JNIEnv * env, jclass clazz, jint p0)
{
	jint ret = (jint) glRenderMode((GLint) p0);
	CHECK_GL_ERROR
	return ret;
}

as for your second question:

The return value of glRenderMode is determined by the render mode at the time glRenderMode is called, rather than by mode. The values returned for the three render modes are as follows:

GL_RENDER
   Zero.

GL_SELECT
   The number of hit records transferred to the select buffer.

GL_FEEDBACK
   The number of values (not vertices) transferred to the feedback buffer.

NOTES
If an error is generated, glRenderMode returns zero regardless of the current render mode.

so  would agree that -1 would be odd, however I don't have a clue for you :/

[riflemon]

not flipping the IntBuffer seems to work... once... I think I'll go on and read about buffers... maybe I should flip AFTER glRenderState()

[riflemon]

And thank you Matzon for the quick reply... it's almost 7 am here... time to call it a day.

[riflemon]

But before I called it a day, I fixed it. It was bad ByteBuffer code. Thing is tough, it shouldn't return -1, it should throw one of those nice JVM dumps... Where is the bugzilla for LWJGL?

[princec]

We're the bugzilla What is "bad ByteBuffer code"?

Cas

[riflemon]

First, I didn't instantiate the ByteBuffer and right away call glSelectBuffer. I didn't do nothing in particular with it, just initialized it in &clinit&
Second, I was calling glSelectBuffer every frame.
Third, I wasn't using glFinalize()
Fourth, I should have cleared the buffer before going into select mode (I was flipping)
Fifth, I should have been rewinding the buffer before accessing it (right after re-entering render mode).

I wasn't very familiar with the java.nio APIs; I'm probably coding it too much, but at least it's working now

[amoeba]

Anychance on posting the working code please?

Especially the PickBuffer creation code.

Cheers.