Ray Picking

[Daslee]

Hello. Does someone have ray picking code for lwjgl, with which I can select objects in 3d scene? I was searching 2 days in google, but which I was found didn't worked. Sometimes z position always stay in -2 and sometimes z position changes only between 0.9 and 1.0. All codes which I was found in google were not working. 

[quew8]

Are you talking about choosing an object you click on with the mouse? If so then what you want is opengls selection feature in which you: give each object to be rendered a unique name, assign a buffer for results to be written to, create a clip such that only the area around the mouse will be drawn to, switch to selection rendering mode, draw the scene and read back the result with the smallest depth value.
If this isn't what your talking about then it's just a matter of testing each object in the scene for intersection with the ray of which there are a huge number of methods like minkowski difference or defining faces algebraically and testing them that way. You could even use the selection mode and create a modelview matrix that "looks along" the ray. You have an infinite number of options.
If you would like to post the code you have which isn't working I'm sure we could also help you get that working.

[Daslee]

Are you talking about choosing an object you click on with the mouse? If so then what you want is opengls selection feature in which you: give each object to be rendered a unique name, assign a buffer for results to be written to, create a clip such that only the area around the mouse will be drawn to, switch to selection rendering mode, draw the scene and read back the result with the smallest depth value.

Yes, that is what I want to do, but can't because I'm newbie to LWJGL and I do not know many things such as buffers float buffers and etc, what they are and what they are doing... I would like to read LWJGL book, but don't have so much time. Mostly when I copy some code I'm playing around two days with it to get know which line what doing and then understanding, but when I watched java basics I didn't seen tutorials about things which is need now.

Code which I was using last time:

FloatBuffer mousePos = getOGLPos(Mouse.getX(), Mouse.getY());
selectObject(mousePos.get(0), mousePos.get(1), mousePos.get(2));

And getOGLPos method:

private FloatBuffer getOGLPos(int mouseX, int mouseY){
		IntBuffer viewport = BufferUtils.createIntBuffer(16);
		FloatBuffer modelview = BufferUtils.createFloatBuffer(16);
		FloatBuffer projection = BufferUtils.createFloatBuffer(16);
		FloatBuffer winZ = BufferUtils.createFloatBuffer(1);
		float winX, winY;
		FloatBuffer position = BufferUtils.createFloatBuffer(3);
		glGetFloat(GL_MODELVIEW_MATRIX, modelview);
		glGetFloat(GL_PROJECTION_MATRIX, projection);
		glGetInteger(GL_VIEWPORT, viewport);
		winX = (float)mouseX;
		winY = (float)viewport.get(3) - (float)mouseY;
		glReadPixels(mouseX, (int)winY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, winZ);
		gluUnProject(winX, winY, winZ.get(), modelview, projection, viewport, position);
		return position;
	}

And selectCube method:

public void selectObject(float mouseX, float mouseY, float mouseZ){
		for(Object obj : world.visibleCollidingObjects){
			if(mouseX > obj.x && mouseX < obj.x + obj.width){
				if(mouseY > obj.y && mouseY < obj.y + obj.height){
					if(mouseZ > obj.z && mouseZ < obj.z + obj.depth){
						obj.selected = true;
					}else{
						obj.selected = false;
					}
				}else{
					obj.selected = false;
				}
			}else{
				obj.selected = false;
			}
		}
	}

But maybe here I need to cast ray from player position to FloatBuffer mousePos?

Can't upload my video example to youtube, because upload speed slow.

[quew8]

First of all, buffers are very simple. Think of them as just a container of an array of values that we use to communicate with openGL. The method you have posted is not one I'm familiar with and after a quick look through I can't see any errors in the code however I cannot see how it would be very efficient and would advise you try out openGls selection mode for which I will write you some sample code. Be warned It is a long time since I have done this and there might be an error in my code.

public int getIndexOfSelectedObject(int mouseX, int mouseY) {
     int bufferSize = 128; //This number is the size of the buffer for openGl to write your results to. I chose 128 because I'm not sure how 
                                 //many objects your likely to be drawing. If the maximum number of objects potentially under the mouse is n then
                                 //the size should be n * (3 + sizeOfNameStack). In this example there will be 1 name per object therefore the 
                                 //name stack size is 1 and the size should be n * 4.
     IntBuffer resultBuffer = BufferUtils.createIntBuffer(bufferSize);
     glSelectBuffer(resultBuffer); //Tells openGl to write results to this buffer.
     glRenderMode(GL_SELECT); 
     
     glInitNames();  //Initialize openGLs name stack into which we will put the name of each object (i will use its index in your array)
     glPushName(0);  //Adds 1 to the size of the name stack (it starts off as 0).
     
     IntBuffer currentViewport = BufferUtils.createIntBuffer(4);
     glGetInteger(GL_VIEWPORT, currentViewport); //If you are storing the values you use for glViewport then you can put them in instead

     //You must have setup the correct projection matrix before you do the next line.
     FloatBuffer currentProjectionMatrix = BufferUtils.createFloatBuffer(16);
     glGetFloat(GL_PROJECTION_MATRIX, currentProjectionMatrix);

     glMatrixMode(GL_PROJECTION);
     glLoadIdentity();
     gluPickMatrix(mouseX, mouseY, 1, 1, currentViewport);
     glMultMatrix(currentProjectionMatrix);
     
     for(int i = 0; i < world.visibleCollidingObjects.length; i++) { // I'm presuming visibleCollidingObjects is an array and not one of the          
                                                                                    // collections
          glLoadName(i) //Replaces the name on the top of the name stack with the argument.
          world.visibleCollidingObjects[i].drawForSelection(); //You should write a new method that only gives position data, no normals, 
                                                                              // colours, textures etc.
     }
     
     int numberOfHitRecords = glRenderMode(GL_RENDER); //Switches back to regular mode and returns the number of hit records written.
                                                                             //If this returns a negative number you need to supply a larger selection buffer.
     //Each hit record contains: the number of names on the stack at the time (always 1 for us), the minimum z value of the object, the
     //maximum z value of the object, a list of the names on the stack (just the objects name for us).

     int minimumZValue = Integer.MAX_VALUE;
     int nameOfClosest = -1;
     for(int i = 0; i < numberOfHitRecords; i++) {
          int stackSize = resultBuffer.get(); //Unused by us. If not 1, somethings gone wrong.
          int thisMinimumZValue = resultBuffer.get();
          int maxZValue = resultBuffer.get(); //Unused by us.
          int name = resultBuffer.get();
          if(thisMinimumZValue < minimumZValue) { //This means this result is closer to the viewer that the previous and is therefore         
                                                                  // preferable.
               minimumZValue = thisMinimumZValue;
               nameOfClosest = name;
          }
          glMatrixMode(GL_PROJECTION); //These lines just return the projection matrix to what it was before. Not necessary but neater.
          glLoadMatrix(currentProjectionMatrix);
          return nameOfClosest ; //Which is the index of the selected object.
     }
}

I checked through this a few times, but as I say it's been a while since I last used this and I apologize in advance if I got something wrong.

[Daslee]

It works, but no so accurately. I'm not sure about this line: int bufferSize = 128;, for now I typed here my objects size in world. And got error here: IntBuffer currentViewport = BufferUtils.createIntBuffer(4);, but when I changed size to 16 then it's working.

Where could be the problem? Maybe you have skype? There we could chat better.

[quew8]

My apologies for the error, I had forgotten that LWJGL requires a buffer size of at least 16 for that method. About "bufferSize," it is the size of the buffer that openGL will write it's results into (4 numbers per result), it isn't all that important as long as it is large enough. Essentially this method just records any and all objects drawn at the point specified whether they are at the front of the scene or the back. Therefore the number of results is the number of objects drawn there even if they have been "drawn over." I can't really find the words to explain, if 128 is working (as in the second glRenderMode isn't returning a negative value) I would just leave it as that, else you can make it larger.
About the inaccuracy: I forgot to comment in the code about the gluPickMatrix(). The third and fourth are respectively width and height of the area to check in. If it is not picking objects when clicking on their edge, you could enlarge these (they are measured in window coordinates).

[Daslee]

It picks objects, but when I rotate camera pitch then it picks wrong. And when I am looking at object center, it picks object next one from that object (only if I'm looking far). I do not know how to actually explain all that thing what is wrong with picking, maybe I'll upload video to youtube when I can and show you here, or you can try to look at my game through teamviewer.

[abcdef]

Its normally recommended not to use the method above, ray picking is the way most people do it. I'm in the process of implementing this in what I am doing so I will post something later today to show how I get the ray vector. Once you have that its just a matter of checking against each object you render.

[Daslee]

Ok I'll be waiting. And why it's not recommended to use that method above?

quew8, I just noticed that when my camera yaw is rotated between 90-180 it works perfect, but when yaw is rotated greater than 180 or lesser than 90, it gets inaccurately.

[quew8]

Really sorry. I have just been doing some reading after seeing abcdef's post. It turns out selection and feedback are deprecated in opengl 3.0 and removed thereafter. It would seem that it is actually done on the CPU rather than the GPU with no hardware acceleration whatsoever.
Really sorry to have led you on this wild goose chase. And thank you abcdef for the correction.

[Daslee]

quew8, So your method is deprecated now? 

[quew8]

In opengl 3.0 it is deprecated and in all versions above 3.1, it is removed from the core profile.

[Daslee]

So what can be another way to pick objects? Maybe you have another code?

[abcdef]

I did the following, it gets you a picking ray (vector) which you then need to check against your render objects and then work out whats the closest to you

I haven't fully implemented mine yet but I wanted to show you the way I was approaching it

   public Vector3 getPickingRay(float cursorX,float cursorY)
    {    
        IntBuffer viewport = ByteBuffer.allocateDirect((Integer.SIZE/8)*16).order(ByteOrder.nativeOrder()).asIntBuffer();
        FloatBuffer modelview = ByteBuffer.allocateDirect((Float.SIZE/8)*16).order(ByteOrder.nativeOrder()).asFloatBuffer();
        FloatBuffer projection = ByteBuffer.allocateDirect((Float.SIZE/8)*16).order(ByteOrder.nativeOrder()).asFloatBuffer();
        FloatBuffer pickingRayBuffer = ByteBuffer.allocateDirect((Float.SIZE/8)*3).order(ByteOrder.nativeOrder()).asFloatBuffer();
        FloatBuffer zBuffer = ByteBuffer.allocateDirect((Float.SIZE/8)*1).order(ByteOrder.nativeOrder()).asFloatBuffer();
        glGetFloat(GL_MODELVIEW_MATRIX, modelview);
        glGetFloat(GL_PROJECTION_MATRIX, projection);
        glGetInteger(GL_VIEWPORT, viewport);
        float winX = (float) cursorX;
        // convert window coordinates to opengl coordinates (top left to bottom left for (0,0)
        float winY = (float) viewport.get(3) - (float) cursorY;
        
        // now unproject this to get the  vector in to the screen
        // take the frustrm and unproject in to the screen
        // frustrum has a near plane and a far plane
        
        // first the near vector
        gluUnProject(winX, winY,  0, modelview, projection, viewport, pickingRayBuffer);        
        Vector3 nearVector = new Vector3(pickingRayBuffer.get(0),pickingRayBuffer.get(1),pickingRayBuffer.get(2));
        
        pickingRayBuffer.rewind();
        
        // now the far vector
        gluUnProject(winX, winY,  1, modelview, projection, viewport, pickingRayBuffer);
        Vector3 farVector = new Vector3(pickingRayBuffer.get(0),pickingRayBuffer.get(1),pickingRayBuffer.get(2));
        
        //save the results in a vector, far-near
        return farVector.subtractVector(nearVector).normalise();
    }

if you are wondering why you use unproject with win z having values 0 and 1 then have a look at

http://www.opengl.org/wiki/GluProject_and_gluUnProject_code

It shows you the code for the unproject function and should help tell you why (0 maps to normalised z of -1 and 1 maps to a normalised value of 0)

[Daslee]

I did the following, it gets you a picking ray (vector) which you then need to check against your render objects and then work out whats the closest to you

I haven't fully implemented mine yet but I wanted to show you the way I was approaching it

   public Vector3 getPickingRay(float cursorX,float cursorY)
    {    
        IntBuffer viewport = ByteBuffer.allocateDirect((Integer.SIZE/8)*16).order(ByteOrder.nativeOrder()).asIntBuffer();
        FloatBuffer modelview = ByteBuffer.allocateDirect((Float.SIZE/8)*16).order(ByteOrder.nativeOrder()).asFloatBuffer();
        FloatBuffer projection = ByteBuffer.allocateDirect((Float.SIZE/8)*16).order(ByteOrder.nativeOrder()).asFloatBuffer();
        FloatBuffer pickingRayBuffer = ByteBuffer.allocateDirect((Float.SIZE/8)*3).order(ByteOrder.nativeOrder()).asFloatBuffer();
        FloatBuffer zBuffer = ByteBuffer.allocateDirect((Float.SIZE/8)*1).order(ByteOrder.nativeOrder()).asFloatBuffer();
        glGetFloat(GL_MODELVIEW_MATRIX, modelview);
        glGetFloat(GL_PROJECTION_MATRIX, projection);
        glGetInteger(GL_VIEWPORT, viewport);
        float winX = (float) cursorX;
        // convert window coordinates to opengl coordinates (top left to bottom left for (0,0)
        float winY = (float) viewport.get(3) - (float) cursorY;
        
        // now unproject this to get the  vector in to the screen
        // take the frustrm and unproject in to the screen
        // frustrum has a near plane and a far plane
        
        // first the near vector
        gluUnProject(winX, winY,  0, modelview, projection, viewport, pickingRayBuffer);        
        Vector3 nearVector = new Vector3(pickingRayBuffer.get(0),pickingRayBuffer.get(1),pickingRayBuffer.get(2));
        
        pickingRayBuffer.rewind();
        
        // now the far vector
        gluUnProject(winX, winY,  1, modelview, projection, viewport, pickingRayBuffer);
        Vector3 farVector = new Vector3(pickingRayBuffer.get(0),pickingRayBuffer.get(1),pickingRayBuffer.get(2));
        
        //save the results in a vector, far-near
        return farVector.subtractVector(nearVector).normalise();
    }

if you are wondering why you use unproject with win z having values 0 and 1 then have a look at

http://www.opengl.org/wiki/GluProject_and_gluUnProject_code

It shows you the code for the unproject function and should help tell you why (0 maps to normalised z of -1 and 1 maps to a normalised value of 0)

So with your code here I need to cast ray from camera position to return farVector.subtractVector(nearVector).normalise(); ? Or? Because now what I know from your code, so I got some coordinates and do not know what to do. If I need to cast ray, so ok I'll try to do something.