Retrieving Raw Mouse Input with GLFW

[CJ Burkey]

I saw at this GLFW issue and this GitHub Milestone that GLFW now includes a method for retrieving the raw mouse delta in GLFW 3.3, which is what LWJGL 3 uses, and was wondering how exactly to use that information. Currently, I use a disabled cursor and calculate delta from the cursor movement callback, but that leads to "clicks" rather than smooth motion, even with a smoothdamp method (stolen from Unity, shh). I know this doesn't have to do with the DPI of my actual mouse as other games and game engines are able to use smoother mouse movement, but it seems with LWJGL there's something I'm missing. Has this been implements in LWJGL 3 yet? I'm using 3.1.7-SNAPSHOT.

[KaiHH]

This will likely depend on the OS you are using.
On Windows, GLFW uses WM_INPUT which will return the unaccelerated mouse motion without Windows applying any "ballistics"/acceleration to the mouse input (what you control via the pointer acceleration dial in the mouse controls dialog).
Basically, this will fire a cursor position callback invocation whenever your mouse registers that it was moved.
I tested that it is indeed working on Windows 10 with dialing the cursor acceleration ALL the way back to the left, so that I really have to move the mouse a few centimeters before the cursor moved even one pixel. With that and cursor mode disabled in GLFW, the mouse cursor callback still fired many many times for the same movement.
But it still will give you only integer numbers (such as 1024.0 or 781.0) and never fractional numbers. Probably this is what leads to your jittery movement in the game.

[josephdoss]

I use a mix of java's mouse logic and glfw. Hopefully you can see what's going on.

class
{
int WIDTH = 1200;
int HEIGHT = 800;

int mouseCenterX = WIDTH / 2;
int mouseCenterY = HEIGHT / 2;


gameLoop()
{

		DoubleBuffer mouseX = BufferUtils.createDoubleBuffer(1);
		DoubleBuffer mouseY = BufferUtils.createDoubleBuffer(1);
		int newMouseX = -1;
		int newMouseY = -1;

		glfwSetInputMode(window, GLFW_CURSOR, GLFW_CURSOR_HIDDEN);

		glfwGetCursorPos(window, mouseX, mouseY);
		mouseX.rewind();
		mouseY.rewind();

		newMouseX = (int) mouseX.get(0);
		newMouseY = (int) mouseY.get(0);

		if (newMouseX < mouseCenterX - 10) {
			p.faceLeft(); // tell the player to turn left
			newMouseX = mouseCenterX;
		}

		if (newMouseX > mouseCenterX + 10) {
			p.faceRight(); // tell the player to turn right
			newMouseX = mouseCenterX;
		}

		if (newMouseY < mouseCenterY - 10) {
			newMouseY = mouseCenterY; // up and down but I don't use these yet
		}

		if (newMouseY > mouseCenterY + 10) {
			newMouseY = mouseCenterY;
		}

		glfwSetCursorPos(window, mouseCenterX, mouseCenterY);

}

}

// to put the mouse back on the screen glfwSetInputMode(window, GLFW_CURSOR, GLFW_CURSOR_NORMAL);
// my MVP matrix uses the player's left/right rotation to determine the view, so move the player and you move the camera

[KaiHH]

@josephdoss remember that for raw input to work you have to use disabled cursor mode in GLFW. Otherwise you will get accelerated cursor positions (at least on Windows).

[josephdoss]

Thanks Kaihh.

I'm developing on Linux where it's working pretty good. I yanked a bunch of code out to clean up that snippet above, so I probably took out what you're talking about.

Thanks man,
JD

[CJ Burkey]

This will likely depend on the OS you are using.
On Windows, GLFW uses WM_INPUT which will return the unaccelerated mouse motion without Windows applying any "ballistics"/acceleration to the mouse input (what you control via the pointer acceleration dial in the mouse controls dialog).
Basically, this will fire a cursor position callback invocation whenever your mouse registers that it was moved.
I tested that it is indeed working on Windows 10 with dialing the cursor acceleration ALL the way back to the left, so that I really have to move the mouse a few centimeters before the cursor moved even one pixel. With that and cursor mode disabled in GLFW, the mouse cursor callback still fired many many times for the same movement.
But it still will give you only integer numbers (such as 1024.0 or 781.0) and never fractional numbers. Probably this is what leads to your jittery movement in the game.

So the callback only returns integers, even though it requires a double? Is there a way to get more precise information with callbacks, or will I have to query the mouse position rather than wait for input?

[KaiHH]

It does not really matter whether those coordinates are integers or fractional numbers. They just have a precision, which by default in GLFW is 1.0. It could have been 0.1 or 0.01 or 0.5. It does not matter. Whenever the mouse moves along the X or Y axis in the smallest measurable distance possible, then that value will be 1.0 in GLFW. Now it is your responsibility to map those units into any other metric useful for your game.
For example, all games provide a mouse sensitivity setting, which essentially just sets a factor to multiply the mouse delta coordinates with in order to accelerate or decelerate the motion.

[CJ Burkey]

It does not really matter whether those coordinates are integers or fractional numbers. They just have a precision, which by default in GLFW is 1.0. It could have been 0.1 or 0.01 or 0.5. It does not matter. Whenever the mouse moves along the X or Y axis in the smallest measurable distance possible, then that value will be 1.0 in GLFW. Now it is your responsibility to map those units into any other metric useful for your game.
For example, all games provide a mouse sensitivity setting, which essentially just sets a factor to multiply the mouse delta coordinates with in order to accelerate or decelerate the motion.

That is what I do, but I don't see the clicking in any other games, they all have a very precise mousse movement system. The system I currently use, however, Moves in pretty large "clicks," this GIF shows what I mean (there is a cube in the center to show the effect) (Also, I have the movement smoothed, so it looks a little nicer):



Here's my delta calculation code:

glfwSetCursorPosCallback(window.getId(), (win, x, y) -> onMouseMove(x, y));

...

private static void onMouseMove(double x, double y) {
	prevMousePos.set(currMousePos);
	currMousePos.set((float) x, (float) y);
	if (firstMouse) {
		prevMousePos.set(currMousePos);
		mouseDelta.set(0.0f, 0.0f);
		firstMouse = false;
		return;
	}
	mouseDelta.set(currMousePos.x - prevMousePos.x, currMousePos.y - prevMousePos.y);
}

[KaiHH]

It all depends on what metric you choose to map the mouse delta values to in-game units. Obviously, you chose a metric that says something like: 1 mouse unit equals some 20 pixels or so.
Simply change the mapping of both metrics to suit your needs better.
Just one thing to make sure: You do get steps of +1.0 or -1.0 in the glfwSetCursorPosCallback's x and y arguments when you very slightly move the mouse in either X or Y direction, do you? It's not that the callback itself will give you very large changes in X and Y, or does it? You sadly did not show the raw glfwSetCursorPosCallback callback argument values when you moved the mouse.
That would obviously be extremely helpful.

[CJ Burkey]

It all depends on what metric you choose to map the mouse delta values to in-game units. Obviously, you chose a metric that says something like: 1 mouse unit equals some 20 pixels or so.
Simply change the mapping of both metrics to suit your needs better.
Just one thing to make sure: You do get steps of +1.0 or -1.0 in the glfwSetCursorPosCallback's x and y arguments when you very slightly move the mouse in either X or Y direction, do you? It's not that the callback itself will give you very large changes in X and Y, or does it? You sadly did not show the raw glfwSetCursorPosCallback callback argument values when you moved the mouse.
That would obviously be extremely helpful.

Ah, no, that is handled elsewhere, here it is:

rotChange.set(Input.getMouseDelta().y, Input.getMouseDelta().x);
rotChange.mul(sensitivity * (float) Time.getPureDeltaTimeF());
goalRotation.add(rotChange);
goalRotation.x = Util.clamp(goalRotation.x, -90.0f, 90.0f);

So, goal rotation is the final rotation, Input.getMouseDelta() returns the mouseDelta from the previous post. I'm not sure if I should be multiplying by delta time, but if I don't, it's far too sensitive if vsync is off. The sensitivity is 120, but still a little slow

[KaiHH]

Again my question: Does the callback given in glfwSetCursorPosCallback return to you values offset by +1.0 or -1.0 in X or Y when you very slightly move the mouse in either of those directions or does it give you larger deltas?

I'm not sure if I should be multiplying by delta time

You of course do not multiply by elapsed delta time. You are processing absolute user input and are not integrating some parameterized equation over time, such as position = dt * velocity.

[CJ Burkey]

Again my question: Does the callback given in glfwSetCursorPosCallback return to you values offset by +1.0 or -1.0 in X or Y when you very slightly move the mouse in either of those directions or does it give you larger deltas?

I'm not sure if I should be multiplying by delta time

You of course do not multiply by elapsed delta time. You are processing absolute user input and are not integrating some parameterized equation over time, such as position = dt * velocity.

Yes, the delta outputs -1.0 to 1.0. I've also just tried removing the multiplication by the delta time (and decreasing sensitivity to 2.0 so nothing much changes). It seems when disabling v-sync, the clicking stops (and sensitivity must be decreased by a ton), so when the fps is higher, the value ends up being more precise, which makes sense. Is there a way to get the position of the mouse more often than once per frame?

[KaiHH]

Is there a way to get the position of the mouse more often than once per frame?

Yes, you can decouple window message event processing (glfwPollEvents() / glfwWaitEvents()) from your rendering loop with the following constraint:
- glfwPollEvents/glfwWaitEvents MUST happen on the main thread (i.e. the thread which called your public static void main(String[] args) method

You are free to call OpenGL methods and glfwSwapBuffers() in any thread you want. Take care that you also call glfwMakeContextCurrent() and LWJGL's GL.createCapabilities() in that thread as well!

[CJ Burkey]

Is there a way to get the position of the mouse more often than once per frame?

Yes, you can decouple window message event processing (glfwPollEvents() / glfwWaitEvents()) from your rendering loop with the following constraint:
- glfwPollEvents/glfwWaitEvents MUST happen on the main thread (i.e. the thread which called your public static void main(String[] args) method

You are free to call OpenGL methods and glfwSwapBuffers() in any thread you want. Take care that you also call glfwMakeContextCurrent() and LWJGL's GL.createCapabilities() in that thread as well!

Well, that sounds good, thanks for the help!