[RFE] Mouse constraint APIs

[princec]

I'm getting a lot of hassle about the way we handle the mouse in our games. Brief history and explanation:

Up until quite recently we'd relied on plain old Mouse.setGrabbed(true) and completely captured the mouse, then relying on getDX()/DY() to update our own rendered mouse cursor. Unfortunately for reasons unexplained getDX()/DY() occasionally produce fairly crazy values and also they appear to be unrelated to the actual system mouse movement, which basically means the mouse does not behave in the same way as the user's desktop mouse and this is poo.

So in the end I've stopped grabbing the mouse, and instead set it to an almost invisible speck, and simply use Mouse.getX()/Y() to get the actual real mouse coordinates as it moves around on the window; this gives me perfect correlation with the user's desktop mouse speed and acceleration and calibration etc. Unfortunately the mouse can now happily leave the window; worse, on OSes other than Windows, it seems the mouse can even wander onto the other screens even when in fullscreen mode.

I propose some new APIs which fix the issue:

boolean Mouse.setClipped(boolean clip); /** Returns true if successful and clip = true, true if clip = false, and false otherwise */
boolean Mouse.isClipped(); /** Returns true if the mouse is genuinely clipped */

Which will physically constrain the mouse to the client (renderable) area of the Display window in both fullscreen and windowed mode.
The reason for the return value of boolean in setClipped() is in case there is no implementation on a particular OS. I thought an exception would be too annoying as it'd probably cause Windows developers to crash their games with an exception rather than just carry on with the minor inconvenience of not getting a constrained mouse.

I'd kill for this to be available in the next couple of weeks and there is a Small Financial Prize Awaiting whoever would like to implement it courtesy of Puppygames

Cas

[kappa]

Just by way of update, we discussed this on IRC yesterday and came to the conclusion that a new API isn't really needed, instead we should fix the existing API.

Basically when you use Mouse.setGrabbed(true), the mouse should be grabbed (invisible) but Mouse.getX() and Mouse.getY() values should continue to return like when the mouse is not grabbed (movement being at the same speed as normal OS mouse), the mouse can't leave the window until released (or the window is unfocused) and the values are clipped to the window. This behaviour should be perfect to satisfy the above RFE.

The good news is that this is already the behaviour on Linux. Mac requires some hackage to get it to match this behaviour but it should be doable (will have an attempt at it on the weekend).

On windows however the current Mouse grabbing implementation needs to be redone. Currently it grabs the mouse and moves it to the centre every update (which is also what it uses to calculate the getDX() and getDY() values). Mouse.getX() and Mouse.getY() do not work at all as needed by the RFE. Grabbing was probably implemented like this because back in the Windows 9x there were no decent API's to clip the mouse for you. WinAPI now has the ClipCursor API which seems like a good API to replace the current grabbing method and should allow Mouse.getX() and getY() to work as needed, with only Mouse.getDX() and getDY() needing to be rewritten.

So anyone up for attempting to implement the above mouse grabbing thing on windows?

[kappa]

Had a look at this on the mac today and found that the behaviour is already the same as that on linux, hence no changes are required. So essentially its the windows implementation that has broken behaviour on Mouse.getX() and getY() when the mouse is grabbed.

[princec]

Sounds good so far - sorry I haven't had time to dig in the Windows implementation yet - who wrote it originally?

Cas

[kappa]

If I was to guess from the commit logs, I'd say elias, he did alot of the heavy lifting.

[Matzon]

I briefly took a look at this (DOTA2 was in maintenance mode  ) and I basically couldn't reproduce the issue.

Using the MouseTest, I got valid and consistent mouse delta values all the time.

I realize that the current implementation may be unorthodox - but if it works, I don't see why we should change it?
Can anyone produce a reliable test that actually shows that the windows implementation is broken when using grabbed modes and getting x/y/dx/dy/dw values?

[princec]

I reliably can get it to not work. Furthermore it doesn't work at all with some mice, and definitely doesn't work at all with tablets.

Cas

[Matzon]

I reliably can get it to not work. Furthermore it doesn't work at all with some mice, and definitely doesn't work at all with tablets.

Care to back it up with a test  I can run?

I don't see how -not- centering will all of a sudden "fix" "some" mice?

Don't know about tablets in general ...

Sorry about being all tongue in cheek - but I just think we can break a lot of existing apps...

[princec]

In theory this shouldn't break anything - the behaviour will simply be the same as the Mac OS and Linux implementations.

The only test I can suggest running is an older version of Revenge of the Titans, which, er, I don't have. Or Droid Assault. I had to resort to the shitty hack of having a mouse speed option in the options screen - crap for starters - and even then, simply roving the mouse around in circles, which on the desktop produces a very reliable circular motion, causes random glitches and movements eventually causing the mouse tracking to wander off, and the occasional ridiculous spike.

Anyway: I've been testing this for years, and this is what it all boils down to. There's a nice new API for Windows since Win2k I believe which is what we should be using now; the current code is a 95 hack and basically does not work properly under a range of circumstances!

Cas

[princec]

So the hacky solution, that would work perfectly fine for me, is really quite trivial to implement. I just remove the mouse centering in Windows and it all behaves exactly as expected, except, of course, that when the mouse is grabbed, getDX() and getDY() no longer work properly. Which is kind of exactly when you want them to work. Any ideas how we might want to progress on that? I tried in vain to find a windows message that gives deltas but it seems such a message is not available.

Cas

[kappa]

hmm, without looking at too much of the windows code here is how I think it might/should/or already does (possibly with a bug) work provided that we are getting correct dx & dy values when the mouse is grabbed (i.e. windowCentre - MousePosition before being reset to centre again).

1) When the mouse is grabbed before it is centred the mouse getX() and getY() values (or the centre of the window) are saved in some variables (e.g. accum_dx and accum_dy).

2) Every time DX and DY are calculated the values are added to accum_dx and accum_dy.

3) The values of accum_dx and accum_dy are clipped to the windows size (0 to width, 0 to height).

4)  When Mouse.getX() or Mouse.getY() is called when the mouse is grabbed the accum_dx and accum_dy values are returned instead of the actual mouse position.

This should give the correct and desired position and behaviour even if the mouse is centred every update. If not then maybe there is a problem with the way the DX and DY values are calculated (maybe they are normalised or adjusted in some way).

[princec]

The code as it is would work perfectly fine... if the call to centre the mouse didn't reset the mouse acceleration parameters (as it seems to), and didn't rely on actual pixel movement (as it seems to). There's no two ways about it, the mouse centering approach has to go. But that means we need an alternative API for dx/dy when the mouse is captured on Windows. We used to use, I think, DirectInput for this, and it worked great. I'm not sure why it was switched away from using DirectInput to get delta values now.

Cas

[princec]

The more I look at the architecture of the mouse handling in LWJGL the more I despair of its opaque complexity

Cas

[kappa]

just a random brain fart for a quick fix, maybe the current code could be changed so that SetCursorPos is only called when the mouse x and y position actually changes (not centre of window) as opposed to centring every frame? might workaround the mouse acceleration reset a little.

[princec]

That's not really going to fix the tablet issue though is it.

Cas