Issues with mouse cursor

[renanse]

Bug reported from one of the jME devs too shy to post... 

After moving from lwjgl 1.0 to 1.1.2, pixels in my mouse cursor that are alpha=0 and rgb all 255 result in XOR rendering of the hardware cursor.  Is this a bug or a feature?

[Matzon]

bit surprised about this one... I'll investigate...

[Matzon]

I'm seeing the exact same behaviour in 1.0 and 1.1.2

[petterm]

This problem is confirmed to exist when using jME and lwjgl - we're seeing it 100% of the time on a lot of machines with various graphics cards (mainly ATIs and NVIDIAs). We haven't seen a single case where alpha for the mouse cursor works since upgrading to jME 1.0 rc1 and lwjgl 1.1.2, so it doesn't seem to be very platform-dependant.

Any ideas what we can do to investigate why and where this problem happens?

[Matzon]

it's not a problem... it's a spec in the CreateIcon of Win32 API:
http://msdn2.microsoft.com/en-us/library/ms648059.aspx

AND bitmask   XOR bitmask   Display
0                      0              Black
0                      1             White
1                      0             Screen
1                      1         Reverse screen

[renanse]

Can you elaborate?  If it was working before but not anymore, is there some setting jME is using that you are now respecting...?

[Matzon]

I tried running the HWCursorTest in 1.0 and 1.1.2, modifying to paint a 0x00 alpha cursor - and it was all XOR, both in 1.0 and 1.1.2 - so I don't see the behavior as changed?
Furthermore I am not aware of any changes to that code...

I'll post a test tonight, unless you beat me to it.

[renanse]

I'm having a hard time understanding why a cursor with pixels set to R=255,G=255,B=255 and A=0 would result in reverse screen behavior.  (this seems illogical when changing just one of those RGB values to 254 makes it transparent.)  Just a special case to allow for reverse screen?

[Matzon]

from the Javadoc:

Constructs a new Cursor, with the given parameters. Mouse must have been created before you can create
    Cursor objects. Cursor images are in ARGB format, but only one bit transparancy is guaranteed to be supported.
     So to maximize portability, lwjgl applications should only create cursor images with 0x00 or 0xff as alpha values.
     The constructor will copy the images and delays, so there's no need to keep them around.

Therefore you should only use 0x00000000 or 0xffxxxxxxxx values. Unless you are explicitly creating cursors for an 8bit alpha platform.

Using HWCursorTest, I can confirm that rendering 0xfeffffff yields the exact same behavior in LWJGL 0.99, 1.0, 1.1, 1.1.2
That is to say, no regression has occured.

[renanse]

The alpha IS indeed always 0x00 or 0xff...  it is the OTHER values (RGB) that are causing the changes in behavior I am describing.

(Edit: Ok, sorry I see what you are saying here, but the javadoc does not say it must be 0x00000000)

[Matzon]

yeah - the docs could be a bit more specific. Can we agree on updating the javadocs to the following:
So to maximize portability, lwjgl applications should only create cursor images with alpha as 0xff and any RGB value (0xffrrggbb) or alpha as 0x00 and RGB also 0x00 (0x00000000).

Also, can we agree that this is not a regression?

[renanse]

Agreed.  My comments on "regression" are related to user complaints (such as the one above) so I have no personal experience to back them up anyhow - just observed current behavior.   In any case, jME has been updated in CVS to force 1bit pixels to 0x00000000 when the incoming alpha value is less than a certain threshold which should bring it into alignment with lwjgl.  Thanks for your help and comments.

[petterm]

And thanks from me as well, glad to have this one clarified - although I would like to propose another solution than just changing the documentation. My reasoning is:

- 0x00ffffff is as I understand it a documented (in the windows API) magic code telling a windows API method to do XOR when creating a mouse cursor, whereas on other platforms (or when viewing the image file in another context), it uses the alpha for transparency and disregards the rest of the data.
- even though this behaviour was documented in the windows API, we (and the jME devs) were unaware of the fact that lwjgl calls this particular windows method. I'd say that kind of leakage (the underlying Windows semantics affecting what a user of lwjgl, which is supposed to be platform-independent) should be stopped as early as feasible. Stopping it with code is far less error-prone than with documentation.

So in order to make lwjgl easier to use and behave in the same way cross-platform, I think that setting the ARGB value to 0x00000000 should be done within lwjgl rather than by lwjgl clients. An option might be to throw an exception if anything other than 0xffxxxxxxxx or 0x00000000 is passed to the Cursor constructor; that would help clients keep their code correct more easily.

Again, thanks for quickly solving this for us - whether you agree with my comments or not, we no longer have mouse cursors with XOR:ed background in our application, and that's the main thing. 

[Matzon]

So in order to make lwjgl easier to use and behave in the same way cross-platform, I think that setting the ARGB value to 0x00000000 should be done within lwjgl rather than by lwjgl clients. An option might be to throw an exception if anything other than 0xffxxxxxxxx or 0x00000000 is passed to the Cursor constructor; that would help clients keep their code correct more easily.

agreed, we'll update lwjgl to automatically make all cursors either 0xffxxxxxx or 0x00000000 on windows