LWJGLX - LWJGL2 Compatibility Layer for LWJGL3

[kappa]

This aim of this code is to enable running LWJGL2 applications on LWJGL3 with minimal modification. It can be used to test how LWJGL2 projects might run on LWJGL3 or used to gradually migrate an existing codebase over to LWJGL3 (by replacing LWJGLX calls with direct LWJGL3 code until there are none left).

So far large parts of the LWJGL2 Display, Mouse, Keyboard & Sys class are implemented. Various other classes are also implemented including parts of LWJGL2's util package (vector, matrices, GLU, etc) and various placeholder classes for the parts of LWJGL2 that are not going to be implemented (allows code compilation without compile errors). I've found that adding API functionality only when needed is the easier way to go rather than than trying to implement the whole API.

In terms of the OpenGL and OpenAL calls the emphasis is to use them directly from LWJGL 3 as they are almost the same and only provide LWJGL2 like replacement classes and methods where they are missing or no longer compatible with the LWJGL2 API.

Whats missing at the moment (but intended to be added) is, notably fullscreen support from the Display class, changing the window icon and mouse cursor and some other miscellaneous bits.

At this stage I don't intend to spend anytime on the JInput, OpenCL and AWT parts of LWJGL2. AWT support should be doable and probably done by using a hidden GLFW window and somehow copying its contents to an AWT surface.

There are some limitations that can't be implemented with GLFW such as changing some window settings without destroying and creating a new window (which was supported for some functions in LWJGL2 like resizing). Also in LWJGL3 the opengl framebuffer size and window size are no longer guaranteed to be the same (especially on OS X), I've added the Display.getFramebufferWidth()/Height() methods to assist with working round this. Another limitation is that GLFW requires the windowing code to run on the main thread, noticed some projects using LWJGL2 were running its code on another thread so they may run into issues.

Easiest way to use the code at the moment is to download the source code and drop it into your project rather than using it as a 3rd party library, that way you can easily see whats going on behind the code and replace bits as needed. You then just replace the import calls for all the non opengl and openal methods from "import org.lwjgl.*" to "import org.lwjglx.*"

The code is currently located here

Some screenshots of projects i've tried:

jMonkeyEngine3 & LWJake2

[kappa]

Thought I'd post a guide on how to run Minecraft on LWJGL3 using the above.

The actual MC source code is not available but there is a nice tool called MCP (really impressive what its able to provide modders, using some really cool magic). ATM (as far as I'm aware) the max version it supports is MC 1.7.10 so that is what we will use below.

A. Setting up in Eclipse
1) Using MCP 9.08 for MC 1.7.10 - decompile MC & open in Eclipse
2) Fix the build errors on the build path, namely remove realms*.jar and authlib*.jar and add the new versions of the same (right click 'Client' -> properties -> libraries tab)
3) You should now be able to run Minecraft in Eclipse (will be using LWJGL2)

B. Add Paulscode Sound System LWJGL Plugin
Since this library plugin which MC depends on uses LWJGL2 it also needs to be ported (source is only 3 classes)
1) Remove the librarylwjglopenal20100824.jar from the build path.
2) Grab the source code for the lwjgl plugin from paulscode site here
3) Copy the contents of the src folder from the above zip into the src folder of the Client project in Eclipse.
MC should continue to run as before at this point

C. Replacing LWJGL
1) Remove lwjgl-2*.jar and lwjgl_util-2*.jar from the build path.
2) Remove/Delete the LWJGL2 natives from the folder Client->jars->versions->1.7.10->1.7.10-natives folder (just the lwjgl and openal natives)
3) Grab the latest nightly build of LWJGL3 from lwjgl.org
4) Add LWJGL3's lwjgl.jar to the build path
5) Copy the relevant LWJGL3 natives to the same location where the LWJGL2 natives were (Client folder jars/versions/1.7.10/1.7.10-natives). This is because the native path is already set to this location and contains the other natives MC uses.

D. Updating the Source
You could manually replace all the non opengl and openal imports from "import org.lwjgl.*" to "import org.lwjglx.*", however since there are so many references we can use some regexp magic to do it automatically for us.

1) Open Eclipses file search dialog (Ctrl + H and click the file search tab)
2) Enter the following regexp in the 'Containing Text' box:

(import org.lwjgl)(.BufferUtils;)|(import org.lwjgl)(.LWJGLException;)|(import org.lwjgl)(.Sys;)|(import org.lwjgl)(.opengl.ContextCapabilities;)|(import org.lwjgl)(.opengl.Display;)|(import org.lwjgl)(.opengl.DisplayMode;)|(import org.lwjgl)(.opengl.GLContext;)|(import org.lwjgl)(.opengl.OpenGLException;)|(import org.lwjgl)(.opengl.PixelFormat;)|(import org.lwjgl)(.openal.AL;)|(import org.lwjgl)(.input.Keyboard;)|(import org.lwjgl)(.input.Mouse;)|(import org.lwjgl)(.util.glu.GLU;)|(import org.lwjgl)(.util.glu.Project;)

3) make sure you tick the regular expression check box and in file name patterns choose *.java files only
4) click the 'Replace...' button and then in the 'With' box enter the following regexp and click OK

$1$3$5$7$9$11$13$15$17$19$21$23$25$27x$2$4$6$8$10$12$14$16$18$20$22$24$26$28

5) That should update all the relevant imports

E. Adding LWJGLX and final fixes
1) Grab/download the source code for LWJGLX from here and copy the contents its src folder into the src folder of the Client project
2) The above should fix all errors in the project except for two lines in Minecraft's OpenGlHelper class
3) On the first error line replace the "ARBShaderObjects" part with "org.lwjglx.opengl.ARBShaderObjectsx"
4) On the second error line replace the "GL20" part with "org.lwjglx.opengl.GL20x"
All errors should be fixed and MC should now run using LWJGL3.

Issues you may encounter
- On systems running in HiDPI resolutions (especially OS X), you may get a quarter sized screen, can be fixed by amending some of the glViewport calls to use Display.getFramebufferWidth() and Display.getFramebufferHeight().

- The textures on the ingame terrain blocks are black, not sure what the cause of this is yet, since not familiar with how its rendered in MC and what opengl calls the terrain code uses (and harder to isolate with decompiled code). I suspect it might be a LWJGL3 issue because we've only changed the windowing code and not touched any of the OpenGL calls. Another possibility is that there has been some API change in one of the LWJGL3 OpenGL calls but the method inputs have remained the same as LWJGL2 hence no errors were shown at compile time causing the output to be different.

[Grum]

I'll take a look in a bit to see what it requires to make this work for Minecraft. Need to do some packaging of libs first though

Oh! Also, awesome stuffs! I did already clone it some days ago

[spasi]

4) Add LWJGL3's lwjgl.jar and disruptor.jar to the build path

Disruptor is not necessary anymore with the latest builds. It will soon be removed from the dependencies.

Another possibility is that there has been some API change in one of the LWJGL3 OpenGL calls but the method inputs have remained the same as LWJGL2 hence no errors were shown at compile time causing the output to be different.

If this is the case, it's a serious issue that I'd very much like to resolve. If anyone figures it out, please let me know.

[Grum]

Having quite some issues at the moment to make this work on the latest commits on my mac.

It seems to demand that I run java with -XstartOnFirstThread which is fine. After doing this it seems to just hang. It took me a little while to figure out it actually created a windows *behind* my editor that was unfocused and didn't create any icon (or anything) in the Dock (taskbar).

It seems to often hang on lwjglx's Sys.getTime() and just not using the high-res timer at least makes it not lock up straight away (just using System.currentTimeMillis() instead).

The next problem I've not been able to solve, I am unable to keep focus in the window at all, none of the callbacks for input are being called and moving the window. Resizing works but there are no icon-hints when hovering the edges (so its quite the guess when you can drag it) so at least some callbacks seem to function.

Running jdk1.7.0_55.jdk from osx 10.9.5, for lwjgl3 I'm using (which i prepared myself from a nightly): 'org.lwjgl.lwjgl:lwjgl:3.0.0-nightly-20150122' and the latest lwjglx is included in the project.

Any help on why the whole window seems to refuse to get focus (glfwWindowHint(GLFW_FOCUSED, GL_TRUE); also didn't work) and thus no input would be much appreciated

[kappa]

Yeh theres been some changes in latest nightly builds of LWJGL 3, so you'll have to use -XstartOnFirstThread on OS X and if there is any AWT usage (which there is) you should use -Djava.awt.headless=true otherwise you'll get a hang.

[Grum]

I just tried adding: -Djava.awt.headless=true
This just causes a hard hang when lwjglx.Display.create() calls GLFW.nglfwCreateWindow()

Removing it gets me past the creation of the window but I have the same issues with focus, no dock icon (and I mean nothing at all shows up, its completely 100% hidden), refusing to take any input (all callbacks log to stdout and do not get called at all).

It seems that for OSX the window simply doesn't exist, quite baffling :/

I'm going to try to setup the simple example on the site and see if I can get the callbacks & normal-window-behavior working there.

[kappa]

you using java 6 or java 7?

[Grum]

Currently using jdk1.7.0_55.jdk @ osx 10.9.5 on a MacBookPro11,3.

[edit1]
It seems the window works fine when created using the code of http://www.lwjgl.org/guide. I really do not get what is going on :/

[edit2]
The LWJGLX Gears demo works, the LWJGLX SpaceInvaders demo doesn't (it hangs after showing the window with a black screen, no errors).

[kappa]

The hang/freeze is most likely due to the AWT loop starting up on the first thread which in recent LWJGL3 builds (on OS X) is now also used by GLFW.

You could give an older version of LWJGL3 a try (from here) before the XstartOnFirstThread changes were made (do remember to remove the -XStartOneFirstThread vm argument).

Its a messy situation due to how AWT was implemented on the OS X JVM as by default it reserves the first thread for AWT which is also what GLFW needs. AWT usage is still pretty widespread so even if -XstartOnFirstThread is used to get the first thread for GLFW, any small reference to AWT will trigger AWT's own loop to start thus causing a hang/freeze. Not sure how spasi had hacked older versions of LWJGL3 nightly builds to work on OS X without running on the first thread.

Going forward the best option would be to start moving away from and phasing out AWT dependencies, its a pretty large and old library now and probably soon to be deprecated by Oracle. Most of the AWT usage in Minecraft isn't really needed or can easily be substituted by lighter and faster alternatives.

[Grum]

Ah that explains why the asteroid demo hangs.

We're using AWT for:

• clipboard access - glfw provides!
• opening local folders - AWT only or gross non-cross-os-shellout
• opening webpages - JavaFX's HostServices can do this but you need to bootstrap a full javafx app
• reading pngs - JavaFX
• writing pngs - AWT only
• HSBtoRGB - trivial to do
• server gui - why do we even have one :/

[edit]
I'm guessing javafx is going to have replacements for those things. Does that one also get special threatment with threads?

[edit2]
Amazingly the only oracle provided way to write png files is through ImageIO ... which loads the whole awt chain :/

[edit3]
So hey, maybe lwjgl3 could add libpng? and perhaps allow opening a URI (file/folder/webpage) using whatever the OS has registered for it?

[kappa]

Going forward, the System JRE's that Java apps have relied on for so long are going to become pretty rare (already have to an extent) and most java apps will have to ship with a stripped down jre (project jigsaw will be pretty useful here). People will want to avoid bundling large frameworks/libraries like AWT in order to keep jre size down.

I'm guessing javafx is going to have replacements for those things. Does that one also get special threatment with threads?

While JavaFX is cool, in many ways its similar to AWT and pretty large. Don't think it requires special treatment with thread on OS X but haven't tested it yet.

[edit2]
Amazingly the only oracle provided way to write png files is through ImageIO ... which loads the whole awt chain :/
[edit3]
So hey, maybe lwjgl3 could add libpng? and perhaps allow opening a URI (file/folder/webpage) using whatever the OS has registered for it?

LWJGL2 had an API to open URI webpages but don't think its implemented in LWJGL3 or GLFW yet. MatthiasM's set of pure java image decoders for PNG (PNGDecoder), JPEG, TGA and BMP are pretty cool, small, fast (as they decode straight into an opengl usable format), use pure java and have a good free licence (BSD). A binding to something like libpng could be cool or some other native library that support more texture formats (think spasi already has some plans for this).

We're using AWT for:

• clipboard access - glfw provides!
• opening local folders - AWT only or gross non-cross-os-shellout
• opening webpages - JavaFX's HostServices can do this but you need to bootstrap a full javafx app
• reading pngs - JavaFX
• writing pngs - AWT only
• HSBtoRGB - trivial to do
• server gui - why do we even have one :/

Having reviewed a lot of the java gui libraries out there, SWT seems like a really good choice now when considered for use with a stripped JRE. Its small (only a couple of megabytes), actively maintained (used by Eclipse, Vuze, etc), uses native gui components and has all the features above (including browser and image reading/writing). The Avian VM examples are like 1mb (includes the VM) using SWT and showcase the full kitchen sink of GUI features, which is really impressive.

[Mickelukas]

I just picked up LWJGL3 and this library helped a whole lot with getting a jump start instead of having to learn everything about GLFW. Thank you so much for creating it!

[Mickelukas]

Going with the latest nightly build the library stopped working as the GLContext is no longer at org.lwjgl.opengl.GLContext and it cannot find the org.lwjgl.opengl.ContextCapabilities either. Where is the context now? I searched through the github but couldn't find a single trace of it.

[spasi]

See this topic.