Libretro porting of LWJGL (and Libgdx)

[msx]

Hi there, i'm trying to implement a Core for Libretro/Retroarch based on libgdx (eventually LWJGL). I asked for help on libgdx forum and they suggested posting here too.

Premise:

Libretro/Retroarch is a generic "game system" that can execute "cores". Each core is an emulator, or a virtual machine (like there's a "Mame" core, a "dosbox" core, etc Doesn't need to be an emulator too, some games are relased as "cores" ). The infrastructure provides for all window and input management, the core has to specify its desired resolution and has a callback method to render the scene (simplifying a lot). Audio, input, etc are unified for all cores. Cores are actually just dll/so with specific callback functions (obviously they can rely on external files). Most important callbacks are "init" (that inizialize the core) and "run" (which is called every frame and should render the scene).

Also, cores come in two variant: the classic variant draw graphics on a generic "ram buffer" that's then sent to the screen (like old 8 bit system), while the opengl variant is opengl-aware and receive an opengl framebuffer onto which to render. (this is becouse opengl is so ubuquitous that they consider skipping a layer of abstraction for performances).
The main gotcha of the opengl core is that you have to render not on the screen but on a framebuffer that is provided to you by libretro infrastructure (so that it can render its own gui on top etc). This shouldn't be a huge deal since, once bound, rendering on the framebuffer should be pretty transparent.

So what i want to do is develop a Core that can run "java games". It would be a c dll/so program that: starts an embedded jvm, load some java "middleware" to setup the system, and then load a jar with a to-be-defined "game format", wiring all callbacks from libretro to methods on a java object.

Now as a proof of concept i was able to do most of the stuff: i took a libretro gl-core demo, added JVM management, loading a jar and calling methods during libretro various backcalls, including the "run" call that's done on each frame.

The problem is: any call to Opengl functions crash

Now, inside the java callback we should already be in a context where we can draw: there's a framebuffer bound and rendering ongoing. For reference, this is the C portion of the libretro demo that makes the java call:

   ...
   ...
   glBindFramebuffer(RARCH_GL_FRAMEBUFFER, hw_render.get_current_framebuffer());

   glClearColor(0.3, 0.4, 0.5, 1.0);
   glViewport(0, 0, width, height);
   glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

   doJavaCall(); // here, the java side is called.

   glUseProgram(prog);

   glEnable(GL_DEPTH_TEST);
   ...
   ...

For the java side, i've tryed calling glClearColor and glClear again, with a different color to have something to actually verify that it's working. But as i said, it crashed.

Investigating, i ended up with the following code on the java side:

      System.out.println("****************CHIAMATO DOCALL***************");
      Configuration.DEBUG.set(true);
      System.out.println("Context: "+WGL.wglGetCurrentContext());
      System.out.println("Device: "+WGL.wglGetCurrentDC());
      System.out.println("lolTest: "+org.lwjgl.opengl.GL11.class); // ensure class has initializers done
      System.out.println("OK");
      
      String s = org.lwjgl.opengl.GL11.glGetString(org.lwjgl.opengl.GL11.GL_VERSION);
      System.out.println("Version is "+s);

Now this code, when i run retroarch, load the core and execute it, produces the following output:

calling doCall
****************CHIAMATO DOCALL***************
[LWJGL] Version: 3.2.3 SNAPSHOT
[LWJGL] OS: Windows 10 v10.0
[LWJGL] JRE: 13.0.1 amd64
[LWJGL] JVM: OpenJDK 64-Bit Client VM v13.0.1+9 by AdoptOpenJDK
[LWJGL] Loading JNI library: lwjgl
[LWJGL] Module: org.lwjgl
[LWJGL] Loaded from org.lwjgl.librarypath: C:\Users\niclugat\AppData\Local\Temp\lwjglniclugat\3.2.3-SNAPSHOT\lwjgl.dll
[LWJGL] Loading JNI library: lwjgl_opengl
[LWJGL] Module: org.lwjgl.opengl
[LWJGL] Loaded from org.lwjgl.librarypath: C:\Users\niclugat\AppData\Local\Temp\lwjglniclugat\3.2.3-SNAPSHOT\lwjgl_opengl.dll
[LWJGL] Loading library: opengl32
[LWJGL] Module: org.lwjgl.opengl
[LWJGL] opengl32.dll not found in org.lwjgl.librarypath=C:\Users\niclugat\AppData\Local\Temp\lwjglniclugat\3.2.3-SNAPSHOT
[LWJGL] Loaded from system paths: C:\WINDOWS\SYSTEM32\OPENGL32.dll
[LWJGL] Loading library: jemalloc
[LWJGL] Module: org.lwjgl.jemalloc
[LWJGL] Loaded from org.lwjgl.librarypath: C:\Users\niclugat\AppData\Local\Temp\lwjglniclugat\3.2.3-SNAPSHOT\jemalloc.dll
[LWJGL] MemoryUtil allocator: JEmallocAllocator
Context: 131072
Device: 251730347
lolTest: class org.lwjgl.opengl.GL11
OK

Now there's some output of LWJGL tellin that it's loading the library, and looks ok.
Then the two WGL calls actually works, and seems to tell that on the java side we actually ARE on a valid opengl context. After the "OK" log i've tryed many opengl calls and they all crash.

Now, since you guys are the wizards of dll loading, perhaps can help me.

My wild guess is that (since libretro is already up and running and lwjgl found all its natives) the java side is loading his own opengl dll and there's some kind of mismatch with what's already loaded and running on Libretro.

Also i found it strange that the WGL calls do indeed works, so i went to look at the sources and i'm under the impression that in LWJGL, WGL and GLxx uses two different approaches: WGL seems to be looking at function pointer and calling native "callPPI" style functions (this is black magic to me), while GL stuff seems to be using regular native functions like "static native void glClearColor" in GL11C. (can someone confirm this supposition?, Also, is there some docs on how all that magic works?)

If so, perhaps the first approach somehow bypass whatever issue is blocking the GL side.

So if anybody have some idea.. Does some "function pointer" styled GL wrapper exists ? Or can be created, perhapse just for glClearColor and glClear ?

[msx]

Ops i did quote instead of modify and created a new reply, sorry

[spasi]

Hey msx,

You're likely missing a call to GL.createCapabilities(). This will detect the OpenGL context that is current in the current thread and configure the LWJGL side of things accordingly. You should then be able to do normal OpenGL calls without issues.

Also note that GL.createCapabilities() is a heavyweight operation and should only be done once per context, not every time you enter from C to Java. If you're moving the context between threads, the returned GLCapabilities object can be reused by calling GL.setCapabilities().

WGL seems to be looking at function pointer and calling native "callPPI" style functions (this is black magic to me), while GL stuff seems to be using regular native functions like "static native void glClearColor" in GL11C.

OpenGL functions are called using static methods. However, OpenGL function addresses are specific to the context that is bound to the current thread. Normally, this would require a thread-local lookup on each method call, which is not terribly expensive, but it adds up and also affects the optimization of surrounding code. Unfortunately, Hotspot cannot hoist the lookup outside of loops, etc. That's why LWJGL handles OpenGL calls differently, to avoid the thread-local lookup completely without sacrificing correctness. This is indeed black magic (i.e. nasty hack that messes with JVM internals), see the ThreadLocalUtil class for more details.

[msx]

Thanks, i'll try that GL.createCapabilities() right away.

Yesterday i made a small progress with this code, that worked correclty:

funcAddr = APIUtil.apiGetFunctionAddress(GL.getFunctionProvider(), "glClearColor");
JNI.callV(funcAddr, 0f, 1f, 0f, 0f);

I was already dreading rewriting all GLxx classes with that style of calls.
I'm not sure i understood that threadlocal lookup things. All opengl calls i used with "apiGetFunctionAddress" and "JNI" are working even if i store the address on a field. Probably the context is not changed and it's always the same thread.

[msx]

Holy cow it worked! You were right, huge thanks!


Here's the (probably) first time a java Core runs on RetroArch:

[msx]

OpenGL functions are called using static methods. However, OpenGL function addresses are specific to the context that is bound to the current thread. Normally, this would require a thread-local lookup on each method call, which is not terribly expensive, but it adds up and also affects the optimization of surrounding code. Unfortunately, Hotspot cannot hoist the lookup outside of loops, etc. That's why LWJGL handles OpenGL calls differently, to avoid the thread-local lookup completely without sacrificing correctness. This is indeed black magic (i.e. nasty hack that messes with JVM internals), see the ThreadLocalUtil class for more details.

Uhm i've tryed to look into the ThreadLocalUtil description but haven't understand much.

it says:

1) The first capabilities instance encountered is stored in a static write-once holder. This is optimistically assumed to be compatible with all other (if
    any) instances we encounter, which is indeed the case for the vast majority of programs. If an incompatible instance is encountered, there is a fall back
    to the thread-local lookup.

So do you think i'm falling on the regular use case or in the fallback? The fallback encounters the performances penality you mentioned early? Am i actually using opengl32.dll that's loaded in java or not?

Anyway, i digged a little more on the Libretro documentations, and it turns out they give you a

retro_hw_get_proc_address_t

lookup function:

A callback to grab OpenGL symbols is exposed via retro_hw_get_proc_address_t. Use this to retrieve symbols and extensions.

So would it be teoretically possible to use that function to get all OpenGL function pointers and then call them via the "callPPI" style, avoiding the need to load any external DLL ? What would the performances be?


All of this becouse my core runs perfectly the first time, but after it's unloaded and reloaded by the frontend, everything crashes. I'm thinking it's some kind of DLL mess up.

Also, if that matters the system will be single threaded (or at lease, all opengl calls will originate from the same thread).

Thanks again.

[spasi]

You shouldn't need to understand how TLU works. Or use libretro's get_proc_address function. If you use GL.createCapabilities and GL.setCapabilities properly, it should work without any issues. The crash you describe sounds like an OpenGL context mismanagement issue. If libretro has a context management API, you should look into that.