[RFE] LWJGL 3.0

[spasi]

There's a WritableImage class in JavaFX since 2.2, which you can modify through a PixelWriter. PixelWriters accept ByteBuffers, so it's straightforward to do a ReadPixels and then directly update the JavaFX image. With the proper thread synchronization of course.

The JavaFX window draws the image, a JavaFX Canvas on top with some 2D shapes, then applies a reflection effect. It runs great on my machine, ~350 fps with -Djavafx.animation.fullspeed=true, perfectly smooth 60 fps at 10% CPU without it.

I'll try to use an offscreen context now with FBOs and double buffered read backs (with glGetTexImage).

[mattdesl]

I'd be happy to help with a utils library; I've already started working on a minimal one.

[kappa]

@spasi - if this offscreen stuff works out then LWJGL Core could be super streamlined, simple and slick, supporting just native Display and Offscreen mode.

We'd just need a small utility class using Offscreen mode to added support for any framework (would even work on the super problematic OS X AWT Applet Mode). All the AWT, AWTGLCanvas, Display.setParent() stuff could be thrown out of the core.

Main overhead to this approach is transferring the OpenGL texture data to the target framework but if this is small enough could be easy to over look (may even be possible to pass a pointer in some situations without a copy).

Offscreen mode could also be very useful in other situations too like rendering directly to a frame buffer on a Raspberry PI without needing to start X or even just used as a utility in games to Render to Texture.

[spasi]

OK, this looks very promising, at least for JavaFX. Results for a plain image in a JavaFX scene without anything else:

- 300x300: ~7500 fps (the size in the screenshot above)
- 720p: ~930 fps
- 1080p: ~370 fps

I created a 1x1 pbuffer as the offscreen context, an FBO for rendering and asynchronous PBO (double-buffered) for read-backs. Using Renderbuffers/ReadPixels is faster than render-to-texture/GetImage on my setup, but we can support either way. I used a CountDownLatch for synchronization between the render thread and the JavaFX application thread.

[jakj]

Getting rid of protected/private qualifications would be MASSIVELY good: It would be so much easier to alter small details or leverage components of the library without having to recompile the entire thing, use reflection, or copy/paste reams of code. Let the programmer have the burden of not screwing things up: If we muck about with things we don't understand, and it fails, then too bad for us.

It's fine to require a higher GL version, but I'd prefer the GL classes not be merged (or else, be merged but have the un-merged ones still available). The current GL11..GL43 naming system is useful for more than just targetting a GL version: It's also useful for keeping track of which versions of GL features are in, and walking back and forth through the specs in my head. It's also nice to be able to simply open up GLxx.java and grok through it for reminders without having to sort through dozens of pages of stuff.

Princec converted me: I also vote for super-streamlined bare-metal core with no added stuff, and the rest can be third-party additional layers.

I finally vote against changing the name from lwjgl, since my fingers have now become used to typing that and seeing it, and I dislike change.

[princec]

Changing the name of LWJGL will be a big mistake, IMHO.

Cas

[Matzon]

The name will not change.

[kappa]

@spasi looks like very good results, is it possible to use FBO's with an offscreen context without pbuffer involvement? (since pbuffers are depreciated).

Certainly looks equally trivial to implement for AWT using a VolatileImage or BufferedImage.

As both AWT and JavaFX have OpenGL pipelines (where detected) we could also investigate using a shared context and then with a bit of reflection voodoo simply replace the textureID to the OpenGL texture inside an AWT BufferedImage (or that of the equivalent in JavaFX). That way we avoid the copy overhead altogether, certainly looks plausible from a quick glance at the Java2D code.

[spasi]

is it possible to use FBO's with an offscreen context without pbuffer involvement? (since pbuffers are depreciated).

Yes, I think most people create a hidden window and create a normal GL context on that. This is not possible in LWJGL atm, but hopefully will be in 3.0. That way we can get rid of pbuffers altogether.

As both AWT and JavaFX have OpenGL pipelines (where detected) we could also investigate using a shared context and then with a bit of reflection voodoo

Voodoo is certainly possible, even without a GL pipeline. For example, JavaFX uses ByteBuffers internally for Image data. We could do a glMapBuffer and inject the returned buffer instance in JavaFX somehow, thus avoiding a copy when transferring data to/from GL.

I'm still working on this btw, but I'm focusing on getting the non-voodoo stuff right first. The streaming FBO is basically done and I'm now going the opposite direction, i.e. rendering a JavaFX node to an LWJGL texture. What I have now is Webkit textured on a quad, behind the 3D gears and all that displayed inside JavaFX.

I'll prepare a working demo and source bundle when it's complete.

[princec]

Here are some of the things I would like to see changed:

1. I'm pretty keen that we completely separate all context management from all OpenGL binding stuff. I would like, ideally, that we provide an extremely lightweight context management interface that would enable us to use contexts from AWT, SWT, NEWT, and even SDL. There was some vague attempt at that with the design (I seem to recall attempting to make GLContext an opaque "handle" back in the day).

2. I want to have removed basically all the C code that we possibly can remove and go for a more SWT-like design of absolutely lightweight wrappers to native method calls. Thus each "platform" would have a single monolithic class of system API in it eg. Win32API, Linux32API, MacOS107API, etc. which would contain all the native methods we could possibly want and the utter minimum of native logic. Then write all the code in Java, implementing some fancy interfaces in the "service provider" factory pattern.

3. Reimplementation of all keyboard/mouse input to use JInput and remove the native code that currently handles it.

4. Output packaging. Please change the distribution to package everything in a single zip download.


Here are some things I would like to stay:

1. The separation of GL11/GL12/etc.

2. I don't really want to deprecate GL11/12/13; they are no trouble at all to keep and maintain and a remarkable amount of tutorial code still uses this stuff, not to mention the fact that old-style drivers (pre 3.3?) are all based on the foundations in GL11 anyway. Save it for LWJGL4.0 when old-style OpenGL is not just deprecated but actually removed from everyone's computers.

3. I am still keen on the original Display interface: it is extremely simple and user-friendly to use. I think we should keep it but ensure that the underlying implementation is actually based on whatever newfangled better design we have. Point advanced users at the underlying implementations; point people writing games, and n00bs, at the simple Display interface.

4. Likewise I am keen on retaining Mouse and Keyboard classes doing what they do. Again, advanced people can use JInput directly. The rest of use don't need any more.


Here are some things I would like to go:

1. All of lwjgl_util.jar
2. Applets. They are dead. Anyone still daft enough to be using applets in this day and age can stick with 2.8.x.
3. All the proprietary vendor extensions that aren't in use by at least all three of NVidia, ATI and Intel.
4. Security context stuff. See #2. No point without applets.


A good first step is deciding on the pure interfaces required.


Cas

[kappa]

@spasi - Oh nice, that sounds like super cool stuff, look forward to it.

Currently the OS X Java 7 implementation has rather flaky (not even sure it still works properly) support for AWT as most methods are now native Cocoa (The AWT stuff is currently just left over from the previous implementation).

So rather than spending time trying to reimplement the AWT stuff for the LWJGL 2.9 release when its planned to be removed in 3.0, might be a better idea to strip all internal AWT use now and go pure Cocoa in readiness for 3.0.

We could then use your Offscreen to AWT stuff behind the current LWJGL OS X AWT API's to keep it backward compatible. Should serve as a nice testing ground while at the same time having the code ready (pending a bit of refactoring) for LWJGL 3.0.

[kappa]

@princec - replacing all input with JInput sounds like a cool idea (especially as it reduces the native code), however is JInput workable and good enough to replace input from the windowing system? does stuff like Keyboard localisation work? are there limitations of using JInput over input from the windowing system?.

If we do go with JInput for input I'd prefer the binaries be merged into the lwjgl ones rather than having to carrying around yet another jar and native for each platform, quite like the idea of the core being one jar, one native (for each platform). Might even be easier to just absorb JInput into lwjgl to allow consistent package names and allowing bugs to be fixed directly.

[jakj]

One jar and one native would be really nice from a development and distribution standpoint, especially for simple hobbyists like myself.

[lhkbob]

I was doubtful when you said pbuffers would go away, but as long as the windowing system is capable of a hidden window for an offscreen context, that meets my needs for context sharing.

I like the simplicity of having an optimized Display and then all other windowing systems are implemented on top of texture transfer.

[spasi]

I just posted a working demo of the JavaFX integration, on JGO for more visibility. Please test and report any issues.