The last planned API refactoring for LWJGL 3.0 has just been completed (https://github.com/LWJGL/lwjgl3/commit/cc1345592f6b03b26c7f48e64f7cac295456e649). It's a major overhaul of structs support and you can try it out with nightly build 3.0.0b #39. I will describe the changes in detail in a following post.
This topic is a request for feedback on the general LWJGL 3 API. The API will officially freeze with the 3.0 release, but I would like the upcoming beta release to be very very close to the final API. I plan to more actively promote LWJGL 3 ahead of the official release and that can only happen if any code/tutorials/videos posted don't become obsolete until then. So, if you have anything to contribute, please speak now:
- Is there anything in the current API hard to use?
- Is there anything in the current API hard to understand?
- Is there anything in the current API ugly or unconventional?
- Is there an implementation detail that you want exposed?
- Is there something complex that you want simplified?
- Do you need more options/configuration?
- Do you frequently get bugs or crashes when using a particular binding or API?
I would prefer any requests or suggestions to be accompanied by example code. Harsh (but constructive) criticism is welcome, you won't hurt my feelings! :)
If this topic stays idle for more than a few days, with a minimum of a week, I will release the 3.0 beta.
There was once a thread about refactoring the GLContext/GLCapabilities classes. Back then I was fine with things turned out to be, but now I somehow am uncomfortable with the way to make the GL context current in different threads.
Provided, there is a way to make the native GL context current in a calling thread, be it glfwMakeContextCurrent or SWT GLCanvas.setCurrent, or manually invoking wglMakeCurrent..., there does not seem to be a proper solution (or I am missing it) to tell LWJGL that the context owner thread has changed.
A call to GL.createCapabilities() seems not well named for that, and it also crashes with me when I make the context current in thread 'main' (which works) which happens-before switching to another spawned thread 'B' (which crashes). There, the call fails with a NPE in org.lwjgl.system.Checks.checkPointer(Checks.java:80).
So, am I probably missing a simple GLContext.makeCurrent() ?
The latest nightly build includes the following changes to structs:
- Methods in bindings that previously used ByteBuffers as pointers to struct values, now use the corresponding struct class. This improves usability and type-safety.
- Methods in bindings that previously used ByteBuffers as pointers to struct arrays, now use a typed subclass of the new org.lwjgl.system.StructBuffer class. For example glfwGetVideoModes now returns an instance of GLFWvidmode.Buffer, where Buffer extends StructBuffer. The StructBuffer API is similar to NIO buffers and PointerBuffer, with the difference that each element is a struct instead of a primitive value. This improves usability and type-safety.
- The struct classes themselves wrap a memory address directly, instead of wrapping a ByteBuffer instance. This reduces overhead. A struct class can still share memory with a ByteBuffer, but it is optional.
- The static API still exists, but now better matches the LWJGL bindings. There are methods that can be used to get/set struct values on a ByteBuffer and there are unsafe versions (with the 'n' prefix) that work with pointer addresses. This offers maximum flexibility, while keeping auto-complete lists clean.
- There are 3 ways to allocate a struct instance or buffer: via malloc, calloc or BufferUtils. You make the choice with static factory methods. If malloc/calloc are used, then you're responsible for freeing the corresponding allocation. This improves performance and allows allocation without zeroing out the struct.
- There is a clear() method that explicitly zeroes out a struct. This is often useful.
Note that a StructBuffer currently wraps a ByteBuffer, which has some overhead: a second instance required and some division operations in methods. This is an implementation detail and can change in the future.
Quote from: Kai on September 29, 2015, 19:02:00There was once a thread about refactoring the GLContext/GLCapabilities classes. Back then I was fine with things turned out to be, but now I somehow am uncomfortable with the way to make the GL context current in different threads.
Provided, there is a way to make the native GL context current in a calling thread, be it glfwMakeContextCurrent or SWT GLCanvas.setCurrent, or manually invoking wglMakeCurrent..., there does not seem to be a proper solution (or I am missing it) to tell LWJGL that the context owner thread has changed.
A call to GL.createCapabilities() seems not well named for that, and it also crashes with me when I make the context current in thread 'main' (which works) which happens-before switching to another spawned thread 'B' (which crashes). There, the call fails with a NPE in org.lwjgl.system.Checks.checkPointer(Checks.java:80).
So, am I probably missing a simple GLContext.makeCurrent() ?
To anyone else reading: this is exactly what I'm looking for in this topic. Would have been perfect with some example code that reproduces the crash(es).
To Kai: You're probably missing a call to GL.setCapabilities(GLCapabilities). But I would like to see some code to make sure. If this works for you, let me know if you have any suggestion on how to make this more discoverable. A method on GLCapabilities perhaps?
Hm, it might be that I am doing something totally wrong, but I just changed the "Multithreaded" (https://github.com/LWJGL/lwjgl3-demos/blob/master/src/org/lwjgl/demo/opengl/glfw/Multithreaded.java) demo in lwjgl3-demos repository to this:
In "main" thread right at the beginning in winProcLoop()
glfwMakeContextCurrent(window);
caps = GL.createCapabilities();
debugProc = GLUtil.setupDebugMessageCallback();
glClearColor(0.3f, 0.5f, 0.7f, 0.0f);
This above happens-before the things now in a spawned "render" thread, right at the beginning of renderLoop():
glfwMakeContextCurrent(window);
GL.setCapabilities(caps);
glClearColor(0.3f, 0.5f, 0.7f, 0.0f);
Now, with this there is no more NPE or any other exception, but the render is complete black, whereas before it was blueish with a white spinning quad.
EDIT:
I committed my changes with: https://github.com/LWJGL/lwjgl3-demos/commit/a7fc7eeeb510f0741b8f8bc422e492dbc0fff617 (https://github.com/LWJGL/lwjgl3-demos/commit/a7fc7eeeb510f0741b8f8bc422e492dbc0fff617)
Before the commit, everything worked. After the commit: black window with me.
I see now. Indeed the previous code was working fine and I'm now getting a black screen too. The problem is that you're making the context current in the secondary thread, while it is still current in the main thread. You can fix the current code by calling:
glfwMakeContextCurrent(NULL);
in the main thread, right before spawning the secondary thread.
edit: as a summary, to "transfer" a context to another thread:
- Make context current in thread A
- caps = GL.createCapabilities() in thread A
- Clear current context in thread A
- Make context current in thread B
- GL.setCapabilities(caps) in thread B
Note that calling GL.createCapabilities() again in thread B is legal, but setCapabilities() is more efficient (creating the GLCapabilities instance can take a few hundred milliseconds).
Ohh.. thanks! That's it. Did not know that one actually has to manually release the context from one thread before making it current in another one.
I think the solution with GL.createCapabilities(), storing it, and making it current in LWJGL's thread local with GL.setCapabilities() is the best thing I can think of right now, concerning clean/convenient API, so no request for change. :)
Nightly build 3.0.0b #40 includes two more changes:
- Structs that are used in an output-only/read-only manner now come without setter methods. This results in a cleaner API, a lot of methods that were never going to be used are now gone. Let me know if I missed a read-only struct or if I marked a read/write struct as read-only.
- The StructBuffer implementations now come with getters/setters that work on the struct value at the current buffer position. This means you can use struct buffers as flyweights. For example, the two loops below are equivalent:
// WARNING: OBSOLETE CODE
IntBuffer count = memAllocInt(1);
GLFWvidmode.Buffer modes = glfwGetVideoModes(glfwGetPrimaryMonitor(), count);
// element access
for ( int i = 0; i < count.get(0); i++ ) {
GLFWvidmode mode = modes.get(i);
System.out.println(mode.getWidth() + " x " + mode.getHeight());
}
// flyweight access
for ( int i = 0; i < count.get(0); i++ ) {
modes.position(i);
System.out.println(modes.getWidth() + " x " + modes.getHeight());
}
memFree(count);
edit: Build 3.0.0b #41 contains a fix that hides parameters such count in glfwGetVideoModes, so the above code becomes:
GLFWvidmode.Buffer modes = glfwGetVideoModes(glfwGetPrimaryMonitor());
for ( int i = 0; i < modes.capacity(); i++ ) {
GLFWvidmode mode = modes.get(i);
System.out.println(mode.getWidth() + " x " + mode.getHeight());
}
for ( int i = 0; i < modes.capacity(); i++ ) {
modes.position(i);
System.out.println(modes.getWidth() + " x " + modes.getHeight());
}
The GLFW structs somehow ended up with their original names all this time. Should I change them to the Java naming convention? That is:
GLFWimage -> GLFWImage
GLFWvidmode -> GLFWVidmode (or GLFWVidMode?)
GLFWgammaramp -> GLFWGammaramp (or GLFWGammaRamp?)
I suggest not to rename them, for the same reason we chose to not remove the suffixes of OpenGL functions.
Quote from: SHC on September 30, 2015, 13:40:33I suggest not to rename them, for the same reason we chose to not remove the suffixes of OpenGL functions.
The same could be said about callbacks though. Callbacks were renamed last December, to be consistent across APIs. Is anyone in favor of reverting their names to match the original C API? Examples:
GLFWErrorCallback -> GLFWerrorfun
GLFWWindowSizeCallback -> GLFWwindowsizefun
GLDebugMessageCallback -> DEBUGPROC
I would go with the java convention since it is easier to read in my opinion.
With todays IDE's a simple lowercase/uppercase mistake should take only the press of a button to fix.
I too think that going with the Java naming conventions is a little bit easier to read:
- GLFWGammaRamp compared to GLFWgammaramp
- GLFWVidMode compared to GLFWvidmode
And it would also allow for the "initials typing scheme" in Eclipse when searching types by name, such as GLFWGR or GLFWVM if that's worth anything for anyone. :)
Quote from: spasi on September 30, 2015, 14:10:09
Is anyone in favor of reverting their names to match the original C API? Examples:
GLFWErrorCallback -> GLFWerrorfun
GLFWWindowSizeCallback -> GLFWwindowsizefun
GLDebugMessageCallback -> DEBUGPROC
I'd stay with what we have now in this case too, the original C API versions are really ugly (at least from a Java programmer point of view), plus the consistency across different bindings is nice.
To stay consistent we should therefore also change the GLFW structs to the Java naming conventions.
Quote from: Kai on September 30, 2015, 14:40:35And it would also allow for the "initials typing scheme" in Eclipse when searching types by name, such as GLFWGR or GLFWVM if that's worth anything for anyone. :)
That's a very good point, IntelliJ has this too and I use it all the time.
+1 on Java convention here (also, hi :D)
I have but one additional request: Can we please make the auto-generated classes not final?
The reason is that one can then extend those classes and provide additional and/or cross-cutting concerns to certain methods of those classes by extending the classes and hiding specific methods.
And since the methods are all static anyway, making the class final will also not have any performance benefit (the methods are linked against statically at the callsite and everything is known about the method invocation at compiletime).
When extending a GLxx class and hiding some methods, the client/user can then just swap the static import from GLxx to `MyGLxx` and the Java compiler would use the "nearest" hidden method version and everything would still compile.
When the GLxx class is final, one can of course still build a MyGLxx but would then be forced to fully selectively import the corresponding methods and cannot use static import anymore, because of ambiguous methods when importing both GLxx and MyGLxx.
Nightly build 3.0.0b #42 is up. Changes:
- GLFW structs have been renamed.
- Struct and binding classes are now not final. Binding classes now include a default constructor, which enables "static extension" as per Kai's request, but it should never be called (throws immediately).
- Added bindings to xxhash (https://github.com/Cyan4973/xxHash).
Just a tiny nitpick, the 'Configuration' class name seems pretty long :), could something smaller like 'Config' work better?
Another possible simplification/idea, the Sys class is now a lot slimmer in LWJGL3 (compared to LWJGL2), rather than have users access another class for configurations, could the enum variables not just be inside Sys? e.g. Sys.LIBRARY_PATH.set("libs"); (http://sys.library_path.set("libs");)
Yes, it could be done. The reason I added a new class and put it in the system package is that I wasn't really sure of the design. I also consider most of the settings "advanced", they won't be of interest to most users. Finally, remember that I have reserved the system package for breaking changes, everything* in there is subject to change between releases.
* except MemoryUtil, which I consider extremely useful and will try to keep stable.
Only reason I mentioned the Sys class was it might be nice to have a single point of getting and setting all LWJGL library related information.
However good point about keeping the system package breakable, another variation could be:
import static org.lwjgl.system.Configuration.*;
...
Sys.set(LIBRARY_PATH, "libs");
and
String libPath = Sys.get(LIBRARY_PATH);
With Config/Configuration then just being a data storage class which could change when needed.
I would keep the Configuration class with a fixed API, and let it act as a facade for all the configuration aspects, where most of that functionality may or may not be implemented by the Sys class, which may of course change.
Sorry for the delayed response, had a very busy couple of days.
Quote from: kappa on October 12, 2015, 13:10:26import static org.lwjgl.system.Configuration.*;
...
Sys.set(LIBRARY_PATH, "libs");
and
String libPath = Sys.get(LIBRARY_PATH);
With Config/Configuration then just being a data storage class which could change when needed.
I see two separate issues with the current solution:
- Configuration is a big name to type and looks ugly I guess.
- The user needs to go into org.lwjgl.system to discover it.
The above fixes the first, but not the second. Also, in the presence of a Configuration static import, why not do:
import static org.lwjgl.system.Configuration.*;
...
LIBRARY_PATH.set("libs");
and
String libPath = LIBRARY_PATH.get();Why involve Sys?
Quote from: Kai on October 12, 2015, 13:17:52I would keep the Configuration class with a fixed API, and let it act as a facade for all the configuration aspects, where most of that functionality may or may not be implemented by the Sys class, which may of course change.
Could you give a code example please? I'm not sure I understand what you mean exactly.
Quote from: spasi on October 14, 2015, 07:52:10
Quote from: Kai on October 12, 2015, 13:17:52I would keep the Configuration class with a fixed API, and let it act as a facade for all the configuration aspects, where most of that functionality may or may not be implemented by the Sys class, which may of course change.
Could you give a code example please? I'm not sure I understand what you mean exactly.
Sure. What I meant was just that having a Configuration class (and I personally would call it Configuration instead of Config, I don't like abbreviations) with a stable API is a good thing, whereas the functionality provided by that Configuration class could as well be implemented by means of the Sys class, to avoid code duplication or something like that.
So we would have a small class Configuration, which could have various methods to configure things, but the actual doing of that configuration in the background inside LWJGL can be implemented by Sys.
By "facade" I just meant exactly this: The Configuration class can provide an interface whereas its functionality can be spread over many other internal classes (such as Sys) by means of delegation, but the user would not need to know.
This isn't really feedback on changes to be made but feedback on the changes being made.
I'm still on the previous stable build, I have read many threads about some of the core changes being made and classes that have been changed / moved / removed etc. Lots of good change, but tracking all the changes is difficult. What would be good, once the API freeze occurs, is to either have a summary of all the changes made or having the getting started page demonstrate everything (it might be large if it had everything so maybe there needs to be a simple and extended example)
Just an idea, I have lost track of all the changes so it would be good to see everything in one place explained.
Did you look at the demo package (https://github.com/LWJGL/lwjgl3/tree/master/modules/core/src/test/java/org/lwjgl/demo) in the LWJGL3 repository? Those cover every aspect of the API (GLFW, registering the GL context in LWJGL, registering callbacks for GLFW, OpenGL, OpenCL, OpenAL, ...).
I also try my very best to keep the lwjgl3-demos (https://github.com/LWJGL/lwjgl3-demos) repository up to date with the very latest changes in LWJGL3.
However, those latter demos only make more use of OpenGL functions. They do not make use of new aspects of LWJGL that aren't already shown in the demos package of LWJGL3.
I have seen them yes, I used them when I first migrated over to LWJGL3. It would be just useful to see a summary of all the changes listed out in one place.
Hi all,
It's been a bit over two weeks that I moved to a new home and I'm still on mobile internet, they haven't transferred my VDSL connection yet. Oh the joys of living in Greece...
Anyway, I used this time to work on some implementation details and settle some long-pending issues that were compromising the library's robustness. Especially worth mentioning is the fix for starting GLFW without -XstartOnFirstThread on OS X, we get an informative exception now. It was really unfortunate that most users' first experience with LWJGL was a nasty JVM crash.
On Configuration, I'm still not convinced it should be taken out of org.lwjgl.system. This is something we can do in a post 3.0 release, when the design and implementation details have settled.
On the other hand, I've been thinking about kappa's point that the Sys class doesn't do anything useful. I don't like its name, Sys, when there's also an org.lwjgl.system package. I also don't like LWJGLUtil as a name. This is what I think should be done:
- Combine the Sys and LWJGLUtil classes.
- The new class should have a new, nicer name. I haven't thought of anything, need suggestions please.
- Move some of LWJGLUtil's functionality that's only used internally (log, getClassTokens) into the org.lwjgl.system package somewhere.
What do you think?
Quote from: abcdef on October 15, 2015, 10:43:53It would be just useful to see a summary of all the changes listed out in one place.
Hey abcdef,
I'll try to post some details in a blog post when 3.0b is released. The site will also be updated with a page that lists all releases and associated release notes. The release notes will probably also be reflected on Github. I haven't bothered with anything like that so far because the library has been going through extensive changes.
Just one note about the Sys class. It mostly seems to be used for build-version querying.
I would move that functionality to a Configure/Configuration/Config class, in the style of GNU Autotools/Autoconf config.h also storing version and defines for detected features. That's the best I can come up with regarding lending a known concept/name from somewhere else people could be comfortable/familiar with.
A major difference is that Configuration contains mutable state. The user may set values there and LWJGL reads those values (usually only once at startup). The Sys and LWJGLUtil fields are all immutable (finalized at runtime, on class init).
Okay, I just had a more closer look at the LWJGLUtil and Sys classes what they actually do.
As I see it, there are the following aspects currently dealt with by those classes:
1. Obtaining the platform that LWJGL is running on (LWJGLUtil)
2. Loading shared libraries (LWJGLUtil)
3. Obtaining the architecture pointer size (Sys)
4. Querying the build version (Sys)
5. Querying the fields of some classes (LWJGLUtil) - it is used in some *_ERROR_TOKENS fields, but I am not sure why and what for
So, if I leave the last point of the above iteration behind, I would do it this way:
Implement the aspects 1., 2. and 3. in a class called "Runtime". This is mainly because in Java the class dealing with shared library loading is also called Runtime and it somewhat suits aspects 1. and 3. as well, I think.
Implement aspect 4. maybe also in "Runtime", or add a small class called "Version" directly under org.lwjgl.
The problem with calling it Runtime is that this would clash with java.lang.Runtime. Maybe something like LWJGLRuntime?
*side question*
Is there a reason why the LWJGLUtil shared library loader tries to individually extract the right library files from a directory with all the platform jars in? I'd have thought it would be much easier to have sub directories per platform / architecture and to just reference everything in that directory. LWJGL2 had a loader which did this (the directory structure was in a JAR).
LWJGL 3 originally had a platform/arch directory structure for natives, but was changed to a monolithic directory due to user feedback. The SharedLibraryLoader just mirrors that decision I guess. Note that it was contributed by Mario Zechner, I only cleaned up the code. Also keep in mind that it works with getResourceAsStream, it doesn't know anything about the library JARs. You could have a JAR per platform or a fat JAR with everything in it and it will still work, as long as the classpath is set up correctly.
Does the current implementation cause any issues for you?
Quote from: Kai on November 05, 2015, 10:38:02
Implement the aspects 1., 2. and 3. in a class called "Runtime". This is mainly because in Java the class dealing with shared library loading is also called Runtime and it somewhat suits aspects 1. and 3. as well, I think.
Implement aspect 4. maybe also in "Runtime", or add a small class called "Version" directly under org.lwjgl.
Quote from: FortressBuilder on November 05, 2015, 17:35:29The problem with calling it Runtime is that this would clash with java.lang.Runtime. Maybe something like LWJGLRuntime?
Runtime sounds good to me. LWJGLRuntime feels verbose ("lwjgl" is already part of the package name). Any other ideas?
"Environment" comes to my mind when thinking about aspects 1. and 3.
However, I think "Environment" does not fit aspect 2. as good as "Runtime."
EDIT: Or better (I think) would be "Platform."
adding the LWJGL* prefix to any class makes it feel verbose :)
FortressBuilder does raise a good point about the Runtime being confusing with java.lang.Runtime as that is a core java vm class.
Other names options could include "Library", "Lib", "LibLWJGL", "LibUtil", "Version" (as mentioned above) or "LRuntime".
Quote from: spasi on November 06, 2015, 10:54:11
LWJGL 3 originally had a platform/arch directory structure for natives, but was changed to a monolithic directory due to user feedback. The SharedLibraryLoader just mirrors that decision I guess. Note that it was contributed by Mario Zechner, I only cleaned up the code. Also keep in mind that it works with getResourceAsStream, it doesn't know anything about the library JARs. You could have a JAR per platform or a fat JAR with everything in it and it will still work, as long as the classpath is set up correctly.
Does the current implementation cause any issues for you?
none :) I already had some code to do this and I package the natives up with a script. It was more from a curiosity perspective as I personally feels its not very extensible.
Did a few experiments on this and in the end decided that most of the functionality in Sys and LWJGLUtil is not really interesting to most LWJGL users. For example, loading shared libraries (the LWJGL JNI code + 3rd party libraries) should work transparently most of the time and only advanced users will need to take control of things. This is exactly what the system package is useful for.
So, in order to avoid making an unfortunate decision now that will be very hard to change later, I moved everything except the version stuff in org.lwjgl.system. Sys and LWJGLUtil are gone and the base package has only 3 classes now: BufferUtils, PointerBuffer and the new Version class. The library loading functionality has been moved to org.lwjgl.system.Library, the platform stuff in org.lwjgl.system.Platform and the rest went into APIUtil, Checks and MemoryUtil (depending on where they fit best). The SharedLibraryLoader was also moved in system and the DynamicLinkLibrary interface renamed to SharedLibrary.
The latest nightly build (3.0.0b #57) includes the above changes, as well as some general cleanup in preparation for the beta release. Tomorrow I'll upgrade the LibOVR bindings to 0.8 and if there are no other suggestions/objections in this thread, I'll release the 3.0.0 beta.
very good changes, really does make the api much nicer and cleaner.
I'm afraid I had to do another breaking change. I was still not at all satisfied with the struct API, it was too heavy and too ugly. The latest nightly build (3.0.0b #60) includes the following changes:
1) Struct member accessors now match the native member names. The get/set prefixes are gone and there are no lower-to-upper case modifications like before. For example:
GLFWImage img = GLFWImage.malloc();
// before
img.setWidth(128);
int w = img.getWidth();
// after
img.width(128);
int w = img.width();
2) All getter implementations have been changed to prefer returning a view of the underlying struct data, instead of a copy of the struct data. This affects getters that return buffers and nested structs.
3) Direct accessors to members of nested structs have been removed. They were making the parent struct classes very heavy. This change will result in more garbage in cold code, but escape analysis should take care of unnecessary instances in hot methods.
4) The static accessors were simplified considerably, there are fewer overloads now. The static API is not as rich as before, but the StructBuffer classes should cover most use-cases.
5) Javadoc is now generated for all accessors. Also, the native struct definition is generated in the class javadoc, along with documentation per struct member.
As always, any user not satisfied with 2, 3 or 4 may use the unsafe tools in MemoryUtil to implement exactly the functionality they need.
I think I'm done for real this time. Let me know what you think and I'll release 3.0.0b if there are no objections.