Help. Problem moving from lwjgl 2 to 3 for my eclipse plugin

[obi]

Hi!

I have been using headless lwjgl 2 in an eclipse plugin and it have worked flawlessly for several years. When I modify and save shader files in my java projects the plugin automatically compile and mark error lines in the shader files. This is a fast way to detect simple typos and error without having to run any programs.

Was going to implements some new features and decided to change the plugin to lwjgl 3 since I use the new lwjgl for everything else. But I get some weird problems with crashing jvm's and buffer overflows. The only thing I have changed is the initialization and how to make a context current.

I this the right way to port the LWJGL 2 to 3 for same functionality?

First this is how I initialize and make the opengl context current in the eclipse builder thread:
LWJGL 2:

private static final Pbuffer buff;
    
    static
    {
        try
        {
            final PixelFormat format = new PixelFormat();
            buff = new Pbuffer(1, 1, format, null);
        }
        catch( Throwable e )
        {
            ObiGLBuilderPlugin.logError( "Failed to initialize OpenGL context", e );
            throw new RuntimeException( e );
        }
    }
    
    public static void bind() 
        throws CoreException
    {
        try
        {
            buff.makeCurrent();
        }
        catch( LWJGLException e )
        {
            throw new CoreException( new Status( IStatus.ERROR, ObiGLBuilder.BUILDER_ID, "Failed to make OpenGL context current", e ) );
        }
    }
    
    public static void release() 
        throws CoreException
    {
        try
        {
            buff.releaseContext();
        }
        catch( LWJGLException e )
        {
            throw new CoreException( new Status( IStatus.ERROR, ObiGLBuilder.BUILDER_ID, "Failed to release OpenGL context", e ) );
        }
    }

LWJGL 3:

private static long           offscreen_context;
    private static GLCapabilities caps;

    static
    {
        try
        {
            GLFW.glfwInit();
            GLFW.glfwWindowHint( GLFW.GLFW_VISIBLE, GLFW.GLFW_FALSE );
            offscreen_context = GLFW.glfwCreateWindow( 16, 16, "headless", MemoryUtil.NULL, MemoryUtil.NULL );

            GLFW.glfwMakeContextCurrent( offscreen_context );
            caps = GL.createCapabilities();
        }
        catch( final Throwable e )
        {
            Activator.logError( "Failed to initialize OpenGL context", e );
            throw new RuntimeException( e );
        }
    }

    public static void bind()
        throws CoreException
    {
        GLFW.glfwMakeContextCurrent( offscreen_context );
        GL.setCapabilities( caps );
    }

    public static void release()
        throws CoreException
    {
        GLFW.glfwMakeContextCurrent( MemoryUtil.NULL );
    }

The error I keep getting right now is that when my plugin compile shaders, the following command sometimes return seemingly random values for length.
final int length = GL20.glGetShaderi( id, GL20.GL_INFO_LOG_LENGTH );

Anyone running lwjgl 3 headless and/or in eclipse plugins?

Is my take on context handling with LWJGL 3 correct?

[spasi]

The error I keep getting right now is that when my plugin compile shaders, the following command sometimes return seemingly random values for length.
final int length = GL20.glGetShaderi( id, GL20.GL_INFO_LOG_LENGTH );

This can happen if your code is generating OpenGL errors. Make sure to check for errors or create a debug context and register an error callback with GLUtil.setupDebugMessageCallback().

[obi]

Everytime the eclipse builder calls my plugin I call my bind() method and when I'm done release();

This can work for a while then just stop working. Then I have to restart eclipse.

If i change my bind() to

GLFW.glfwMakeContextCurrent( offscreen_context );
GL.setCapabilities( caps );
GLUtil.checkGLError();

When it stops working the checkGLError() returns

org.lwjgl.opengl.OpenGLException: Operation illegal in current state [0x502]
   at org.lwjgl.opengl.GLUtil.checkGLError(GLUtil.java:35)
   at obigl.eplug.builder.shader.java.OpenGLContext.bind(OpenGLContext.java:108)

so is my way of switching contexts bad?

[quew8]

Creating/switching contexts shouldn't cause an OpenGL error, the illegal operation must be being caused in the code where you actually compile the shaders. Find out which function is causing the error.

[obi]

Yeah. But the weird thing is that I have used this code for years under lwjgl2. Only thing I have changed is the posted code above creating the context and using and releasing it.
I have added GLUtil.checkGLError(); after every GL call in the plugin.

I get no errors from my plugin until after I call MakeContextCurrent and set the capabilities. Edit: when it stops working calling MakeContextCurrent with the windowPtr will not make any context current.

If I have a hidden(headless) GLFW window that is idle and never made current for a period of time can the context be deleted by itself? All I have is a static long that is the windowPtr.
Its just seems so random when the context stops working.

[Cornix]

1) Could it be that your code is called from a different thread? Try to print the current thread when your MakeContextCurrent method is called.

2) Could it be that your OpenGL classes have been garbage collected?

[obi]

That seems to be it. Eclipse have multiple worker threads building projects. So my shader compiler can be called by different threads. But why did this work flawlessly with lwjgl 2?

I guess my plugin will have to run its own thread and just have eclipse threads add operations to an event queue.

[Kai]

You can perfectly share the same OpenGL context among different threads, given that only one thread at a time calls GL methods using that context.
When a thread calls into your plugin then do a GLFW.glfwMakeContextCurrent(offscreen_context) and when the thread exits your plugin you MUST call GLFW.glfwMakeContextCurrent(NULL), because otherwise the OpenGL context will stay current in the last thread for which GLFW.glfwMakeContextCurrent was called with a non-null argument.
OpenGL does not know that when you call GLFW.glfwMakeContextCurrent in thread B that it has to detach that OpenGL context from thread A.

[Cornix]

That seems to be it. Eclipse have multiple worker threads building projects. So my shader compiler can be called by different threads. But why did this work flawlessly with lwjgl 2?

Just because it never broke down before doesnt mean it worked flawlessly. It could have had the same problem but by extreme coincidence it never occured. Or perhaps something was changed with the new version of eclipse. We certainly cant give you an answer without seeing all of your code.

[obi]

I made the hidden window visible and it randomly timeouts/vanishes. I can't find any obvious reason, garbage collection?

I'm not versed in how GLFW and LWJGL 3 works behind the scene. When the window vanishes the window pointer get invalidated, using it in calls can crash the JVM.
Is there a way to just get a context without using GLFW like in lwjgl 2, would be perfect for headless opengl.

Will have to revert back to lwjgl 2 for now.

[spasi]

I made the hidden window visible and it randomly timeouts/vanishes. I can't find any obvious reason, garbage collection?

The Java GC does not interact with GLFW in any way. Only callbacks need special treatment, you must have strong references to callback instances while they are active.

I'm not versed in how GLFW and LWJGL 3 works behind the scene. When the window vanishes the window pointer get invalidated, using it in calls can crash the JVM.

A few things you can try:

- Create a debug context (see glfwWindowHints) and use GLUtil.setupDebugMessageCallback() to create an OpenGL debug message callback. This is the best way to get immediate feedback on OpenGL errors, without any other changes to your code.
- Register a GLFW error callback with glfwSetErrorCallback. GLFW performs a lot of checks internally and you may get some info this way.
- Enable LWJGL debug mode with -Dorg.lwjgl.util.Debug=true or Configuration.DEBUG.set(true).

It's very unlikely that you won't get something useful from one of the above. If not, try to simplify your code and post a minimum program that reproduces the problem. There must be a bug in your code, GLFW windows do not disappear without a good reason.

Also, on what OS are you running this?

[obi]

Have been pulling my hair some more over this.

Firstly I just couldn't understand why I didn't get any messages. Had to read some code and figure out that you *must* call Configuration.setDebugStreamConsumer before GLFW.glfwSetErrorCallback

Because APIUtil.DEBUG_STREAM is initialized when calling glfwSetErrorCallback and the DebugStreamConsumer must have been created and set before that.

But still The only output I get is:
[LWJGL] Version: 3.0.0 SNAPSHOT
[LWJGL]     OS: Windows 7 v6.1
[LWJGL]    JRE: 1.8.0_77 amd64
[LWJGL]    JVM: Java HotSpot(TM) 64-Bit Server VM v25.77-b03 by Oracle Corporation
[LWJGL] Loaded library from java.library.path: lwjgl
[LWJGL] MemoryUtil accessor: MemoryAccessorUnsafe
[LWJGL] Loaded native library: jemalloc.dll
[LWJGL] MemoryUtil allocator: DebugAllocator
[LWJGL] Loaded native library: glfw.dll
[LWJGL] Loaded native library: C:\Windows\system32\opengl32.dll
[LWJGL] Failed to locate address for GL function glNamedBufferPageCommitmentARB
[LWJGL] Failed to locate address for GL function glVertexWeighthNV
[LWJGL] Failed to locate address for GL function glVertexWeighthvNV
[LWJGL] Failed to locate address for GL function wglCopyImageSubDataNV
[LWJGL] [GL] WGL_NV_copy_image was reported as available but an entry point is missing.
[LWJGL] [GL] Using OpenGL 4.3 for error logging.

This is my debug Initialization:

            final DebugStreamConsumer consumer = new Configuration.DebugStreamConsumer()
            {                
                @Override
                public void accept( final String msg )
                {
                    Activator.out.println( msg );
                }
            };                
            Configuration.DEBUG.set( true );
            Configuration.DEBUG_MEMORY_ALLOCATOR.set( true );
            Configuration.setDebugStreamConsumer( consumer );

            GLFW.glfwSetErrorCallback( ecb = new GLFWErrorCallback() 
            {
                private final GLFWErrorCallback delegate = GLFWErrorCallback.createPrint(System.err);

                @Override
                public void invoke(final int error, final long description) 
                {
                    Activator.out.println( delegate.getDescription( description ) );
                    delegate.invoke(error, description);
                }
            });
                        
            if (GLFW.glfwInit() != GL11.GL_TRUE)
                throw new IllegalStateException("Unable to initialize GLFW");
            GLFW.glfwWindowHint( GLFW.GLFW_OPENGL_DEBUG_CONTEXT, GLFW.GLFW_TRUE );
            GLFW.glfwWindowHint( GLFW.GLFW_VISIBLE, GLFW.GLFW_FALSE );
            offscreen_context = GLFW.glfwCreateWindow( 16, 16, "headless", MemoryUtil.NULL, MemoryUtil.NULL );

            GLFW.glfwMakeContextCurrent( offscreen_context );
            caps = GL.createCapabilities( true );
            
            debugProc = GLUtil.setupDebugMessageCallback();

I'll try to make as minimalistic eclipse plugin as possible and see if I get the same issue.

[obi]

This is the simplest eclipse plugin I could do. eplug.lwjgldebug.zip

1. Just import it as a plugin project.
2. Launch it as an Eclipse application.
3. Create a test project
4. Create a file called test.txt

Everytime this file is changed the plugin will make the context current, write a text line to console then release the context.
The glfw window will be visible. Just wait for it to vanish which should happen in a few minutes. Modify the test.txt file to confirm that the context can no longer be made current.

[Cornix]

A few minutes? Might it be that eclipse is killing the process / thread? I am not too familiar with the way eclipse handles plugins but it really sounds more like some shady business going on behind your back which you are not aware of.

[obi]

Yeah. I got no clue about this one.. usually can figure out how to solve most problems without resorting to the forum

Since I just need a headless opengl context. I never set GLFW callbacks, swap any buffers or poll any events. Maybe GLFW don't like that?