Free LWJGL/OpenGL resources

[peterdr]

What's the best way to free all resources linked to an OpenGL context?

I have an application that uses LWJGL3 to do offscreen rendering. The result is copied to a JavaFX Canvas.
When using the application, rendering needs to be done for multiple Canvas-es that can be dynamically created. Whenever a Canvas is shown, a separate OpenGL context is created. Whenever the canvas is hidden, I would like to free all associated resources.

I use the following OpenGL calls to delete shaders, buffers, textures, etc.:

• glDeleteShader
• glDeleteProgram
• glDeleteRenderbuffers
• glDeleteFramebuffers
• glDeleteTextures
• glDeleteBuffers
• glDeleteVertexArrays

Afterwards i call glfwDestroyWindow to destroy the window.

The system memory used by the Java application increases whenever a new Canvas is shown (a new context is set-up). It does however not reduce when it is hidden (freed). I monitored the heap size used by the application and there are no memory leaks (the used heap size does not increase). If I disable the LWJGL rendering code, the system memory usage is stable. So I assume I am not freeing the used resources correctly?

[Kai]

You don't need to free the resources manually. Destroying the OpenGL context, which glfwDestroyWindow() does internally, will dispose of any resources allocated in the context of that OpenGL context, unless they are shared with another context.
I have a very very complex OpenGL application, with multiple FBOs, VBOs, VAOs and shaders, whose initialization I just put in an endless loop and destroying the GLFW window at the end of each iteration, and I see no memory leak.

Do you use explicit System.gc() ? And over how long of a period of time (or invocations) did you measure the increase in memory? The JVM does not immediately return allocated memory pages to the OS, but keeps it around in a buffer for further object allocations. It could very well be that LWJGL's capabilities object, which is the only one you cannot explicitly free (unless unset the ThreadLocal) and which is created for each OpenGL context, will not immediately be reclaimed by the GC.

Also what exactly do you mean with "disabling the LWJGL rendering code?" What does that code incorporate exactly? The creating of the GLFW window and implicitly the creating of the context, but just not creating your OpenGL resources?

[Cornix]

I use the following OpenGL calls to delete shaders, buffers, textures, etc.:

• glDeleteShader
• glDeleteProgram
• glDeleteRenderbuffers
• glDeleteFramebuffers
• glDeleteTextures
• glDeleteBuffers
• glDeleteVertexArrays

Afterwards i call glfwDestroyWindow to destroy the window.

If you destroy the context anyways you dont need to delete your shaders and buffers manually. Destroying an OpenGL context always frees up all resources.

The system memory used by the Java application increases whenever a new Canvas is shown (a new context is set-up). It does however not reduce when it is hidden (freed). I monitored the heap size used by the application and there are no memory leaks (the used heap size does not increase). If I disable the LWJGL rendering code, the system memory usage is stable. So I assume I am not freeing the used resources correctly?

How do you observe the memory usage? Because it could be that you look at the memory reserved by the JVM instead of the memory that is actually needed. The JVM will often reserve more memory then it actually uses, just in case, to improve performance. It will do so as long as memory is available and the reserved memory is below a certain threshold.

[peterdr]

Thanks for the replies.

In a timespan of 15 minutes the memory usage has not gone down. I observe the memory usage by looking at the "Activity Monitor" of the operating system. Each time a new Canvas is created I can see a significant increase in the used memory (+25MB). At the same time, the used and allocated heap sizes (inspected using VisualVM) do not really increase.
By "disabling the LWJGL rendering code" I mean disabling all LWJGL-related code.

The application is built as follows:
- I created an "GLFWWindowManager" class for performing actions meant for the "main thread". The instance of this class (singleton) has an ExecutorService (single thread executor) to make sure all actions are performed on the same thread.
- When initalized, the GLFWWindowManager instance calls glfwSetErrorCallback and glfwInit and sets the necessary window hints.
- I created an "OpenGLExecutor" class for performing actions associated to a certain OpenGL context. The instances of this class also each have an ExecutorService (single thread executor) to make sure all actions related to an OpenGLContext are performed on the same thread.
- For each Canvas that is needed, an OpenGLExecutor object is created. The OpenGLExecutor object uses the GLFWWindowManager to create a window (glfwCreateWindow is called on the main thread, the resulting window handle is returned). Subsequently the OpenGLExecutor object calls glfwMakeContextCurrent (using the window handler) and GL.createCapabilities on its own thread.
- OpenGLExecutor performs OpenGL tasks on its own Thread
- If the Canvas is no longer needed, the OpenGLExecutor calls glfwMakeContextCurrent(0) to make sure its window handler is no longer used. Subsequently, it uses the GLFWWindowManager to call glfwDestroyWindow(window) (with its window handle) on the main thread.

[peterdr]

I created a small test-application to demonstrate the problem. You can find it here: https://github.com/pderoovere/lwjgl-offscreen-test

If you run the application you can see the used heap size. You have a button to add an "executor". Using your operating system's activity monitor you can see the used memory going up. If you add a couple of "executors", the memory increases significantly. The second button clears the context and removes all "executors". The heap size goes down. The used system memory goes down, but not enough.

Am I missing something here?

[UPDATE] I also added the option to destroy GLFW (after clearing all executors). This seems to clear a large portion of the used memory, but that should not be necessary I assume?

[spasi]

Do you see any difference if you run the test with "-Dorg.lwjgl.system.allocator=system"?

[peterdr]

The results are (approximately) the same and in this order of magnitude:

• Application start: 95MB
• Adding 10 executors: 950MB
• Clearing all executors: 570MB
• Again adding 10 executors: 1300MB
• Clearing all executors: 950MB

[spasi]

Also try "-Dorg.lwjgl.util.DebugAllocator=true". Do you see any output on exit?

[peterdr]

I get the following after clearing the contexts:

Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const: conn 0x2fd8b token 0xbffffffffff71e
Feb 26 16:01:21  java[14662] <Warning>: Backtrace (at 24328,4):
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  0   CoreGraphics                        0x00007fff8dd37ecd CGSDisableUpdateToken + 210
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  1   AppKit                              0x00007fff9002ac20 ___disable_updates_sync_block_invoke_2 + 17
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  2   libdispatch.dylib                   0x00007fff8bce633f _dispatch_client_callout + 8
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  3   libdispatch.dylib                   0x00007fff8bce7926 _dispatch_barrier_sync_f_invoke + 74
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  4   AppKit                              0x00007fff90029de9 NSCGSDisableUpdates + 213
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  5   AppKit                              0x00007fff9002afff NSCGSTransactionRunPreCommitActionsForOrder_ + 156
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  6   AppKit                              0x00007fff9002a014 NSCGSTransactionRunPreCommitActions_ + 21
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  7   AppKit                              0x00007fff9002a4ea -[_NSCGSTransaction synchronize] + 33
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  8   AppKit                              0x00007fff9002ab38 NSCGSTransactionSynchronize + 76
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  9   AppKit                              0x00007fff8fdfee09 -[NSSurface syncSurfaceWantsExtendedDynamicRange] + 150
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  10  AppKit                              0x00007fff8f7373d9 -[NSSurface _createSurface] + 677
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  11  AppKit                              0x00007fff8f736d34 -[NSSurface setFrame:] + 785
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  12  AppKit                              0x00007fff8fdfef0f __38-[NSSurface syncToViewUnconditionally]_block_invoke + 154
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  13  AppKit                              0x00007fff8ff2b659 NSPerformVisuallyAtomicChange + 147
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  14  AppKit                              0x00007fff8f736658 -[NSSurface syncToViewUnconditionally] + 100
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  15  AppKit                              0x00007fff8fdfefa8 __37-[NSSurface orderSurface:relativeTo:]_block_invoke + 44
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  16  AppKit                              0x00007fff8ff2b659 NSPerformVisuallyAtomicChange + 147
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  17  AppKit                              0x00007fff8f73652d -[NSSurface orderSurface:relativeTo:] + 160
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  18  AppKit                              0x00007fff8fa03ece NSOpenGLContextAttachOnScreenViewSurface + 183
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  19  AppKit                              0x00007fff8fd479ea __27-[NSOpenGLContext setView:]_block_invoke + 192
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  20  AppKit                              0x00007fff8ff2b659 NSPerformVisuallyAtomicChange + 147
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  21  AppKit                              0x00007fff8f9ef204 -[NSOpenGLContext setView:] + 119
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  22  libglfw.dylib                       0x00000001325dba0a _glfwCreateContext + 794
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  23  libglfw.dylib                       0x00000001325d8fc6 _glfwPlatformCreateWindow + 726
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  24  libglfw.dylib                       0x00000001325d5591 glfwCreateWindow + 513
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  25                                   0x000000010dd9e954 0x0 + 4527352148
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  26                                   0x000000010dd90760 0x0 + 4527294304
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  27                                   0x000000010dd90760 0x0 + 4527294304
Feb 26 16:01:21  java[14662] <Warning>: void CGSUpdateManager::log() const:  28                                   0x000000010dd90760 0x0 + 4527294304

And the following after glfwTerminate:

CoreAnimation: warning, deleted thread with uncommitted CATransaction; set CA_DEBUG_TRANSACTIONS=1 in environment to log backtraces.

[spasi]

Tried your code and there are a few issues:

1) You are not shutting down the ExecutorService in SingleThreadExecutor explicitly and rely on GC. The System.gc() call you do is done when the ExecutorService is not yet eligible for GC.

2) Many GLFW functions require that they run on the main thread only. Also, on OS X the main thread must also be the first thread in the process (see -XstartOnFirstThread). The singleThreadExecutor in GLFWWindowManager is neither the main, nor the first thread.

3) You're using JavaFX, which doesn't play well with GLFW because of 2). This causes the errors in your post above.

[peterdr]

1) You're absolutely right! I changed it. But it did not really affect the memory issue.

2+3) I added the possibility of running the Application without JavaFX. Running Application2 starts the application without UI. It is possible to add/clear executors by typing add or clear using the console/cmd, or to destory GLFW by typing destroy. The calls from GLFWWindowManager now occur on the main thread.

But the memory issue seems to be the same.

[spasi]

Windows 10, Java 8, Nvidia GTX 970:

Application start: 13MB
Adding 10 executors: 612MB
Clearing all executors: 123MB
Again adding 10 executors: 632MB
Clearing all executors: 137MB
Destroy: no difference

OS X, Java 8, Intel Iris 6100:

Application start: 15MB
Adding 10 executors: 806MB
Clearing all executors: 430MB
Again adding 10 executors: 1170MB
Clearing all executors: 815MB
Destroy: 51MB

Looks like it's an issue on OS X only.

[peterdr]

Will try to test on more devices the next couple of days. But on VM:

Windows 8 (Parallels, so OpenGL 2.1), Java 8

Application start: 16MB
Adding 10 executors: 833MB
Clearing all executors: 469MB
Adding 10 executors: 1233MB
Clearing all executors: 851MB
Destroy: no difference

[spasi]

Did some more testing, the above behavior goes away if:

a) I do not call glBufferData at all.
b) I call glBufferData with ARRAY_BUFFER_LENGTH * 4, i.e. initialize the buffer store but never upload any data.
c) b) + glMapBuffer + glUnmapBuffer.

If I do c) and actually use the mapped memory to write the FloatBuffer data, the behavior returns. I also tried sharing the resources of all contexts, no difference.

It doesn't feel like a bug to me (certainly not LWJGL's), but just the way the OpenGL driver manages memory on OS X. There might be a heuristic that says "if you needed that much memory once, you probably going to need it again". Have you tried putting memory pressure on the system by launching other processes? What happens when memory is close to exhausted?

[peterdr]

I did some testing on Windows and as you said, the behavior on Windows seems to be normal. On OSX, I tried to put pressure on the memory, but it did not affect the numbers. It is however possible to far exceed the available memory. Also, I tried the following:
- Adding 100 Executors - resulted in java.lang.OutOfMemoryError: Direct buffer memory
- Starting two application, adding 50 Executors in each - No errors
- Looping the following a 100 times in 1 application: Adding 50 Executors, followed by clearing all Executors - this did not result in any errors
When the memory increases, a large part of the memory is however indicated as compressed.

I also had a look at the the active contexts using the OpenGL Profiler on OSX. A strange thing I noticed is in the Context Id comboboxes, the destroyed contexts stay green until glfwTerminate is called. Also, when attaching the profiler after contexts were created and destroyed, they are still shown. It is difficult to find information about the way this list is created, or the meaning of the colors, so it is hard to say if this really means something...

Anyway, thank you for the feedback so far!