Buffer is not freed - OutOfMemory error

[fireinwinterfell]

Hi,
I am sorry if question is too naive, I am learning Opengl and lwjgl as well. When I try to load data using Buffer, the memory consumption is crazy too much it usually hits error "OutOfMemory". What is wrong with my code here:

private  void storeDataInAttributeList(int attributeNum, int size, float[] vertices) {
        int vboId = glGenBuffers();
        vbos.add(vboId);
        glBindBuffer(GL_ARRAY_BUFFER, vboId);

        FloatBuffer vertexBuffer = storeDataInFloatBuffer(vertices);
        glBufferData(GL_ARRAY_BUFFER, vertexBuffer, GL_STATIC_DRAW);
        MemoryUtil.memFree(vertexBuffer);

        GL30.glVertexAttribPointer(attributeNum, size, GL_FLOAT, false, 0, 0);
        glBindBuffer(GL_ARRAY_BUFFER, 0);


    }

    private FloatBuffer storeDataInFloatBuffer(float[] vertices) {
        FloatBuffer vertexBuffer =  MemoryUtil.memAllocFloat(vertices.length);
        vertexBuffer.put(vertices).flip();
        return vertexBuffer;
    }

[fireinwinterfell]

Someone please help!
The heap showing in Visualvm is just fine but the Windows TaskManager shows that it consumes a lot.
If I use glVertex2f() to set vertex by vertex at rendering time it is just fine (200mb for my program) but when using buffers, it consumes about more than 2GB.

I knew that data goes off-heap but could you give me some hints to explicitly, effectively free it once it has been loaded GPU? I seem to be failed with MemoryUtil.memfree() and MemoryStack, too. Is it possibly a problem with machine with an integrated intel GPU?

[spasi]

Hey fireinwinterfell,

The code above looks fine. The memory allocated in storeDataInFloatBuffer will always be freed by memFree. You could to isolate what exactly causes the apparent increase in memory usage, two ideas:

- Run the code as is, but do not call glBufferData.
- Replace the temporary FloatBuffer with a single, fixed buffer that is being reused between calls.

How many times do you call storeDataInAttributeList and with how much data?

[fireinwinterfell]

Thank God, you are in !!
I will try everyone of your ideas.

How many times do you call storeDataInAttributeList and with how much data?

thousands -> a few hundred thousand, a few 2d vertices (lines) to  about a few hundred (polylines).

[fireinwinterfell]

Hi spasi
Thanks for your ideas.

- Run the code as is, but do not call glBufferData.

->
Without glBufferData, memory consumption showing in Windows TaskManager is about X2 instead of X5-X10.I left indices, texture as is (still call glBufferData).

- Replace the temporary FloatBuffer with a single, fixed buffer that is being reused between calls.

->Memory consumption is a about 20 % lower (fixed FloatBuffer is 1024).

[spasi]

Sounds like there's no problem with the buffer alloc/free then, the memory is consumed elsewhere. Probably in the OpenGL driver, which copies (and holds on to) the data passed to glBufferData.

usually hits error "OutOfMemory"

Btw, do you mean a java.lang.OutOfMemoryError or a process-OOM error followed by a JVM crash?

[fireinwinterfell]

Btw, do you mean a java.lang.OutOfMemoryError or a process-OOM error followed by a JVM crash?

Now I got the log (first chunk):

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1048576 bytes. Error detail: AllocateHeap
# Possible reasons:
#   The system is out of physical RAM or swap space
#   This process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Unscaled Compressed Oops mode in which the Java heap is
#     placed in the first 4GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (allocation.cpp:44), pid=19920, tid=10312
#
# JRE version: Java(TM) SE Runtime Environment Oracle GraalVM 22+36.1 (22.0+36) (build 22+36-jvmci-b02)
# Java VM: Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 22+36.1 (22+36-jvmci-b02, mixed mode, sharing, tiered, jvmci, jvmci compiler, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#

---------------  S U M M A R Y ------------

Command Line: -XX:ThreadPriorityPolicy=1 -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCIProduct -XX:-UnlockExperimentalVMOptions -Xmx128m -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2023.1.5\lib\idea_rt.jar=65095:C:\Program Files\JetBrains\IntelliJ IDEA 2023.1.5\bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.stderr.encoding=UTF-8 gridx.lightning.RenderPDFApp -Xmx128m

Host: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 8 cores, 7G,  Windows 11 , 64 bit Build 22621 (10.0.22621.3374)
Time: Fri May 10 17:46:57 2024 SE Asia Standard Time elapsed time: 136.733497 seconds (0d 0h 2m 16s)

---------------  T H R E A D  ---------------

Current thread (0x000002c9d6f237f0):  VMThread "VM Thread"          [id=10312, stack(0x000000dd85300000,0x000000dd85400000) (1024K)]

Stack: [0x000000dd85300000,0x000000dd85400000]
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x6e2349]  (no source info available)
V  [jvm.dll+0x874ed3]  (no source info available)
V  [jvm.dll+0x8772fe]  (no source info available)
V  [jvm.dll+0x8779d3]  (no source info available)
V  [jvm.dll+0x27fcb6]  (no source info available)
V  [jvm.dll+0xbe877]  (no source info available)
V  [jvm.dll+0x348d4f]  (no source info available)
V  [jvm.dll+0x3436fe]  (no source info available)
V  [jvm.dll+0x32bf1c]  (no source info available)
V  [jvm.dll+0x370134]  (no source info available)
V  [jvm.dll+0x87d870]  (no source info available)
V  [jvm.dll+0x87ebf7]  (no source info available)
V  [jvm.dll+0x87f170]  (no source info available)
V  [jvm.dll+0x87f413]  (no source info available)
V  [jvm.dll+0x81d50b]  (no source info available)
V  [jvm.dll+0x6e09e5]  (no source info available)
C  [ucrtbase.dll+0x29333]  (no source info available)
C  [KERNEL32.DLL+0x1257d]  (no source info available)
C  [ntdll.dll+0x5aa48]  (no source info available)

VM_Operation (0x000000dd84cff370): G1CollectFull, mode: safepoint, requested by thread 0x000002c9bb39fb90


---------------  P R O C E S S  ---------------

Threads class SMR info:
_java_thread_list=0x000002c986feabb0, length=15, elements={
0x000002c9bb39fb90, 0x000002c9d6f3e250, 0x000002c9d6f3ed80, 0x000002c9d6f48c20,
0x000002c9d6f4b700, 0x000002c9d6f52930, 0x000002c9d6f53740, 0x000002c9d6f575e0,
0x000002c9d6f59bf0, 0x000002ca1c12acd0, 0x000002ca1c4377b0, 0x000002ca1c43a1a0,
0x000002c986913010, 0x000002c986f22900, 0x000002c986f1c760
}

Java Threads: ( => current thread )
  0x000002c9bb39fb90 JavaThread "main"                              [_thread_blocked, id=9540, stack(0x000000dd84c00000,0x000000dd84d00000) (1024K)]
  0x000002c9d6f3e250 JavaThread "Reference Handler"          daemon [_thread_blocked, id=9416, stack(0x000000dd85400000,0x000000dd85500000) (1024K)]
  0x000002c9d6f3ed80 JavaThread "Finalizer"                  daemon [_thread_blocked, id=21680, stack(0x000000dd85500000,0x000000dd85600000) (1024K)]
  0x000002c9d6f48c20 JavaThread "Signal Dispatcher"          daemon [_thread_blocked, id=13612, stack(0x000000dd85600000,0x000000dd85700000) (1024K)]
  0x000002c9d6f4b700 JavaThread "Attach Listener"            daemon [_thread_blocked, id=772, stack(0x000000dd85700000,0x000000dd85800000) (1024K)]
  0x000002c9d6f52930 JavaThread "Service Thread"             daemon [_thread_blocked, id=20400, stack(0x000000dd85800000,0x000000dd85900000) (1024K)]
  0x000002c9d6f53740 JavaThread "Monitor Deflation Thread"   daemon [_thread_blocked, id=20476, stack(0x000000dd85900000,0x000000dd85a00000) (1024K)]
  0x000002c9d6f575e0 JavaThread "JVMCI-native CompilerThread0" daemon [_thread_blocked, id=2716, stack(0x000000dd85a00000,0x000000dd85b00000) (1024K)]
  0x000002c9d6f59bf0 JavaThread "C1 CompilerThread0"         daemon [_thread_blocked, id=9012, stack(0x000000dd85b00000,0x000000dd85c00000) (1024K)]
  0x000002ca1c12acd0 JavaThread "Common-Cleaner"             daemon [_thread_blocked, id=11452, stack(0x000000dd85c00000,0x000000dd85d00000) (1024K)]
  0x000002ca1c4377b0 JavaThread "Monitor Ctrl-Break"         daemon [_thread_in_native, id=19160, stack(0x000000dd85f00000,0x000000dd86000000) (1024K)]
  0x000002ca1c43a1a0 JavaThread "Notification Thread"        daemon [_thread_blocked, id=12584, stack(0x000000dd86000000,0x000000dd86100000) (1024K)]
  0x000002c986913010 JavaThread "Cleaner-0"                  daemon [_thread_blocked, id=7280, stack(0x000000dd86a00000,0x000000dd86b00000) (1024K)]
  0x000002c986f22900 JavaThread "Java2D Disposer"            daemon [_thread_blocked, id=12664, stack(0x000000dd87200000,0x000000dd87300000) (1024K)]
  0x000002c986f1c760 JavaThread "AWT-Windows"                daemon [_thread_in_native, id=8200, stack(0x000000dd87400000,0x000000dd87500000) (1024K)]
Total: 15

Other Threads:
=>0x000002c9d6f237f0 VMThread "VM Thread"                           [id=10312, stack(0x000000dd85300000,0x000000dd85400000) (1024K)]
  0x000002c9d6f0ac00 WatcherThread "VM Periodic Task Thread"        [id=16800, stack(0x000000dd85200000,0x000000dd85300000) (1024K)]
  0x000002c9bb3beba0 WorkerThread "GC Thread#0"                     [id=10944, stack(0x000000dd84d00000,0x000000dd84e00000) (1024K)]
  0x000002c986ed7a40 WorkerThread "GC Thread#1"                     [id=4672, stack(0x000000dd86b00000,0x000000dd86c00000) (1024K)]
  0x000002c986ed7df0 WorkerThread "GC Thread#2"                     [id=22468, stack(0x000000dd86c00000,0x000000dd86d00000) (1024K)]
  0x000002c986ee0490 WorkerThread "GC Thread#3"                     [id=12832, stack(0x000000dd86d00000,0x000000dd86e00000) (1024K)]
  0x000002c986ee0ca0 WorkerThread "GC Thread#4"                     [id=11124, stack(0x000000dd86e00000,0x000000dd86f00000) (1024K)]
  0x000002c986ee1860 WorkerThread "GC Thread#5"                     [id=22836, stack(0x000000dd86f00000,0x000000dd87000000) (1024K)]
  0x000002c986ee1c10 WorkerThread "GC Thread#6"                     [id=2976, stack(0x000000dd87000000,0x000000dd87100000) (1024K)]
  0x000002c986ee4a90 WorkerThread "GC Thread#7"                     [id=16996, stack(0x000000dd87100000,0x000000dd87200000) (1024K)]
  0x000002c9bb3c0b90 ConcurrentGCThread "G1 Main Marker"            [id=22072, stack(0x000000dd84e00000,0x000000dd84f00000) (1024K)]
  0x000002c9bb3c2860 WorkerThread "G1 Conc#0"                       [id=9748, stack(0x000000dd84f00000,0x000000dd85000000) (1024K)]
  0x000002c986f06290 WorkerThread "G1 Conc#1"                       [id=5188, stack(0x000000dd87300000,0x000000dd87400000) (1024K)]
  0x000002c9bb45d0a0 ConcurrentGCThread "G1 Refine#0"               [id=16136, stack(0x000000dd85000000,0x000000dd85100000) (1024K)]
  0x000002c9bb45d830 ConcurrentGCThread "G1 Service"                [id=15272, stack(0x000000dd85100000,0x000000dd85200000) (1024K)]
Total: 15

Threads with active compile tasks:
Total: 0

VM state: at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x00007ffa29e0cbc8] Threads_lock - owner thread: 0x000002c9d6f237f0
[0x00007ffa29e0ccc8] Heap_lock - owner thread: 0x000002c9bb39fb90

Heap address: 0x00000000f8000000, size: 128 MB, Compressed Oops mode: 32-bit

CDS archive(s) mapped at: [0x000002c9d7000000-0x000002c9d7c60000-0x000002c9d7c60000), size 12976128, SharedBaseAddress: 0x000002c9d7000000, ArchiveRelocationMode: 1.
Compressed class space mapped at: 0x000002c9d8000000-0x000002ca18000000, reserved size: 1073741824
Narrow klass base: 0x000002c9d7000000, Narrow klass shift: 0, Narrow klass range: 0x41000000

GC Precious Log:
 CardTable entry size: 512
 Card Set container configuration: InlinePtr #cards 5 size 8 Array Of Cards #cards 12 size 40 Howl #buckets 4 coarsen threshold 1843 Howl Bitmap #cards 512 size 80 coarsen threshold 460 Card regions per heap region 1 cards per card region 2048
 CPUs: 8 total, 8 available
 Memory: 8108M
 Large Page Support: Disabled
 NUMA Support: Disabled
 Compressed Oops: Enabled (32-bit)
 Heap Region Size: 1M
 Heap Min Capacity: 8M
 Heap Initial Capacity: 128M
 Heap Max Capacity: 128M
 Pre-touch: Disabled
 Parallel Workers: 8
 Concurrent Workers: 2
 Concurrent Refinement Workers: 8
 Periodic GC: Disabled

Heap:
 garbage-first heap   total reserved 131072K, committed 131072K, used 102751K [0x00000000f8000000, 0x0000000100000000)
  region size 1024K, 10 young (10240K), 4 survivors (4096K)
 Metaspace       used 14982K, committed 15360K, reserved 1114112K
  class space    used 1175K, committed 1344K, reserved 1048576K

Heap Regions: E=young(eden), S=young(survivor), O=old, HS=humongous(starts)

[spasi]

The log shows that the JVM itself is trying to allocate some memory and fails. Are you maybe running too many programs and the system as a whole runs OOM?

In any case, this doesn't look like an issue with LWJGL.

[fireinwinterfell]

The log shows that the JVM itself is trying to allocate some memory and fails. Are you maybe running too many programs and the system as a whole runs OOM?

In any case, this doesn't look like an issue with LWJGL.

Not really many programs, just a few like intellij and a web browser. But yes, the whole system get frozen. It ate up all mermory.

Here I found a chunk with exceptions in the log file, but currently I do not know how to render it:
Internal exceptions (20 events):
Event: 3.745 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fec54778}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeVirtual(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object)'> (0x00000000fec54778)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 3.796 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fecf1d08}: 'long java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java.lang.Object, java.lang.Object)'> (0x00000000fecf1d08)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 3.997 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/ClassNotFoundException'{0x00000000fe937b88}: javax/smartcardio/CardPermission> (0x00000000fe937b88)
thrown [s\open\src\hotspot\share\classfile\systemDictionary.cpp, line 313]
Event: 4.003 Thread 0x000002c9bb39fb90 Exception <a 'java/io/FileNotFoundException'{0x00000000fe947640}> (0x00000000fe947640)
thrown [s\open\src\hotspot\share\prims\jni.cpp, line 520]
Event: 4.083 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/ClassNotFoundException'{0x00000000fe8745b0}: sun/awt/resources/spi/awtProvider> (0x00000000fe8745b0)
thrown [s\open\src\hotspot\share\classfile\systemDictionary.cpp, line 313]
Event: 4.104 Thread 0x000002c9bb39fb90 Exception <a 'java/io/FileNotFoundException'{0x00000000fe8b5618}> (0x00000000fe8b5618)
thrown [s\open\src\hotspot\share\prims\jni.cpp, line 520]
Event: 4.115 Thread 0x000002c9bb39fb90 Exception <a 'java/io/FileNotFoundException'{0x00000000fe8e66b8}> (0x00000000fe8e66b8)
thrown [s\open\src\hotspot\share\prims\jni.cpp, line 520]
Event: 4.345 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000ffd3f8b0}: 'float java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, float, float)'> (0x00000000ffd3f8b0)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 5.883 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fc5f3870}: 'int java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object)'> (0x00000000fc5f3870)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 10.827 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fce6e108}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.newInvokeSpecial(java.lang.Object, java.lang.Object, java.lang.Object, double)'> (0x00000000fce6e108)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 10.827 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fce72350}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object, double)'> (0x00000000fce72350)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 10.828 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fce75cb0}: 'java.lang.Object java.lang.invoke.Invokers$Holder.linkToTargetMethod(java.lang.Object, java.lang.Object, double, java.lang.Object)'> (0x00000000fce75cb0)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 10.831 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fce7e4a8}: 'double java.lang.invoke.DirectMethodHandle$Holder.invokeVirtual(java.lang.Object, java.lang.Object)'> (0x00000000fce7e4a8)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 12.854 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fe208458}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object, java.lang.Object)'> (0x00000000fe208458)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 12.994 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fdf08dd8}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, long)'> (0x00000000fdf08dd8)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 12.995 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fdf0dd58}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.newInvokeSpecial(java.lang.Object, long)'> (0x00000000fdf0dd58)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 12.999 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fdf14590}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, long, java.lang.Object)'> (0x00000000fdf14590)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 13.000 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fdf17c18}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, long, java.lang.Object)'> (0x00000000fdf17c18)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 13.244 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fdead098}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object, long)'> (0x00000000fdead098)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]
Event: 13.245 Thread 0x000002c9bb39fb90 Exception <a 'java/lang/NoSuchMethodError'{0x00000000fdeb0b18}: 'java.lang.Object java.lang.invoke.Invokers$Holder.linkToTargetMethod(java.lang.Object, java.lang.Object, java.lang.Object, long, java.lang.Object)'> (0x00000000fdeb0b18)
thrown [s\open\src\hotspot\share\interpreter\linkResolver.cpp, line 774]