[FIXED]Out of memory. MAYDAY! please help

[jakethesnake]

So I've had a bug for quite some time now. It doesn't happen to me on my computer, nor any of the handful of computers I have access to, but as I've recently started distributing my app I get reports concerning it. This seems to happen exclusively on 32b windows

It has alwas manifested itself when textures are created. Either by STBImage.stbi_load or by glTexImage2D. And it's always at the same stage in the application, when two 4096x4096 .png textures are supposed to be uploaded to the GPU. I get the errors from either glGetError() or STBImage.stbi_failure_reason().

As I have stated, I don't have this problem on my rig. On my rig windows task manager shows that the JVM process is using ~130MB. GPUZ is showing used gpu memory is 1000MB, an increase of ~200MB when not running the app. In other words, my app is using rougly 130MB ram and 200MB vram. And that is completely according to my own calculations. And it's not growing.

I'm not doing anything very fancy in my app. I have 2 RBG8 textures, 4096x4096. A couple of FBO's  the same size as the screen and about 5MB worth of VBO's. Everything works for 90% of the population of earth.

This is the flow of the program leading up to the bug:

GLFW & OpenGl & OpenAL Initialization
2x4096*256 textures created
vbo's are streamed (glbuffersubdata each frame)
User clicks play
textures are released
2*4096*4096 textures are created
Out of memory!


My theory is this:

There really should be enough memory around. The streaming VBO's fragment the off heap memory to a state where there isn't enough room for 64MB + overhead to be allocated in a continuous chunk. But instead of freeing memory or compacting the heap, it fails and I have to crash. Could also be page size?

I haven't found any explanation on how to monitor off heap memory allocated by lwjgl. If it's hiding somewhere, I want to see it.


So, the app is out there, if any of you would like to help. This might seem like a call for attention to my app, but I assure you it's not. I'm going insane from this bug. One of us needs o be destroyed.

https://songsofsyx.itch.io/songs-of-syx

The bug should manifest itself when you press play in the main menu.

I've recently got this error report from one of those who have this issue, where I've intentionally crashed the app prior to the bug and printed out memory info.

MEM DIAGNOSE
--JRE Memory
--JRE Total: 499
--JRE Free: 477
--JRE Used: 22
--JRE Max: 999
NVIDIA: 
--GPU Dedicated: 0
--GPU Total Available: 0
--GPU Current Available: 0
--GPU Evictions: 0
--GPU Evicted: 0
ATI: 
--Renderbuffer Free: 7265316
--Texture Free: 7265316
--Vbo Free: 7265316
|---JRE Input Arguments : -Xms512m, -Xmx1024m, 

[LWJGL] Version: 3.1.2 build 29
[LWJGL]                OS: Windows 10 v10.0
[LWJGL]               JRE: 1.8.0_221 x86
[LWJGL]               JVM: Java HotSpot(TM) Client VM v25.221-b11 by Oracle Corporation
[LWJGL] Loading library (system): lwjgl32
[LWJGL]               Found at: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\lwjgl32.dll
[LWJGL]               Loaded from org.lwjgl.librarypath: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\lwjgl32.dll
[LWJGL] MemoryUtil accessor: MemoryAccessorUnsafe
[LWJGL] Loading library: jemalloc32
[LWJGL]               Found at: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\jemalloc32.dll
[LWJGL]               Loaded from org.lwjgl.librarypath: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\jemalloc32.dll
[LWJGL] MemoryUtil allocator: DebugAllocator
[LWJGL] Loading library: glfw32
[LWJGL]               Found at: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\glfw32.dll
[LWJGL]               Loaded from org.lwjgl.librarypath: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\glfw32.dll
[LWJGL] Loading library (system): lwjgl_stb32
[LWJGL]               Found at: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\lwjgl_stb32.dll
[LWJGL]               Loaded from org.lwjgl.librarypath: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\lwjgl_stb32.dll
[LWJGL] Loading library (system): lwjgl_opengl32
[LWJGL]               Found at: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\lwjgl_opengl32.dll
[LWJGL]               Loaded from org.lwjgl.librarypath: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\lwjgl_opengl32.dll
[LWJGL] Loading library: opengl32
[LWJGL]               opengl32.dll not found in org.lwjgl.librarypath=C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29
[LWJGL]               Loaded from system paths
[LWJGL] [GL] Using KHR_debug for error logging.
[LWJGL] [GL] Warning: A non-debug context may not produce any debug output.
---Max Texture Dim: 16384
---Version: 3.3.13558 Core Profile Forward-Compatible Context 26.20.11015.5009
---SL Version: 4.60
---glRenderer: ATI Technologies Inc., Radeon RX Vega
---Forward compatible: true

SHADER COMPILED: class snake2d.VboSpritePoints$ShaderDebug 1
SHADER COMPILED: class snake2d.VboParticles$ShaderDebug 5
SOUND
[LWJGL] Loading library: OpenAL32
[LWJGL]               Found at: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\OpenAL32.dll
[LWJGL]               Loaded from org.lwjgl.librarypath: C:\Users\tobip\AppData\Local\Temp\lwjgltobip\3.1.2-build-29\OpenAL32.dll
---AL version : 1.1 ALSOFT 1.17.2
---AL vendor : OpenAL Community
---AL renderer : OpenAL Soft
---OpenALC10: true
---OpenALC11: true
---ALC_FREQUENCY: 48000Hz
---ALC_REFRESH: 50Hz
---ALC_SYNC: false
---Created Mono Sources : 10
---Created Stereo Sources : 6

SHADER COMPILED: class snake2d.VboParticles$ShaderTexture 8
---Initiating random, seed: 6663922946396309042

class snake2d.SoundCore sucessfully destroyed
class snake2d.GraphicContext was successfully destroyed
Core was successfully disposed

The MEM-DIAGNOSE code looks like this:

static void diagnozeMem() {
		
		int mb = 1014*1024;
		
		
		System.err.println("MEM DIAGNOSE");
		Runtime run = Runtime.getRuntime();
		// available memory
		System.err.println("--JRE Memory");
		System.err.println("--JRE Total: " + run.totalMemory() / mb);
		System.err.println("--JRE Free: " + run.freeMemory() / mb);
		System.err.println("--JRE Used: "
				+ (run.totalMemory() - run.freeMemory()) / mb);
		System.err.println("--JRE Max: " + run.maxMemory() / mb);
		int i;
		System.err.println("NVIDIA: ");
		i = glGetInteger(org.lwjgl.opengl.NVXGPUMemoryInfo.GL_GPU_MEMORY_INFO_DEDICATED_VIDMEM_NVX);
		System.err.println("--GPU Dedicated: " + i);
		i = glGetInteger(org.lwjgl.opengl.NVXGPUMemoryInfo.GL_GPU_MEMORY_INFO_TOTAL_AVAILABLE_MEMORY_NVX);
		System.err.println("--GPU Total Available: " + i);
		i = glGetInteger(org.lwjgl.opengl.NVXGPUMemoryInfo.GL_GPU_MEMORY_INFO_CURRENT_AVAILABLE_VIDMEM_NVX);
		System.err.println("--GPU Current Available: " + i);
		i = glGetInteger(org.lwjgl.opengl.NVXGPUMemoryInfo.GL_GPU_MEMORY_INFO_EVICTION_COUNT_NVX);
		System.err.println("--GPU Evictions: " + i);
		i = glGetInteger(org.lwjgl.opengl.NVXGPUMemoryInfo.GL_GPU_MEMORY_INFO_EVICTED_MEMORY_NVX);
		System.err.println("--GPU Evicted: " + i);
		
		System.err.println("ATI: ");
		i = glGetInteger(org.lwjgl.opengl.ATIMeminfo.GL_RENDERBUFFER_FREE_MEMORY_ATI);
		System.err.println("--Renderbuffer Free: " + i);
		i = glGetInteger(org.lwjgl.opengl.ATIMeminfo.GL_TEXTURE_FREE_MEMORY_ATI);
		System.err.println("--Texture Free: " + i);
		i = glGetInteger(org.lwjgl.opengl.ATIMeminfo.GL_VBO_FREE_MEMORY_ATI);
		System.err.println("--Vbo Free: " + i);
		glGetError();
	}

This person was on ATI and I have no idea what it returns. Bytes, KB's MB's? Who knows. If it's bytes, then his 8GB GPU is indeed out of memory, but I wager it's KB's and that it's RAM that is suffering. The JVM tells me nothing useful.

Here are a few key point of my code. I've read through them hundreds of times myself and can't find anything.


Texture loading:

package snake2d.util.file;

import java.io.File;
import java.nio.ByteBuffer;
import java.nio.IntBuffer;

import org.lwjgl.BufferUtils;
import org.lwjgl.stb.STBImage;
import org.lwjgl.stb.STBImageWrite;
import org.lwjgl.system.MemoryStack;

import snake2d.util.misc.ErrorHandler;

public final class SnakeImage {

	private final ByteBuffer image;
	public final int height;
	public final int width;
	public final ImageGraphics rgb = new ImageGraphics();
	
	public SnakeImage(String path) {

		if (!new File(path).exists())
			throw new ErrorHandler.DataError("File doesn't exist", path);
		
		try (MemoryStack stack = MemoryStack.stackPush()) {

			IntBuffer w = stack.mallocInt(1);
			IntBuffer h = stack.mallocInt(1);
			IntBuffer comp = stack.mallocInt(1);

			image = STBImage.stbi_load(path, w, h, comp, 4);
			
//			if (comp.get() != 4)
//				throw new ErrorHandler.DataError("Failed to load a texture file!"
//						+ "not RGBA data", path);

			if (image == null)
				throw new RuntimeException("Failed to load a texture file!"
						+ System.lineSeparator() + STBImage.stbi_failure_reason());

			width = w.get();
			height = h.get();

		}
		
	}
	
	public SnakeImage(String path, int width, int height) {
		if (!new File(path).exists())
			throw new ErrorHandler.DataError("File doesn't exist", path);
		
		try (MemoryStack stack = MemoryStack.stackPush()) {

			IntBuffer w = stack.mallocInt(1);
			IntBuffer h = stack.mallocInt(1);
			IntBuffer comp = stack.mallocInt(1);
			image = STBImage.stbi_load(path, w, h, comp, 4);
			if (comp.get() != 4)
				throw new ErrorHandler.DataError("Failed to load a texture file!"
						+ "not RGBA data", path);

			if (image == null)
				throw new RuntimeException("Failed to load a texture file!"
						+ System.lineSeparator() + STBImage.stbi_failure_reason());

			this.width = w.get();
			this.height = h.get();
			
			if (this.width != width || this.height != height) {
				throw new ErrorHandler.DataError("Image has wrong dimentions. Resize to: " + width + "x" + height, path);
			}

		}
	}
	
	public SnakeImage(int width, int height){
		try {
			image = BufferUtils.createByteBuffer(width*height*4);
		}catch(OutOfMemoryError e) {
			e.printStackTrace();
			int s = width*height*4;
			throw new RuntimeException(s + " " + Runtime.getRuntime().totalMemory() + " " + Runtime.getRuntime().freeMemory());
			
		}
		
		this.width = width;
		this.height = height;
	}
	
	public ByteBuffer data() {
		image.rewind();
		return image;
	}
	

	
	public void save(String path) {
		image.rewind();
		STBImageWrite.stbi_write_png(path, width, height, 4, image, 0);
	}
	
	public final class ImageGraphics {
		
		private ImageGraphics() {
			
		}
		
		private void boundCheck(int x, int y) {
			if (x < 0 || y < 0 || x >= width || y >= height)
				throw new RuntimeException(x + " " +y  + "is out of bounds");
		}
		
		public void set(int x, int y, int r, int g, int b, int a) {
			boundCheck(x, y);
			int i = 4*(x + y*width);
			image.position(i);
			image.put((byte)r).put((byte)g).put((byte)b).put((byte)a);
		}
		
		public void set(int x, int y, int c) {
			int r = (c >> 24) & 0x0FF;
			int g = (c >> 16) & 0x0FF;
			int b = (c >> 8) & 0x0FF;
			int a = (c) & 0x0FF;
			set(x, y, r, g, b, a);
		}
		
		public int get(int x, int y) {
			boundCheck(x, y);
			int i = 4*(x + y*width);
			image.position(i);
			int res = (image.get() & 0x0FF) << 24;
			res |= (image.get() & 0x0FF) << 16;
			res |= (image.get() & 0x0FF) << 8;
			res |= (image.get() & 0x0FF);
			return res;
		}
		
	}

}

texture opengl

package snake2d;

import static org.lwjgl.opengl.GL11.*;
import static org.lwjgl.opengl.GL13.*;
import static org.lwjgl.opengl.GL31.*;

import java.nio.ByteBuffer;

import org.lwjgl.opengl.GL11;

import snake2d.util.file.SnakeImage;

class _TextureDiffuse {
	
	final int id;
	final int width;
	final int height;
	
	private boolean desposed = false;
	
	private static int MAX_SIZE = 0x04000;

	_TextureDiffuse(SnakeImage i, boolean pixelated){
		
		width = i.width;
		height = i.height;
		
		if (width > MAX_SIZE || height > MAX_SIZE)
			throw new RuntimeException();
		
		id = glGenTextures();
		
		glActiveTexture(GL_TEXTURE0);
		glBindTexture(GL_TEXTURE_RECTANGLE, id);
		
		//some strange filters. Experiment!
		glTexParameteri(GL_TEXTURE_RECTANGLE, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
		glTexParameteri(GL_TEXTURE_RECTANGLE, GL_TEXTURE_MAG_FILTER, pixelated ? GL_NEAREST : GL_LINEAR);
		glTexImage2D(GL_TEXTURE_RECTANGLE, 0, GL_RGBA8, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, i.data());
		
		int e = GL11.glGetError();
		if (e != GL_NO_ERROR) {
			GlHelper.diagnozeMem();
			throw new RuntimeException("Texture Error " + width + " " + height + " " + e);
		}
		
		
		GlHelper.checkErrors();

	}
	
	public void bind(){
		if (desposed)
			throw new IllegalStateException("trying to bind a texture that was disposed");
		else
			glActiveTexture(GL_TEXTURE0);
			glBindTexture(GL_TEXTURE_RECTANGLE, id);
	}
	
	void dis() {
		glDeleteTextures(id);
		GlHelper.checkErrors();
		desposed = true;
	}
	
	public void uploadPixels(int px, int width, int py, int height, ByteBuffer pixels){
		glActiveTexture(GL_TEXTURE0);
		glTexSubImage2D(GL_TEXTURE_RECTANGLE, 0, px, py, width, height, GL_RGBA, GL_UNSIGNED_BYTE, pixels);
	}
	
	public void uploadPixel(int px, int py, ByteBuffer pixel){
		glActiveTexture(GL_TEXTURE0);
		glTexSubImage2D(GL_TEXTURE_RECTANGLE, 0, px, py, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, pixel);
	}
	
	public int getWidth(){return width;}
	public int getHeight(){return height;}
	
}

VBO

package snake2d;

import static org.lwjgl.opengl.GL11.*;
import static org.lwjgl.opengl.GL15.*;
import static org.lwjgl.opengl.GL20.*;
import static org.lwjgl.opengl.GL30.*;

import java.nio.ByteBuffer;

import org.lwjgl.opengl.GL11;
import org.lwjgl.system.MemoryUtil;

abstract class VboAbs {

    private final int vertexArrayID;
    private final int attributeElementID;
    
    final ByteBuffer buffer;
    
    final int MAX_ELEMENTS;
    final int ELEMENT_SIZE;
    private final int BUFFER_SIZE;
    
    private final int NR_OF_ATTRIBUTES;
    
    protected int count = 0;
    
	protected int[] vFrom = new int[255];
	protected int[] vTo = new int[255];
	protected int current = 0;
	private final int type;
	private final int indexMul;
    
	VboAbs(int type, int maxElements, VboAttribute... attributes){
		
		MAX_ELEMENTS = maxElements;
		NR_OF_ATTRIBUTES = attributes.length;
		
		this.type = type;
		int vertecies = 0;
		if (type == GL_TRIANGLES) {
			indexMul = 6;
			vertecies = 4;
		}else if (type == GL_POINTS) {
			indexMul = 1;
			vertecies = 1;
		}else {
			throw new RuntimeException("unsupported type");
		}

		int byteStride = 0;
		for (VboAttribute v : attributes) {
			byteStride += v.sizeInBytes;
		}
		
		if (byteStride % 4 != 0)
			throw new RuntimeException(byteStride + " Needs padding with " + (4 - (byteStride % 4)));

		ELEMENT_SIZE = byteStride*vertecies;
		BUFFER_SIZE = ELEMENT_SIZE * MAX_ELEMENTS;
		buffer = MemoryUtil.memAlloc(BUFFER_SIZE);

		vertexArrayID = glGenVertexArrays();
		glBindVertexArray(vertexArrayID);

		attributeElementID = glGenBuffers();
		glBindBuffer(GL_ARRAY_BUFFER, attributeElementID);

		int index = 0;
		int pointerOffset = 0;
		for (VboAttribute v : attributes) {
			if (v.isInt)
				glVertexAttribIPointer(index, v.amount, v.glType, byteStride, pointerOffset);
			else
				glVertexAttribPointer(index, v.amount, v.glType, v.normalized, byteStride, pointerOffset);
			index ++;
			pointerOffset += v.sizeInBytes;
		}

		glBufferData(GL_ARRAY_BUFFER, buffer, GL_STREAM_DRAW);
		if (type == GL_TRIANGLES) {
			VboElementArrays.quadBind();
		}else if (type == GL_POINTS) {
			VboElementArrays.pointBind();
		}
		
		
		glBindVertexArray(0);

		GlHelper.checkErrors();

		
	}
    
    final void flush(int from, int to){

    	if (from == to)
    		return;
		glDrawElements(type, (to - from)*indexMul, GL11.GL_UNSIGNED_INT, from * 4* indexMul);
    	
    }
    
    void clear(){
    	current = 0;
    	buffer.clear();
    	count = 0;
    }
    
    void dis(){
    	
    	// Disable the VBO index from the VAO attributes list
    	glBindVertexArray(vertexArrayID);

		glBindBuffer(GL_ARRAY_BUFFER, attributeElementID);

		for (int i = 0; i < NR_OF_ATTRIBUTES; i++) {
			glDisableVertexAttribArray(i);
		}

    	// Dispose the buffer object
        glBindBuffer(GL_ARRAY_BUFFER, 0);
        glDeleteBuffers(attributeElementID);
    	
    	// Dispose the vertex array
        glBindVertexArray(0);
        glDeleteVertexArrays(vertexArrayID);


        // Dispose the element buffer object
        VboElementArrays.dispose();
        GlHelper.checkErrors();
        
        MemoryUtil.memFree(buffer);
    }
    
	final void bindAndUpload() {
		if (count == 0) {
			return;
		}

		buffer.flip();

		bind();

		glBufferSubData(GL_ARRAY_BUFFER, 0, buffer);

	}

	void bind() {
		glBindVertexArray(vertexArrayID);

		glBindBuffer(GL_ARRAY_BUFFER, attributeElementID);

		for (int i = 0; i < NR_OF_ATTRIBUTES; i++) {
			glEnableVertexAttribArray(i);
		}
		
		
	}
	
	static class VboAttribute {

		private final boolean isInt;
		private final int amount;
		private final int glType;
		private final int sizeInBytes;
		private final boolean normalized;
		
		public VboAttribute(int amount, int glType, boolean normalized, int sizeInBytes){
			isInt = false;
			this.amount = amount;
			this.glType = glType;
			this.sizeInBytes = sizeInBytes*amount;
			this.normalized = normalized;
		}

		public VboAttribute(int amount, int glType, int sizeInBytes){
			isInt = true;
			this.amount = amount;
			this.glType = glType;
			this.sizeInBytes = sizeInBytes*amount;
			this.normalized = false;
		}
		
	}

}

If you can trigger the bug, or if you see anything in my code that could be a memory leak, please help me! I can't sleep any more...

[jakethesnake]

Good news, I can now reproduce the problem.

I ran my app on a 32-bit JVM and boom!

It is completely dependant on what max heap-size arguments I set for the JVM (-Xmx)

The app works perfectly on -Xmx256m. Less and I actually run out of JVM (non-native) heap when allocating buffers. This is expected behavior.
Then, as I increase max heap the app has a higher chance of crashing. Reading the heap usage before these crashes indicated that I have plenty of heap space left for this allocation. It must then be direct memory that is the problem.
-Xmx1500m the game always crashes.

So, the higher I set the memory limit, the higher the chance that the app runs out of memory...

I've found out that direct memory is set to the same size as the heap. Could it then be that there's some boundary somwhere, which causes the problem? I have no idea.

I really want to get to the bottom of this.

[spasi]

I can reproduce this too, on Windows 10 with Java 8 x86 and -Xmx1500m.

It always fails in SnakeImage when trying to stbi_load res/data/texture/cache/texture/diffuse.png (a 4kx4k image). Afaict, this is a simple exhaustion of memory address space in the 32-bit process. The JVM has claimed most of the 2GB available (even if it's unused) and malloc fails when trying to claim enough memory for the image buffer.

The issue is easily fixable by launching the game with a reasonable -Xmx. It works fine with -Xmx128m, without frequent GCs happening. You could probably go lower than that, if you avoid BufferUtils (i.e. ByteBuffer.allocateDirect and the -XX:MaxDirectMemorySize limit) and use MemoryUtil for more allocations.

Btw, on monitoring memory allocations: in LWJGL 3.2.0 or higher (prefer the latest version) the debug allocator tracks allocations happening in shared libraries when possible. This includes the stb library. I tried the game with 3.2.0 (works fine), but I couldn't get the debug allocator to work with -Dorg.lwjgl.util.Debug=true -Dorg.lwjgl.util.DebugAllocator=true. The problem seems to be in the GraphicContext constructor, you're calling Configuration.DEBUG_MEMORY_ALLOCATOR.set(sett.debugMode()), which is always false and overrides the JVM argument. If you fix that, you should be able to monitor all off-heap allocations done via MemoryUtil and inside shared libraries. Also see the MemoyUtil.memReport methods.

[jakethesnake]

Thank you spasi!

Where did you go to wizard school?

I'll sure enable DebugAllocator in the next release. But when I enable it now it doesn't print anything, not even when I trigger a crash. I only get this:

Game version : Embryo 1.0

Input Arguments : 
Starting the Launcher....

SYSTEM INFO
---Running on a: Windows 10, x86 Platform.  <= [b]Interesting how the JVM thinks I'm on a 32 bit architecture only because I'm running a 32-bit JVM ??? 
[/b]---jre: 1.8.0_221 bits: 32
---JRE Input Arguments : -Xmx1500m, -Dfile.encoding=Cp1252, 
[LWJGL] Version: 3.1.2 build 29
[LWJGL] 	 OS: Windows 10 v10.0
[LWJGL] 	JRE: 1.8.0_221 x86
[LWJGL] 	JVM: Java HotSpot(TM) Client VM v25.221-b11 by Oracle Corporation
[LWJGL] Loading library (system): lwjgl32
[LWJGL] 	Found at: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\lwjgl32.dll
[LWJGL] 	Loaded from org.lwjgl.librarypath: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\lwjgl32.dll
[LWJGL] MemoryUtil accessor: MemoryAccessorUnsafe
[LWJGL] Loading library: jemalloc32
[LWJGL] 	Found at: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\jemalloc32.dll
[LWJGL] 	Loaded from org.lwjgl.librarypath: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\jemalloc32.dll
[LWJGL] MemoryUtil allocator: DebugAllocator
[LWJGL] Loading library: glfw32
[LWJGL] 	Found at: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\glfw32.dll
[LWJGL] 	Loaded from org.lwjgl.librarypath: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\glfw32.dll
[LWJGL] Loading library (system): lwjgl_stb32
[LWJGL] 	Found at: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\lwjgl_stb32.dll
[LWJGL] 	Loaded from org.lwjgl.librarypath: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\lwjgl_stb32.dll
[LWJGL] Loading library (system): lwjgl_opengl32
[LWJGL] 	Found at: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\lwjgl_opengl32.dll
[LWJGL] 	Loaded from org.lwjgl.librarypath: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\lwjgl_opengl32.dll
[LWJGL] Loading library: opengl32
[LWJGL] 	opengl32.dll not found in org.lwjgl.librarypath=C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29
[LWJGL] 	Loaded from system paths
[LWJGL] [GL] GL_ARB_shader_subroutine was reported as available but an entry point is missing.
[LWJGL] [GL] Using KHR_debug for error logging.
[LWJGL] Loading library: OpenAL32
[LWJGL] 	Found at: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\OpenAL32.dll
[LWJGL] 	Loaded from org.lwjgl.librarypath: C:\Users\MAIL__~1\AppData\Local\Temp\lwjglmail__000\3.1.2-build-29\OpenAL32.dll

Is there something wrong with my setup? I put a fresh 3.2.2 build 10 lwjgl into the core yesterday.

Anyway. This is solved, I suppose. It has been bothering me for many months. And it's not exactly intuitive.

I can reproduce this too, on Windows 10 with Java 8 x86 and -Xmx1500m.

It always fails in SnakeImage when trying to stbi_load res/data/texture/cache/texture/diffuse.png (a 4kx4k image). Afaict, this is a simple exhaustion of memory address space in the 32-bit process. The JVM has claimed most of the 2GB available (even if it's unused) and malloc fails when trying to claim enough memory for the image buffer.

The issue is easily fixable by launching the game with a reasonable -Xmx. It works fine with -Xmx128m, without frequent GCs happening. You could probably go lower than that, if you avoid BufferUtils (i.e. ByteBuffer.allocateDirect and the -XX:MaxDirectMemorySize limit) and use MemoryUtil for more allocations.

Btw, on monitoring memory allocations: in LWJGL 3.2.0 or higher (prefer the latest version) the debug allocator tracks allocations happening in shared libraries when possible. This includes the stb library. I tried the game with 3.2.0 (works fine), but I couldn't get the debug allocator to work with -Dorg.lwjgl.util.Debug=true -Dorg.lwjgl.util.DebugAllocator=true. The problem seems to be in the GraphicContext constructor, you're calling Configuration.DEBUG_MEMORY_ALLOCATOR.set(sett.debugMode()), which is always false and overrides the JVM argument. If you fix that, you should be able to monitor all off-heap allocations done via MemoryUtil and inside shared libraries. Also see the MemoyUtil.memReport methods.

[spasi]

Interesting how the JVM thinks I'm on a 32 bit architecture only because I'm running a 32-bit JVM

As far as the JVM is concerned, there's no difference between running a 32-bit process on a 64-bit OS and running a 32-bit process on a 32-bit OS.

Btw, other than using an explicit -Xmx parameter, you should also make sure to run with the -server parameter on Windows x86. Without that parameter, Java launches with the Client VM by default, which is REALLY bad for peak performance. This is not an issue on x64 (there's no client VM, only server).

Is there something wrong with my setup? I put a fresh 3.2.2 build 10 lwjgl into the core yesterday.

The log says "Version: 3.1.2 build 29", the classpath is still picking up 3.1.2 somehow. Make sure it's not baked into the SyxProto001.jar.

[jakethesnake]

Thanks! I had f****d up again of course. Now it works like a charm, and there were actually a few memory leaks. (STBImage.stbi_load)

I think the Javadoc of this method would benefit from saying that one needs to do stbi_free. I've been wondering over this myself and haven't found any info.

Wonderful feature that memory debug thing!

Interesting how the JVM thinks I'm on a 32 bit architecture only because I'm running a 32-bit JVM

As far as the JVM is concerned, there's no difference between running a 32-bit process on a 64-bit OS and running a 32-bit process on a 32-bit OS.

Btw, other than using an explicit -Xmx parameter, you should also make sure to run with the -server parameter on Windows x86. Without that parameter, Java launches with the Client VM by default, which is REALLY bad for peak performance. This is not an issue on x64 (there's no client VM, only server).

Is there something wrong with my setup? I put a fresh 3.2.2 build 10 lwjgl into the core yesterday.

The log says "Version: 3.1.2 build 29", the classpath is still picking up 3.1.2 somehow. Make sure it's not baked into the SyxProto001.jar.