Memory leak that I can't trace.

[Aisaaax]

Hello. I've recently noticed a memory leak in my application, and upon some digging it seems that that C-code doesn't release the memory that was given to it by MemoryUtil.

First, here's how the problem looks on my end.
We have a simple script that tracks system memory used by JVM. We have limited the Heap maximum size, so all the deltas (in brackets, in KB) that we observe is the off-heap memory that gets allocated by the application.
The application itself is creating polygons, triangulating them and sending them as meshes to OpenGL. I'm doing this using MemoryUtil and FloatBuffer and IntBuffer (read the specifics below)

...
PSS: 408608 (4)
PSS: 408608 (0)
PSS: 408608 (0)
PSS: 408607 (-1)
PSS: 408608 (1)
PSS: 408608 (0)
PSS: 408608 (0)
PSS: 408612 (4)
PSS: 408612 (0)
PSS: 408612 (0)
PSS: 408631 (19)
PSS: 408712 (81)
PSS: 408740 (28)
PSS: 408840 (100)
PSS: 408875 (35)
PSS: 408943 (68)
PSS: 409024 (81)
PSS: 409084 (60)
PSS: 409208 (124)
...

As you can see, at first the application doesn't seem to consume much. Any changes that we see can be explained by other sources and other C code that is running. However after about 3-4 minutes we observe an abrupt spike, and it starts consuming 50-70 KB every 5 seconds, which is way more than any amount of data I'm actually passing to the GPU.

Now let me talk about how I'm creating and modifying meshes.
I used to do so roughly like this:

public Mesh(float[] vertexes, int[] indexes)
    {
        FloatBuffer vertexesBuffer = null;
        IntBuffer indexesBuffer = null;

        try
        {
            meshAccessor.acquire();

            int vboID;

            vaoID = glGenVertexArrays();
            vboIDList = new ArrayList<>();
            glBindVertexArray(vaoID);

            // Vertices VBO generation
            vboID = glGenBuffers();
            vboIDList.add(vboID);

            vertexCount = vertexes.length;
            this.vertexSize = vertexSize;
            vertexesBuffer = MemoryUtil.memAllocFloat(vertexSize);        // !!! Must Be manually freed!
            vertexesBuffer.put(vertexes).flip();        
            glBindBuffer(GL_ARRAY_BUFFER, vboID);
            glBufferData(GL_ARRAY_BUFFER, vertexesBuffer, GL_STATIC_DRAW);
            glVertexAttribPointer(0, vertexSize, GL_FLOAT, false, 0, 0);
            vertexAttrArrCount++;

            // Indices VBO generation
            vboID = glGenBuffers();
            vboIDList.add(vboID);
            indexCount = indexes.length;
            IntBuffer indexesBuffer = getIntBuffer(indexCount);
            indexesBuffer = MemoryUtil.memAllocInt(indexBufferLength);             // !!! Must be manually freed!
            indexesBuffer.put(indexes).flip();
            glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, vboID);
            glBufferData(GL_ELEMENT_ARRAY_BUFFER, indexesBuffer, GL_STATIC_DRAW);

            // unbinding
            glBindBuffer(GL_ARRAY_BUFFER, 0);
            glBindVertexArray(0);
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
        finally
        {
            if (vertexesBuffer != null) {
                MemoryUtil.memFree(vertexesBuffer);                             // Freeing vertex buffer
            }

            if (indexesBuffer != null) {
                MemoryUtil.memFree(indexesBuffer);                              // Freeing indices buffer
            }

            meshAccessor.release();
        }}

We have observed this leak in this version at first. It seemed to us that whatever memory we allocate for our buffers - we don't get it back. It DOES get "freed" by MemoryUtin.memFree, but it does not get returned to the system or to Java application. It kind of stays reserved, and so eventually we run out of memory.
We have reached that conclusion by creating bigger and bigger meshed incrementially and observing the program eating up that memory. But then if we reset the mesh size and start creating smaller meshes, i.e. smaller buffers - the new memory does not get allocated until we have reached the previous sizes, at which point it starts requesting new memory again.

Since then, we have tried moving to one static Floatbuffer and one Intbuffer. Basically the idea is to allocate the memory once, and then keep re-using it every time we need to send new data.
However the behavior seems to be pretty much the same, with the application running normally at first, and then starting to request obscene amounts of data.

I have since then narrowed the problem to mesh creation. If I leave everything intact (contour creation, triangulation, all other logic), but just comment the mesh class constructors and calls - the memory almost doesn't grow at all and we observe deltas within expected levels (0-4 KB).

Right now, I'm pretty much at a loss. It seems that even if I use the same buffer (the same memory) to send the data to lwjgl - It still makes copies of it somewhere internally and eventually runs out of pre-allocated space and starts requesting more from the system, without ever freeing it.

I've tried using debug allocator mode to see if it catches any leaks, but it doesn't. It reports just the 2 static objects that I've created and that I know are leaking at the moment.
Tried to use another allocator (jeallocator) and the result is the same.
Finally, tried looking into verbose:gc, and everything seems fine there.
I'm running it under Linux64 and Arm32 (linux OS) for the same result.

Basically, it looks like the program doesn't consider that memory to have been leaked, but if running for long enough it just takes everything and causes OutOfMemoryException.


Please help, even if just figuring out the next steps. Unfortunately I can't provide the project for study.

[jakethesnake]

If you get a java exception, then the problem is probably not GPU memory related. The way I've understood it is that a JVM uses two portions of memory. One for the JVM process and one for "system stuff". Buffers created by lwjgl uses this system stuff, that actually doesn't show up in the task manager to my knowledge.

I've had some trouble with this myself, and it results in a JVM fatal error crash, not an exception.

So if it's java heap space that's the problem, I recommend launching Java Visual JM, which is in your jdk folder, and have a look there. You should be able to see what's going on with your heap there.

[spasi]

Hey Aisaaax,

LWJGL does not make any copies. A copy of the data you pass to glBufferData may be stored in system/host memory (i.e. not on the GPU) by the OpenGL driver. This could account for the increase in memory usage of your process. That memory may not be released by the process until there's enough pressure (e.g. other applications that need memory).

However, you're seeing an OutOfMemoryException, which suggests a Java-side issue. It would be hard to make suggestions without a demo that reproduces this, but this code looks suspicious:

IntBuffer indexesBuffer = getIntBuffer(indexCount);
indexesBuffer = MemoryUtil.memAllocInt(indexBufferLength);

What does the getIntBuffer method do?

[Aisaaax]

Oh, sorry, this is the method in the new iteration that I talked about, that uses 1 single buffer for everything.

The proper OLD version looks like this:

public Mesh(float[] vertexes, int[] indexes)
    {
        FloatBuffer vertexesBuffer = null;
        IntBuffer indexesBuffer = null;

        try
        {
            meshAccessor.acquire();

            int vboID;

            vaoID = glGenVertexArrays();
            vboIDList = new ArrayList<>();
            glBindVertexArray(vaoID);

            // Vertices VBO generation
            vboID = glGenBuffers();
            vboIDList.add(vboID);

            vertexCount = vertexes.length;
            this.vertexSize = vertexSize;
            vertexesBuffer = MemoryUtil.memAllocFloat(vertexSize);        // !!! Must Be manually freed!
            vertexesBuffer.put(vertexes).flip();       
            glBindBuffer(GL_ARRAY_BUFFER, vboID);
            glBufferData(GL_ARRAY_BUFFER, vertexesBuffer, GL_STATIC_DRAW);
            glVertexAttribPointer(0, vertexSize, GL_FLOAT, false, 0, 0);
            vertexAttrArrCount++;

            // Indices VBO generation
            vboID = glGenBuffers();
            vboIDList.add(vboID);
            indexCount = indexes.length;
            indexesBuffer = MemoryUtil.memAllocInt(indexBufferLength);             // !!! Must be manually freed!
            indexesBuffer.put(indexes).flip();
            glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, vboID);
            glBufferData(GL_ELEMENT_ARRAY_BUFFER, indexesBuffer, GL_STATIC_DRAW);

            // unbinding
            glBindBuffer(GL_ARRAY_BUFFER, 0);
            glBindVertexArray(0);
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
        finally
        {
            if (vertexesBuffer != null) {
                MemoryUtil.memFree(vertexesBuffer);                             // Freeing vertex buffer
            }

            if (indexesBuffer != null) {
                MemoryUtil.memFree(indexesBuffer);                              // Freeing indices buffer
            }

            meshAccessor.release();
        }}

So that line wasn't supposed to be there, I forgot to remove it when posting the code.

Just in case, here's the BufferController class that holds that method:

public class BufferController
{
    protected static FloatBuffer floatBuffer             = MemoryUtil.memAllocFloat(1000000);
    protected static IntBuffer intBuffer               = MemoryUtil.memAllocInt(1000000);
    protected static ByteBuffer byteBuffer              = ByteBuffer.allocateDirect(1000000);

    public static FloatBuffer getFloatBuffer(float[] array)
    {
        floatBuffer.clear();
        floatBuffer.put(array);
        floatBuffer.flip();
//        FloatBuffer result = MemoryUtil.memSlice(floatBuffer,0,array.length);
        return floatBuffer;
    }

    public static FloatBuffer getFloatBuffer(int size)
    {
        floatBuffer.clear();
        floatBuffer.put(new float[size]);
        floatBuffer.flip();
        return floatBuffer;
    }

    public static IntBuffer getIntBuffer(int[] array)
    {
        intBuffer.clear();
        intBuffer.put(array);
        intBuffer.flip();
        return intBuffer;
    }

    public static IntBuffer getIntBuffer(int size)
    {
        intBuffer.clear();
        intBuffer.put(new int[size]);
        intBuffer.flip();
        return intBuffer;
    }

    public static ByteBuffer getByteBuffer(byte[] array)
    {
        byteBuffer.clear();
        byteBuffer.put(array);
        byteBuffer.flip();
        ByteBuffer result = MemoryUtil.memSlice(byteBuffer,0,array.length);
        return result;
    }

    public static ByteBuffer getByteBuffer(int size)
    {
        byteBuffer.clear();
        ByteBuffer result = MemoryUtil.memSlice(byteBuffer,0,size);
        return result;
    }
}

This was an attemt to prevent java from allocating extra memory for buffers, because at some point we thought that that's the issue. I might revert that change in the future or leave it, because in my mind not processing new allocatuins is faster.

Anyway here's the code that I run currently that uses this class. I should note that the memory behavior is similar in both cases.

public Mesh(float[] vertexes, int[] indexes, int lengthStep, int vertexSize, int vertexLayoutSlot)
    {
        bufferLengthStep = lengthStep;

        try
        {
            meshAccessor.acquire();

            int vboID;

            vaoID = glGenVertexArrays();
            vboIDList = new ArrayList<>();
            glBindVertexArray(vaoID);

            // Vertices VBO generation
            vboID = glGenBuffers();
            vboIDList.add(vboID);

            vertexCount = vertexes.length;
            this.vertexSize = vertexSize;
            vertexBufferLength = calculateBufferLength(vertexCount, vertexSize);

            FloatBuffer vertexesBuffer = getFloatBuffer(vertexes);

            glBindBuffer(GL_ARRAY_BUFFER, vboID);
            glBufferData(GL_ARRAY_BUFFER, vertexesBuffer, GL_STATIC_DRAW);
            glVertexAttribPointer(vertexLayoutSlot, vertexSize, GL_FLOAT, false, 0, 0);
            vertexAttrArrCount++;

            // Indices VBO generation
            vboID = glGenBuffers();
            vboIDList.add(vboID);
            indexCount = indexes.length;
            indexBufferLength = calculateBufferLength(indexCount, vertexSize);

            IntBuffer indexesBuffer = getIntBuffer(indexes);
            
            glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, vboID);
            glBufferData(GL_ELEMENT_ARRAY_BUFFER, indexesBuffer, GL_STATIC_DRAW);

            // unbinding
            glBindBuffer(GL_ARRAY_BUFFER, 0);
            glBindVertexArray(0);
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
        finally
        {
            meshAccessor.release();
        }
    }

As for memory leak happening elsewhere in java, as I said, I've tested it with everything else enabled BUT the mesh construction. Basically, think of it as having a return statement in the beginning of that method that skips everything here. So everything else in the program works, all the triangulation, array shuffling, data exchange between various threads, etc. No memory leaks occur. But once I return mesh creation - it leaks.

I have an idea today to keep everything in this method but only remove openGL calls. if I don't see any leak, this should mean that the memory gobbling happens somewhere in the native code. Not necessarily LWJGL, but maybe driver.

[spasi]

I don't see how an OutOfMemoryException is possible with the above code. Could you post the full stacktrace please?

[Aisaaax]

The stacktrace won't help. Because the exception occurs randomly in random parts of the program. It's a toss of the coin when the memory would end and some new variable can't be created.

[spasi]

Sounds like there's something else about the Mesh class that's causing the issue. The memAlloc calls allocate off-heap memory, which has nothing to do with the Java heap and won't cause an OOME. The buffer objects are simple Java instances that wrap the off-heap addresses, they're really small and are eligible for GC as soon as the Mesh constructor returns.

[KaiHH]

It has been suggested before that you should use a Java memory profiler, like JVisualVM to diagnose the problem and not stab in the dark and guess as to what the issue _could_ be.
Just let your application run uninstrumented for some time and then take a heap snapshot which you can then analyze.
And if that is not enough and you want to know which code is actually allocating what objects, then you need to run your application via CPU instrumentation in the profiler, which will result in more memory being allocated due to instrumentation and more calls/objects escaping - but it will still show you your actual problem.

[Aisaaax]

Found these yesterday:
https://www.opennet.ru/tips/3184_telegram_memory_debug.shtml
https://github.com/jemalloc/jemalloc/issues/2069

It seems that the problem may actually have something to do with c core under linux, and that it doesn't properly return memory to the system kernel. I will investigate this today, including trying the older version of jemalloc that seems to not have that issue. I'll post the results later, but still decided to make this update.

[Aisaaax]

Ok, so the problem was on my side after all. I was forgetting to clean up some of the meshes, to delete VBO's. The worst thing is that I noticed that by accident while tinkering with another part of the code.