How is native memory handled in LWJGL?

[bcbradle]

For instance, there seems to be 3 or 4 different ways to create https://javadoc.lwjgl.org/org/lwjgl/vulkan/VkAllocationCallbacks.html

What are the benefits and drawbacks of each particular way of allocating memory?

Does LWJGL use finalize to reclaim the off heap memory after the object is garbage collected?

[spasi]

LWJGL leaves memory management decisions to the user. At the same time, it provides utilities and a flexible API that cover that basics and simplify usage. That's why there are multiple ways to allocate a struct and there are 3 different utility classes for memory management (MemoryStack, MemoryUtil, BufferUtils). You can read more details in the Memory FAQ and associated blog post.

[mudlee]

I'd like to expand the question a little bit. I read the LWJGL3 memory management, I know how to use it. My question is, what are the use cases, when should it be used instead of Java variables?

[spasi]

My question is, what are the use cases, when should it be used instead of Java variables?

There are two problems when interacting with native code from Java:

- You cannot pass a pointer to a variable in the Java thread stack to native code.
- You cannot pass a pointer to a heap-allocated Java object to native code. Java references are not pointers and jobject handles in JNI code cannot be used directly in native functions. Even if you could (there is a trick to get the raw object pointer...), JNI code is implicitly at a safepoint, meaning the GC can kick in at any time and move the object allocation (= the JVM crashes if the native function tries to access it).

The only safe and officially supported way to pass a pointer to fixed (non-moving) data to native code, is ByteBuffer.allocateDirect(). That's why LWJGL has always depended on it. So, no stack allocation and the only usable heap memory is off-Java-heap memory.

MemoryStack emulates a native thread stack. In reality it's heap-allocated memory attached to a ThreadLocal, with a simple stack API. Whenever in C you would pass a pointer to a stack-allocated variable to another function, you can use MemoryStack in Java. This is quite common actually and it's usually small data that is short-lived.

MemoryUtil is simply a replacement for BufferUtils and ByteBuffer.allocateDirect(). It sacrifices automatic GC for more control and better performance.

In any case, you should use the above as little as possible and only when interacting with native code.

[mudlee]

That was a correct, detailed explanation, thank you!

[bcbradle]

I have a few other questions too:

When a buffer is freed does it automatically free the contents of the buffer?

For instance, does freeing a VkCommandPoolCreateInfo.Buffer free the VkCommandPoolCreateInfo instances inside it?

I'm trying to put together a clojure library for lwjgl vulkan that exposes keywords instead of integer handles, vecs instead of native arrays, and maps instead of structs (efficient by aggressive caching), and which provides a view of Vulkan as if the functions had no side effects (i have a few strategies for that) and which automatically manages memory by choosing what and when to free. I can use the vulkan spec to help me understand when I should free memory, but I'm a little confused about precisely how memory is freed.

I always thought java objects were actually heap allocated, and the name is always tantamount to a pointer somewhere in the heap. I recall somewhere that native types like int and float can be stack allocated in java automatically, but subclasses of Object are always heap allocated.

What confuses me is that the contents of a buffer like VkCommandPoolCreateInfo appear to be VkCommandPoolCreateInfo instances, but really that just means its a kind of container that contains pointers to heap allocated objects, right?

Does that mean the following is correct or incorrect?

VkCommandPoolCreateInfo a = VkCommandPoolCreateInfo.malloc();
VkCommandPoolCreateInfo b = VkCommandPoolCreateInfo.calloc();
VkCommandPoolCreateInfo.Buffer foo = VkCommandPoolCreateInfo.malloc(2);
foo.put(a);
foo.put(b);
foo.flip();
//...
foo.free();
// do i need to free a and b?

[Kai]

When a buffer is freed does it automatically free the contents of the buffer?

A "buffer", such as the VkCommandPoolCreateInfo.Buffer is a Java object holding the address of an "off-heap" memory area in the process'es address space.
That off-heap memory is _not_ managed by the JVM.
Instances of VkCommandPoolCreateInfo are also just simple wrappers for holding the address of a native memory region which fits a native VkCommandPoolCreateInfo struct instance.
The reason why a VkCommandPoolCreateInfo.Buffer offers put/get methods to handle VkCommandPoolCreateInfo is because of convenience. Because the VkCommandPoolCreateInfo.Buffer knows how large a native VkCommandPoolCreateInfo struct is, its put/get methods can memcpy the native memory pointed to by the VkCommandPoolCreateInfo instances at the correct position in the VkCommandPoolCreateInfo.Buffer's memory (for put methods).

What is important to realize here is that VkCommandPoolCreateInfo instances only exist whenever you want to handle a native VkCommandPoolCreateInfo struct in a typed way in Java via a VkCommandPoolCreateInfo Java wrapper object. That wrapper object gets created on-demand and initialized with the address offset of the requested VkCommandPoolCreateInfo instance into the memory allocated by the VkCommandPoolCreateInfo.Buffer.

For instance, does freeing a VkCommandPoolCreateInfo.Buffer free the VkCommandPoolCreateInfo instances inside it?

If you manually allocated a VkCommandPoolCreateInfo instance via the malloc methods, then: no.
If you obtained a VkCommandPoolCreateInfo Java instance by using VkCommandPoolCreateInfo.Buffer.get() then: yes.
But we must separate two things here:
a) the Java VkCommandPoolCreateInfo instances which are _not_ the content or parts of the VkCommandPoolCreateInfo.Buffer but merely wrapper to hold a native memory address pointing into the buffer's memory region;
b) the native memory allocated explicitly by certain LWJGL methods, such as the various malloc() methods.
That means: The on-heap memory of the Java wrapper object managed by the JVM is different from the native memory region it should represent in LWJGL for the native structs in Vulkan or GLFW, etc.

Again, the important thing here is to ask: Who allocated the native memory first via one of the malloc() methods (i.e. who is responsible for the memory and responsible for deallocating it again)?
Generally, it is the object you created with one of the malloc() methods. And when you created a Java object via one of the malloc() methods, it means that this Java object now holds the address of a newly allocated memory area, which you must also deallocate via free().

I always thought java objects were actually heap allocated, and the name is always tantamount to a pointer somewhere in the heap. I recall somewhere that native types like int and float can be stack allocated in java automatically, but subclasses of Object are always heap allocated.

How the JVM manages Java objects is completely orthogonal to the discussion about how to handle native memory. The JVM may decide to allocate a Java object on the heap or may not allocate it at all if during Escape Analysis it found that the object cannot escape the "scope of optimization." But again, how the JVM manages memory for Java objects has nothing to do with the management of native memory, which is the topic here.

Does that mean the following is correct or incorrect?
...code snippet...

In this example, you MUST manually call free() on the VkCommandPoolCreateInfo instances 'a' and 'b', because you also manually allocated them via malloc().

[bcbradle]

So would I be right in assuming that this is correct behavior, to prevent unnecessary copying?

VkCommandPoolCreateInfo.Buffer foo = VkCommandPoolCreateInfo.malloc(2);
foo.get().set(42, 42, 42 ,42); // whatever numbers actually go in set to initialize the VkCommandPoolCreateInfo struct
foo.get().set(0, 1, 2, 3); // whatever numbers actually go in set to initialize the VkCommandPoolCreateInfo struct
foo.flip();
//...
foo.free();

[Kai]

So would I be right in assuming that this is correct behavior, to prevent unnecessary copying?

VkCommandPoolCreateInfo.Buffer foo = VkCommandPoolCreateInfo.malloc(2);
foo.get().set(42, 42, 42 ,42); // whatever numbers actually go in set to initialize the VkCommandPoolCreateInfo struct
foo.get().set(0, 1, 2, 3); // whatever numbers actually go in set to initialize the VkCommandPoolCreateInfo struct
foo.flip();
//...
foo.free();

Yes. Exactly like that.

[spasi]

Random thoughts:

Instances of struct classes and struct buffer classes are both simple wrappers over a pointer address + byte size. In the case of structs, the byte size is sizeof(<struct type>), whereas for struct buffers it's N * sizeof(<struct type>).

In both cases, there's only one Java object (if not eliminated by escape analysis). A struct buffer does not pre-allocate struct instances at each offset, there's only one Java instance and a contiguous block of off-heap memory. While iterating over a struct buffer you may allocate temporary struct instances (e.g. using get()), but those will hopefully be eliminated via EA.

You can iterate over a struct buffer using the flyweight pattern, if you don't trust EA (or profiling has shown that EA doesn't do a good job at a particular hot method):

// This:
VkCommandPoolCreateInfo.Buffer foo = VkCommandPoolCreateInfo.malloc(2);
foo.get(1).queueFamilyIndex(0);

// is equivalent to this:
VkCommandPoolCreateInfo.Buffer foo = VkCommandPoolCreateInfo.malloc(2);
foo.position(1);
foo.queueFamilyIndex(0);

When you put() a struct on a struct buffer, you are not copying a pointer address; you're doing a memcpy of the struct's data to the buffer at the specified offset. Changes to the struct will NOT be reflected to the struct buffer, and vice versa.

When you get() from a struct buffer, you're creating a new struct wrapper, pointing at the memory address at the specified offset. Changes to the new wrapper will BE reflected in the original struct buffer and vice versa. The struct instance in this case should not be freed of course, because the original struct buffer owns its memory.

You must carefully track every memory allocation in your program. This is tedious and error-prone and Java's type system does not help with making it easier. But, you need to keep in mind that it's no different in C (and LWJGL tries to "emulate" writing C in Java). Even in C, when you have a random pointer to a struct, you don't know if it should be freed or not without additional information about what the program is doing. You don't know if it's allocated by you, by another library, or even if it's allocated on the heap or the stack. It's the same thing in LWJGL. You must think in terms of memory allocations; who allocated it and owns them memory, by what means (malloc vs ByteBuffer.allocateDirect vs MemoryStack), who else has live pointers to that memory. The nice thing in Java* is that it's easier to hide the complexity in friendly abstractions and to debug problems when they occur (imho).

* replace Java with your favorite JVM language.

[Cornix]

You must carefully track every memory allocation in your program. This is tedious and error-prone and Java's type system does not help with making it easier. But, you need to keep in mind that it's no different in C (and LWJGL tries to "emulate" writing C in Java). Even in C, when you have a random pointer to a struct, you don't know if it should be freed or not without additional information about what the program is doing. You don't know if it's allocated by you, by another library, or even if it's allocated on the heap or the stack. It's the same thing in LWJGL. You must think in terms of memory allocations; who allocated it and owns them memory, by what means (malloc vs ByteBuffer.allocateDirect vs MemoryStack), who else has live pointers to that memory. The nice thing in Java* is that it's easier to hide the complexity in friendly abstractions and to debug problems when they occur (imho).

Would it be possible to do something with SoftReference and finalize()-methods to somehow make memory leaks less likely? A kind of safety net?

[spasi]

Would it be possible to do something with SoftReference and finalize()-methods to somehow make memory leaks less likely? A kind of safety net?

The safety net already exists; use memory allocated via ByteBuffer.allocateDirect. Deallocation is already handled automatically with a reference queue. Though, you sacrifice EA (the buffer reference escapes by definition) and can't be sure when or even if the buffer has been freed (2 GC cycles minimum for it to happen).

I follow these rules and never had serious issues:

- Use native memory as little as possible.
- When you do need it, use the MemoryStack for small, short-lived allocations.
- If the MemoryStack is not applicable, use explicit malloc/free if cleanup is straightforward.
- If not, use BufferUtils (i.e. ByteBuffer.allocateDirect).