LibFFI struct creation

[Nostritius]

Hello everyone,
I'm trying to develop a small wrapper class for the spacenavd driver to access my 3d Mouse but I have come over a problem where I need your help. I'm using the libFFI binding for this. I know, that there is also a java sdk, but it hasn't all the functions I need. My Problem is, that one of the main methods look like this

int spnav_wait_event(spnav_event *event);

So I need a pointer to the spnav_event struct which looks like this:

struct spnav_event_motion {
	int type;
	int x, y, z;
	int rx, ry, rz;
	unsigned int period;
	int *data;
};

struct spnav_event_button {
	int type;
	int press;
	int bnum;
};

typedef union spnav_event {
	int type;
	struct spnav_event_motion motion;
	struct spnav_event_button button;
} spnav_event;

So I need to create this struct in Memory, but I haven't found a method to do this. My Question is, is there a function to create this struct and later get the values from it, and if not could it be added?

[Kai]

Hello,

well, what you can do without altering LWJGL is to allocate a direct ByteBuffer via BufferUtils.createByteBuffer(int) and hand the address of this buffer to your spnav_wait_event function.
You can obtain the address of the underlying native buffer store via MemoryUtil.memAddress(ByteBuffer).

In order to read back the values filled by the function, you know the layout of the struct, and you can then read back the values of individual members of the structs via ByteBuffer.getInt(int) for all int and uint and ByteBuffer.getLong(int) on a 64 bit architecture for the "data" pointer.

This would be a fast and hacky solution. Otherwise, you would have to add a new Kotlin template with a struct definition to LWJGL's sources, which you can clone on github.

[spasi]

Just be careful with the struct layout, it can be tricky to get right.

Even though libFFI supports defining structs at runtime and can calculate their layout, it doesn't expose any API for the user to retrieve the sizeof(struct) or field offsets. Both C/Invoke and dyncall do, but C/Invoke hasn't been updated since 2007 and dyncall has inefficient function calling (for Java at least). It might be useful for cases like this to provide functionality in LWJGL, written in Java, that automatically calculates struct layouts.

ByteBuffer.getLong(int) on a 64 bit architecture for the "data" pointer.

This isn't necessary. You should use PointerBuffer.get(ByteBuffer buffer, int index), which will do the correct read on both x86 and x64 architectures. You should also handle the spnav_event_motion struct size similarly, you can use Pointer.POINTER_SIZE for that.

[Nostritius]

Ok, I've tried to create some FFITypes for the three different events

spnav_motion_event = new FFIType();
		spnav_motion_event.setType(LibFFI.FFI_TYPE_STRUCT);
		PointerBuffer elementsBuffer = PointerBuffer.allocateDirect(9);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_uint);
		elementsBuffer.put(LibFFI.ffi_type_pointer);
		spnav_motion_event.setElements(elementsBuffer.getBuffer());
		spnav_motion_event.setAlignment(Pointer.POINTER_SIZE);
		spnav_motion_event.setSize(7*Pointer.POINTER_SIZE);
		
		spnav_button_event = new FFIType();
		spnav_button_event.setType(LibFFI.FFI_TYPE_STRUCT);
		elementsBuffer = PointerBuffer.allocateDirect(3);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		spnav_button_event.setElements(elementsBuffer.getBuffer());
		spnav_motion_event.setAlignment(4);
		spnav_motion_event.setSize(3*4);
		
		spnav_event = new FFIType();
		spnav_button_event.setType(LibFFI.FFI_TYPE_STRUCT);
		elementsBuffer = PointerBuffer.allocateDirect(3);
		elementsBuffer.put(LibFFI.ffi_type_sint);
		elementsBuffer.put(spnav_button_event.getPointer());
		elementsBuffer.put(spnav_button_event.getPointer());
		spnav_event.setElements(elementsBuffer.getBuffer());
		spnav_event.setAlignment((int) spnav_motion_event.getSize());
		spnav_event.setSize(spnav_motion_event.getSize()*3);

And call the method with this

public int spnav_wait_event(SpnavEvent e)
	{
		ByteBuffer args, ret;
		
		ByteBuffer event = spnav_event.buffer();
		
		
		args = (ByteBuffer) ByteBuffer.allocateDirect(Pointer.POINTER_SIZE);
		if(Pointer.BITS32)args.putInt((int) MemoryUtil.memAddress(event));
		else if(Pointer.BITS64) args.putLong(MemoryUtil.memAddress(event));
		ret = (ByteBuffer) ByteBuffer.allocateDirect(4).rewind();
		
		call("spnav_wait_event", spnavWaitEvents, args, ret);
				
		return ret.getInt();
	}

and this

private DynamicLinkLibrary lib;

//...

protected void call(String functionname, FFICIF cif, ByteBuffer args, ByteBuffer ret)
	{
		//get the pointer to the function
		long fpointer = lib.getFunctionAddress(functionname);
		
		//check if the function is valid
		if(fpointer == 0L)
		{
			System.out.println(functionname + " is no valid function in " + lib.getName());
			return;
		}
		
		//call the function
		LibFFI.ffi_call(cif.buffer(), fpointer, ret, args);
	}

But now it works calling exactly 26 times (sometimes 36) calling it in a loop and then crashes with a segment violation. It is defined as blocking polling function, that means, only if an event occurs it will be written to the spnav_event object. Am I doing alright?

[spasi]

Not sure why it (even) works a few times, but your code has obvious issues:

a) You don't have to describe the structs to libFFI. You'd only need to do that if you had to pass or return the spnav structs by value.

b) You're passing the address of spnav_event.buffer() to spnav_wait_event. This is the ffi_type struct that *describes* the spnav_event struct, not the spnav_event struct itself. You need to properly allocate memory for the spnav_event struct and use that instead.

c) As I said above, don't use conditional putInt/putLong for passing the memAddress to args. Use PointerBuffer.put(buffer, index) instead.

d) Never use allocateDirect directly, always use the BufferUtils create methods to allocate buffers. In the code above, you have forgotten to set the native ByteOrder on the allocated buffers.

[Nostritius]

I've corrected the methods now and changed the function spnav_wait_event to this

public int spnav_wait_event(SpnavEvent e)
{
	ByteBuffer args, ret;
	
	ByteBuffer event = BufferUtils.createByteBuffer(300);
	
	args = BufferUtils.createPointerBuffer(1).put(event).getBuffer();
	ret = (ByteBuffer) BufferUtils.createByteBuffer(4);
	call("spnav_wait_event", spnavWaitEvents, args, ret);
			
	return ret.getInt();
}

and deleted all the Type definitions in the constructor.

public Libspnav()
	{
		super("spnav");
		
		spnavOpen = new FFICIF();
		spnavClose = new FFICIF();
		spnavFd = new FFICIF();
		spnavSensitivity = new FFICIF();
		spnavWaitEvents = new FFICIF();
		
		PointerBuffer voidbuffer = BufferUtils.createPointerBuffer(0);
		PointerBuffer sensitivitybuffer = BufferUtils.createPointerBuffer(1).put(LibFFI.ffi_type_double);
		PointerBuffer eventbuffer = BufferUtils.createPointerBuffer(1).put(LibFFI.ffi_type_pointer);
		
		//create the bindings
		LibFFI.ffi_prep_cif(spnavOpen.buffer(), LibFFI.FFI_DEFAULT_ABI, LibFFI.ffi_type_sint, voidbuffer);
		LibFFI.ffi_prep_cif(spnavClose.buffer(), LibFFI.FFI_DEFAULT_ABI, LibFFI.ffi_type_sint, voidbuffer);
		LibFFI.ffi_prep_cif(spnavFd.buffer(), LibFFI.FFI_DEFAULT_ABI, LibFFI.ffi_type_sint, voidbuffer);
		LibFFI.ffi_prep_cif(spnavSensitivity.buffer(), LibFFI.FFI_DEFAULT_ABI, LibFFI.ffi_type_sint, sensitivitybuffer);
		LibFFI.ffi_prep_cif(spnavWaitEvents.buffer(), LibFFI.FFI_DEFAULT_ABI, LibFFI.ffi_type_sint, eventbuffer);
	}

Because I haven't the knowledge to calculate the exact layout of the struct I've set it to a big value. I tried this configuration but it doesn't work either. It gives me this error:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x00007f04010139fb, pid=8129, tid=139655568623360
#
# JRE version: OpenJDK Runtime Environment (8.0_40-b20) (build 1.8.0_40-b20)
# Java VM: OpenJDK 64-Bit Server VM (25.40-b23 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# j  sun.misc.Unsafe.allocateMemory(J)J+0
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/patrick/git/pentaquin/test/hs_err_pid8129.log
Compiled method (c1)    1899  113       3       java.nio.Bits::byteOrder (20 bytes)
 total in heap  [0x00007f040113f010,0x00007f040113f318] = 776
 relocation     [0x00007f040113f138,0x00007f040113f168] = 48
 main code      [0x00007f040113f180,0x00007f040113f240] = 192
 stub code      [0x00007f040113f240,0x00007f040113f2d0] = 144
 metadata       [0x00007f040113f2d0,0x00007f040113f2d8] = 8
 scopes data    [0x00007f040113f2d8,0x00007f040113f2e0] = 8
 scopes pcs     [0x00007f040113f2e0,0x00007f040113f310] = 48
 dependencies   [0x00007f040113f310,0x00007f040113f318] = 8
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

I've tried also setting the native byte order, but the error message was the same. And still it works a few times before this message appears.

[spasi]

The one bug I see is that you're not flipping the sensitivitybuffer and eventbuffer. PointerBuffer works exactly like standard NIO buffers.

Also, a couple of optimizations you could make:

- The empty voidbuffer isn't necessary, simply pass null to ffi_prep_cif.
- Cache the function addresses instead of looking them up on every call, i.e. make the call method accept the function pointer directly, instead of the function name.

If you're still having trouble, please post the full code and attach the crash log.

[Nostritius]

I've tried flipping but still it hungs up with a segemnt violation. Also I've tried the same code in c++ and it works. Here is the whole code of the libspnav class, the abstract Library class and the error log.

[spasi]

It might be a GC issue. Does it work if you create the event, args and ret buffers as fields in Libspnav (not as local variables in spnav_wait_event)?

Also, don't flip() the event buffer after creation.