Using OpenVR's action system (getting a handle)

[Richtea]

I'm trying to upgrade JMonkey's* VR implementation to stop using the deprecated LWJGL OpenVR VRSystem.VRSystem_GetControllerState and start using the newer action system.

I have been tracking some non java documentation and trying to translate it.

After the VR environment is initialised I'm calling

           

VRInput.VRInput_SetActionManifestPath("C:/Users/richa/Documents/Development/jmonkeyVrTest/src/main/resources/actionManifest.json"); //hard coded for experimental purposes

And it seems happy with that. SteamVr is happy if there is a default binding for my device and unhappy if not (which feels like a good sign).

I then tried to get a handle to an action but I'm slightly unsure what the pHandle that is passed to VRInput#VRInput_GetActionHandle is for (as the javadoc says it returns a handle and an int is returned, I'm assuming the handle gets put into the longBuffer for now). Regardless I tried to make this call and created a longBuffer for this purpose

LongBuffer longBuffer = LongBuffer.allocate(100); //100 chosen arbitrarily
int handle = VRInput.VRInput_GetActionHandle("/actions/main/in/OpenInventory", longBuffer);

However this ends up causing an EXCEPTION_ACCESS_VIOLATION

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffb7b22788a, pid=6804, tid=24960
#
# JRE version: OpenJDK Runtime Environment Corretto-11.0.11.9.1 (11.0.11+9) (build 11.0.11+9-LTS)
# Java VM: OpenJDK 64-Bit Server VM Corretto-11.0.11.9.1 (11.0.11+9-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [vrclient_x64.dll+0x1c788a]
#

What am I doing wrong here. How should I go about getting the action handle?

*As it may become clear during this discussion that I don't 100% know what I'm doing I feel the need to make it clear I'm not an official maintainer for JMonkey.

[spasi]

Hey Richtea,

The function returns an integer error code and accepts two parameters: 1) the action name 2) a buffer to which the action handle will be written. That means, if the call is successful, you can get the handle value with longBuffer.get(0).

The reason it's crashing is simple: The buffer you've allocated is a NIO buffer backed by a long[] array. You need to allocate an off-heap buffer using ByteBuffer.allocateDirect, or more easily, using LWJGL's BufferUtils or MemoryStack classes. See Memory management in LWJGL for more information.

[Richtea]

Thanks Spasi, that's very helpful.

LongBuffer longBuffer = BufferUtils.createLongBuffer(1);
int handleErrorCode = VRInput.VRInput_GetActionHandle("/actions/main/in/OpenInventory", longBuffer);
openInventoryHandle = longBuffer.get(0);

I think I'll need the action set handle as well

VRInput.VRInput_GetActionSetHandle("/actions/main", longBuffer);
actionSetHandle = longBuffer.get(0);

Based on that approach I can see that the call to VRInput_UpdateActionState will similarly need a byteBuffer for VRActiveActionSet and VRInput_GetDigitalActionData will need one for the inputDigitalActionData

vrActiveActionSetByteBuffer = BufferUtils.createByteBuffer(VRActiveActionSet.SIZEOF);
inputDigitalActionDataByteBuffer = BufferUtils.createByteBuffer(InputDigitalActionData.SIZEOF);

(I've done that during application initialisation and reused them each frame, although the talk of escape analysis in the tutorial you post suggests maybe I could reallocate every frame without leaking.)

During each frame I've done the following

VRActiveActionSet.Buffer vrBuffer = new VRActiveActionSet.Buffer(vrActiveActionSetByteBuffer);
vrBuffer.ulActionSet(actionSetHandle);
vrBuffer.ulRestrictedToDevice(VR.k_ulInvalidInputValueHandle); // both hands
VRInput.VRInput_UpdateActionState(vrBuffer, VRActiveActionSet.SIZEOF);

InputDigitalActionData inputDigitalActionData = new InputDigitalActionData(inputDigitalActionDataByteBuffer);

VRInput.VRInput_GetDigitalActionData(openInventoryHandle, inputDigitalActionData, VR.k_ulInvalidInputValueHandle);
if (inputDigitalActionData.bState()){
       System.out.println("active");
}

I understand that restrictToDevice is to do with left hand vs right hand. And I believe VR.k_ulInvalidInputValueHandle means either hand.

This sadly seems not to work; in the sense that "active" is never printed because inputDigitalActionData.bState() is never true. (inputDigitalActionData.bActive() is always true though). I have in my manifest the action of type "/actions/main/in/OpenInventory" which for my device (Oculus Quest 2 but I believe OpenVr thinks its OculusTouch) that is bound to "/user/hand/left/input/x"

[Richtea]

Significant progress. I've now got it so that it works, but only based on manually configured bindings.

I now have it as follows.

During start up

            VRInput.VRInput_SetActionManifestPath("C:/Users/richa/Documents/Development/android/jmonkeyVrTest/src/main/resources/actionManifest.json"); //hard coded for experimental purposes

            //LongBuffer longBuffer = ByteBuffer.allocate( (Long.SIZE / 8) * 100 ).order(java.nio.ByteOrder.nativeOrder()).asLongBuffer();
            LongBuffer longBuffer = BufferUtils.createLongBuffer(1);
            int error1 = VRInput.VRInput_GetActionHandle("/actions/main/in/OpenInventory", longBuffer);
            openInventoryHandle = longBuffer.get(0);

            int error2 = VRInput.VRInput_GetActionSetHandle("/actions/main", longBuffer);
            actionSetHandle = longBuffer.get(0);

            System.out.println("Handle: " + openInventoryHandle);

            activeActionSets = VRActiveActionSet.create(1);
            activeActionSets.ulActionSet(actionSetHandle);
            activeActionSets.ulRestrictedToDevice(VR.k_ulInvalidInputValueHandle); // both hands

            clickTriggerActionData = InputDigitalActionData.create();

Each loop

        int error4 = VRInput.VRInput_GetDigitalActionData(openInventoryHandle, clickTriggerActionData, VR.k_ulInvalidInputValueHandle);
        if (clickTriggerActionData.bState()){
            System.out.println("active");
        }
        if (clickTriggerActionData.bChanged()){
            System.out.println("changed");
        }

It might just be that I set a custom bindings at some point and need to clear it in steamVr, or it might be that my default bindings are in fact wrong

[Richtea]

It was indeed just my that default bindings were wrong. For the purposes for future googlers, Oculus headsets (including the Oculus quest) call their controller_type "oculus_touch", not as I had got it into my head "oculusTouch".

Further for future googlers my action manifest looked like:

{
  "default_bindings": [
    {
      "controller_type": "oculus_touch",
      "binding_url": "oculusTouchDefaults.json"
    }
  ],
  "actions": [
    {
      "name": "/actions/main/in/OpenInventory",
      "requirement": "mandatory",
      "type": "boolean"
    },
    {
      "name": "/actions/main/in/test2",
      "requirement": "mandatory",
      "type": "boolean"
    },
    {
      "name": "/actions/main/in/scroll",
      "type": "vector2",
      "requirement": "mandatory"
    }
  ],
  "action_sets": [
    {
      "name": "/actions/main",
      "usage": "leftright"
    }
  ],
  "localization" : [
    {
      "language_tag": "en_us",
      "/actions/main" : "My Game Actions",
      "/actions/main/in/OpenInventory" : "Open Inventory"

    }
  ]
}

And my oculus default bindings looked like

{
  "action_manifest_version" : 0,
  "bindings": {
    "/actions/main": {
      "sources" : [
        {
          "inputs" : {
            "click" : {
              "output" : "/actions/main/in/OpenInventory"
            }
          },
          "mode" : "button",
          "path" : "/user/hand/left/input/x"
        },
        {
          "inputs" : {
            "click" : {
              "output" : "/actions/main/in/test2"
            }
          },
          "mode" : "button",
          "path" : "/user/hand/left/input/y"
        },
        {
          "inputs" : {
            "position" : {
              "output" : "/actions/main/in/scroll"
            }
          },
          "mode" : "joystick",
          "path" : "/user/hand/left/input/joystick"
        }
      ]
    }
  },
  "category" : "steamvr_input",
  "controller_type" : "oculus_touch",
  "description" : "Bindings for the OpenVR SDK \"hellovr_opengl\" demo for a oculusTouch controller",
  "name" : "HelloVR bindings for a oculusTouch controller",
  "options" : {},
  "simulated_actions" : []
}

[spasi]

Hey Richtea, thanks for the update.

Just wanted to let you know that the next snapshot (3.3.1 build 3) will include OpenXR bindings. I'm hoping that we won't have to maintain the OVR and OpenVR bindings for much longer.

[Richtea]

Hi Spasi,

Yes I saw that (which is excellent news!). I suspect hot on the heals of upgrading from OpenVR legacy to OpenVR action-based I'll need to upgrade to OpenXR. It feels like a useful learning exercise though and will give JMonkey a VR system that is 1 generation behind rather than 2 generations behind