Hello Guest

Using OpenVR's action system (getting a handle)

  • 6 Replies
  • 2731 Views
Using OpenVR's action system (getting a handle)
« on: January 05, 2022, 14:55:32 »
I'm trying to upgrade JMonkey's* VR implementation to stop using the deprecated LWJGL OpenVR VRSystem.VRSystem_GetControllerState and start using the newer action system.

I have been tracking some non java documentation and trying to translate it.

After the VR environment is initialised I'm calling

           
Code: [Select]
VRInput.VRInput_SetActionManifestPath("C:/Users/richa/Documents/Development/jmonkeyVrTest/src/main/resources/actionManifest.json"); //hard coded for experimental purposes
And it seems happy with that. SteamVr is happy if there is a default binding for my device and unhappy if not (which feels like a good sign).

I then tried to get a handle to an action but I'm slightly unsure what the pHandle that is passed to VRInput#VRInput_GetActionHandle is for (as the javadoc says it returns a handle and an int is returned, I'm assuming the handle gets put into the longBuffer for now). Regardless I tried to make this call and created a longBuffer for this purpose

Code: [Select]
LongBuffer longBuffer = LongBuffer.allocate(100); //100 chosen arbitrarily
int handle = VRInput.VRInput_GetActionHandle("/actions/main/in/OpenInventory", longBuffer);

However this ends up causing an EXCEPTION_ACCESS_VIOLATION

Code: [Select]
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffb7b22788a, pid=6804, tid=24960
#
# JRE version: OpenJDK Runtime Environment Corretto-11.0.11.9.1 (11.0.11+9) (build 11.0.11+9-LTS)
# Java VM: OpenJDK 64-Bit Server VM Corretto-11.0.11.9.1 (11.0.11+9-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [vrclient_x64.dll+0x1c788a]
#

What am I doing wrong here. How should I go about getting the action handle?

*As it may become clear during this discussion that I don't 100% know what I'm doing I feel the need to make it clear I'm not an official maintainer for JMonkey.
« Last Edit: January 05, 2022, 18:52:42 by Richtea »

*

Online spasi

  • *****
  • 2261
    • WebHotelier
Re: Using OpenVR's action system (getting a handle)
« Reply #1 on: January 05, 2022, 16:11:29 »
Hey Richtea,

The function returns an integer error code and accepts two parameters: 1) the action name 2) a buffer to which the action handle will be written. That means, if the call is successful, you can get the handle value with longBuffer.get(0).

The reason it's crashing is simple: The buffer you've allocated is a NIO buffer backed by a long[] array. You need to allocate an off-heap buffer using ByteBuffer.allocateDirect, or more easily, using LWJGL's BufferUtils or MemoryStack classes. See Memory management in LWJGL for more information.

Re: Using OpenVR's action system (getting a handle)
« Reply #2 on: January 05, 2022, 18:49:28 »
Thanks Spasi, that's very helpful.

Code: [Select]
LongBuffer longBuffer = BufferUtils.createLongBuffer(1);
int handleErrorCode = VRInput.VRInput_GetActionHandle("/actions/main/in/OpenInventory", longBuffer);
openInventoryHandle = longBuffer.get(0);

I think I'll need the action set handle as well

Code: [Select]
VRInput.VRInput_GetActionSetHandle("/actions/main", longBuffer);
actionSetHandle = longBuffer.get(0);


Based on that approach I can see that the call to VRInput_UpdateActionState will similarly need a byteBuffer for VRActiveActionSet and VRInput_GetDigitalActionData will need one for the inputDigitalActionData

Code: [Select]
vrActiveActionSetByteBuffer = BufferUtils.createByteBuffer(VRActiveActionSet.SIZEOF);
inputDigitalActionDataByteBuffer = BufferUtils.createByteBuffer(InputDigitalActionData.SIZEOF);

(I've done that during application initialisation and reused them each frame, although the talk of escape analysis in the tutorial you post suggests maybe I could reallocate every frame without leaking.)

During each frame I've done the following

Code: [Select]
VRActiveActionSet.Buffer vrBuffer = new VRActiveActionSet.Buffer(vrActiveActionSetByteBuffer);
vrBuffer.ulActionSet(actionSetHandle);
vrBuffer.ulRestrictedToDevice(VR.k_ulInvalidInputValueHandle); // both hands
VRInput.VRInput_UpdateActionState(vrBuffer, VRActiveActionSet.SIZEOF);

InputDigitalActionData inputDigitalActionData = new InputDigitalActionData(inputDigitalActionDataByteBuffer);

VRInput.VRInput_GetDigitalActionData(openInventoryHandle, inputDigitalActionData, VR.k_ulInvalidInputValueHandle);
if (inputDigitalActionData.bState()){
       System.out.println("active");
}

I understand that restrictToDevice is to do with left hand vs right hand. And I believe VR.k_ulInvalidInputValueHandle means either hand.

This sadly seems not to work; in the sense that "active" is never printed because inputDigitalActionData.bState() is never true. (inputDigitalActionData.bActive() is always true though). I have in my manifest the action of type "/actions/main/in/OpenInventory" which for my device (Oculus Quest 2 but I believe OpenVr thinks its OculusTouch) that is bound to "/user/hand/left/input/x"
« Last Edit: January 05, 2022, 18:51:56 by Richtea »

Re: Using OpenVR's action system (getting a handle)
« Reply #3 on: January 06, 2022, 14:30:28 »
Significant progress. I've now got it so that it works, but only based on manually configured bindings.

I now have it as follows.

During start up

Code: [Select]
            VRInput.VRInput_SetActionManifestPath("C:/Users/richa/Documents/Development/android/jmonkeyVrTest/src/main/resources/actionManifest.json"); //hard coded for experimental purposes

            //LongBuffer longBuffer = ByteBuffer.allocate( (Long.SIZE / 8) * 100 ).order(java.nio.ByteOrder.nativeOrder()).asLongBuffer();
            LongBuffer longBuffer = BufferUtils.createLongBuffer(1);
            int error1 = VRInput.VRInput_GetActionHandle("/actions/main/in/OpenInventory", longBuffer);
            openInventoryHandle = longBuffer.get(0);

            int error2 = VRInput.VRInput_GetActionSetHandle("/actions/main", longBuffer);
            actionSetHandle = longBuffer.get(0);

            System.out.println("Handle: " + openInventoryHandle);

            activeActionSets = VRActiveActionSet.create(1);
            activeActionSets.ulActionSet(actionSetHandle);
            activeActionSets.ulRestrictedToDevice(VR.k_ulInvalidInputValueHandle); // both hands

            clickTriggerActionData = InputDigitalActionData.create();

Each loop

Code: [Select]
        int error4 = VRInput.VRInput_GetDigitalActionData(openInventoryHandle, clickTriggerActionData, VR.k_ulInvalidInputValueHandle);
        if (clickTriggerActionData.bState()){
            System.out.println("active");
        }
        if (clickTriggerActionData.bChanged()){
            System.out.println("changed");
        }

It might just be that I set a custom bindings at some point and need to clear it in steamVr, or it might be that my default bindings are in fact wrong

Re: Using OpenVR's action system (getting a handle)
« Reply #4 on: January 07, 2022, 09:35:42 »
It was indeed just my that default bindings were wrong. For the purposes for future googlers, Oculus headsets (including the Oculus quest) call their controller_type "oculus_touch", not as I had got it into my head "oculusTouch".

Further for future googlers my action manifest looked like:

Code: [Select]
{
  "default_bindings": [
    {
      "controller_type": "oculus_touch",
      "binding_url": "oculusTouchDefaults.json"
    }
  ],
  "actions": [
    {
      "name": "/actions/main/in/OpenInventory",
      "requirement": "mandatory",
      "type": "boolean"
    },
    {
      "name": "/actions/main/in/test2",
      "requirement": "mandatory",
      "type": "boolean"
    },
    {
      "name": "/actions/main/in/scroll",
      "type": "vector2",
      "requirement": "mandatory"
    }
  ],
  "action_sets": [
    {
      "name": "/actions/main",
      "usage": "leftright"
    }
  ],
  "localization" : [
    {
      "language_tag": "en_us",
      "/actions/main" : "My Game Actions",
      "/actions/main/in/OpenInventory" : "Open Inventory"

    }
  ]
}

And my oculus default bindings looked like

Code: [Select]
{
  "action_manifest_version" : 0,
  "bindings": {
    "/actions/main": {
      "sources" : [
        {
          "inputs" : {
            "click" : {
              "output" : "/actions/main/in/OpenInventory"
            }
          },
          "mode" : "button",
          "path" : "/user/hand/left/input/x"
        },
        {
          "inputs" : {
            "click" : {
              "output" : "/actions/main/in/test2"
            }
          },
          "mode" : "button",
          "path" : "/user/hand/left/input/y"
        },
        {
          "inputs" : {
            "position" : {
              "output" : "/actions/main/in/scroll"
            }
          },
          "mode" : "joystick",
          "path" : "/user/hand/left/input/joystick"
        }
      ]
    }
  },
  "category" : "steamvr_input",
  "controller_type" : "oculus_touch",
  "description" : "Bindings for the OpenVR SDK \"hellovr_opengl\" demo for a oculusTouch controller",
  "name" : "HelloVR bindings for a oculusTouch controller",
  "options" : {},
  "simulated_actions" : []
}

*

Online spasi

  • *****
  • 2261
    • WebHotelier
Re: Using OpenVR's action system (getting a handle)
« Reply #5 on: January 07, 2022, 12:45:41 »
Hey Richtea, thanks for the update.

Just wanted to let you know that the next snapshot (3.3.1 build 3) will include OpenXR bindings. I'm hoping that we won't have to maintain the OVR and OpenVR bindings for much longer.

Re: Using OpenVR's action system (getting a handle)
« Reply #6 on: January 07, 2022, 13:46:26 »
Hi Spasi,

Yes I saw that (which is excellent news!). I suspect hot on the heals of upgrading from OpenVR legacy to OpenVR action-based I'll need to upgrade to OpenXR. It feels like a useful learning exercise though and will give JMonkey a VR system that is 1 generation behind rather than 2 generations behind