LWJGL 3 with Oculus Rift

[BrickFarmer]

thanks for the tips   Getting close I think!

I had a huge confusion with the OVRGLTexture, since creating the OVRSwapTextureSet(1 per eye) provides the textures already!  In fact it creates 2 textures per eye. 

Anyway I think I managed to work that out, and I get 4 textures numbered 1,2,3,4 which seems sensible.  But I'm still getting a native crash EXCEPTION_ACCESS_VIOLATION on the SubmitFrame call.  I've traced it back to the setColorTexture, ie. if I comment that line(463) out then I dont get the crash, but obviously also no drawing

If you get time, please could you inspect my textureSet creation code:

https://github.com/WhiteHexagon/example-lwjgl3-rift/blob/master/src/main/java/com/sunshineapps/riftexample/RiftClient0600.java#L424

[edit] I've also tried using a single texture instead of texture per eye, but same sort of crash

[spasi]

This was a bug in LWJGL's generator, which I just fixed. ovrLayerEyeFov.ColorTexture is an array of pointers to ovrSwapTextureSet structs, but the generator was treating it as a nested struct array. Do a pull and rebuild LWJGL, it should work fine.

[BrickFarmer]

Confirmed that fixes the problem   Now my code loops.  Next job for me is to get something to display, and the headset doesnt seem to activate.  But that's my problems.  I'll let you know if I make further progress, and thanks for the quick fix!  Peter.

[edit]

I was just looking to add the mirror buffers to get something to display at least in the desktop window.  It looks like ovrHmd_CreateMirrorTextureGL is a candidate for a PointerBuffer? even if this is a single Texture being generated (not a set). ie. ovrTexture** outMirrorTexture

I did take a look at the last fix you did, but I cant figure out how that generation works, or if the above is valid, sorry!

PS I asked over on the oculus forum to see the rift would run without GLFW.  Since I think the rift manages it's own window.  But then I started thinking how would the glContext work in that case, since it seems to be attached to a window.  Not important, I was just interested.

[spasi]

I was just looking to add the mirror buffers to get something to display at least in the desktop window.  It looks like ovrHmd_CreateMirrorTextureGL is a candidate for a PointerBuffer? even if this is a single Texture being generated (not a set). ie. ovrTexture** outMirrorTexture

There is already a PointerBuffer version of ovrHmd_CreateMirrorTextureGL, just like ovrHmd_CreateSwapTextureSetGL.

PS I asked over on the oculus forum to see the rift would run without GLFW.  Since I think the rift manages it's own window.  But then I started thinking how would the glContext work in that case, since it seems to be attached to a window.  Not important, I was just interested.

LWJGL doesn't care who manages the OpenGL context. GLContext.createFromCurrent() will work, as long as there is a current OpenGL context in the current thread. But I'm pretty sure LibOVR does not create a context of its own, it uses the context provided by the application.

[BrickFarmer]

There is already a PointerBuffer version of ovrHmd_CreateMirrorTextureGL, just like ovrHmd_CreateSwapTextureSetGL.

Ah yes, it was a long day   I got the mirror mode working, and it turns out the blank display on the Rift just needed a PC reboot.

Anyway I have something that at least now shows a clearColor!  Still that's quite exciting for me after 3 days

It can be used without a Rift, although obviously no tracking.

https://github.com/WhiteHexagon/example-lwjgl3-rift

My scene doesnt yet render, no idea why at this point, but that's todays challenge.

[BrickFarmer]

So I've tracked my issue down to my use of ovrMatrix4f_Projection.  Please can you confirm the __result type that this is expecting?  I'm passing a OVRMatrix4f buffer but not sure that is correct?

[spasi]

Yes, it is correct. The native function returns an ovrMatrix4f struct by value. Since this can't be translated to Java, LWJGL creates the virtual __result parameter, that must be the address to an ovrMatrix4f struct. The JNI code looks like this:

ovrFovPort *fov = (ovrFovPort *)(intptr_t)fovAddress;
*((ovrMatrix4f*)(intptr_t)__result) = ovrMatrix4f_Projection(*fov, znear, zfar, projectionModFlags);

What exactly is the problem you're having? Does the projection struct stay empty after a call to ovrMatrix4f_Projection?

[BrickFarmer]

Thanks, I just wanted to be sure because the parameter looked a bit different from other methods.  This is what I'm calling. 

OVRUtil.ovrMatrix4f_Projection(fovPorts[eye].buffer(), 0.5f, 500f, OVRUtil.ovrProjection_RightHanded, projections[eye].buffer());

gives

0.92978895, 0.0, 0.01567176, 0.0, 
0.0, 0.7509745, 0.0, 0.0, 
0.0, 0.0, -1.001001, -0.5005005, 
0.0, 0.0, -1.0, 0.0,

(actually it's the same matrix for both eyes, even if my FOVs are different between left and right)

This is how I load it into GL_PROJECTION.

glLoadMatrixf(projections[eye].buffer());

What I noticed today, is if I change line to

glFrustum(-1.0f * canvasRatio, canvasRatio, -1.0f, 1.0f, .5f, 500.0f);

I get a very similar matrix being put into the GL_PROJECTION

0.28125, 0.0, 0.0, 0.0, 
0.0, 0.5, 0.0, 0.0, 
0.0, 0.0, -1.002002, -1.0, 
0.0, 0.0, -1.001001, 0.0,

but my scene actually renders.  So I'm missing something in my understanding.  Is glFrustrum doing something other than setting the projection matrix, or is it my loading of the matrix thats incorrect?  From the SDK it sounds like it might be a transpose matrix, but when they seem to be so similar I'm not so sure.  To be honest this is a bit out of my comfort zone...

[spasi]

Hmm, not sure.

McGuire's sample uses ovrProjection_RightHanded and glLoadTransposeMatrixf. Have you tried ovrProjection_ClipRangeOpenGL?

[BrickFarmer]

OK thanks, I will experiment some more with that, or just stick with the glFrustum call for now   Eventually I will move this example over to more modern OpenGL, but for now I just try to use what I previously learned a long time ago...

I just tried to move all the GLFW stuff into an optional class, ie since the mirroring is optional.  However I realised I have no way to create an OpenGL context by doing this.  I assume the Oculus SDK has it's own context which does not seem to be exposed, and probably best that way!   
ie. the only method I can find in LWJGL for creating a context is by referenceing a window.  I supose I could create a 1x1 borderless window to get around this, but just wanted to check that I didnt miss anything? and if that is maybe the best way, at least then I could still get some keyboard events too

[Kai]

Hi,
your experimentation with the Rift seems quite cool.
If you manage to get the projection settings working, which looks to be quite complicated, involving some number of parameters, we could somehow wrap that in a nice method and add that to a coming JOML-LWJGL integration library, so that one would not have to fiddle around with all that native structs and buffer stuff, which looks like it can be hidden away quite effectively, especially because... well... the Rift is just the Rift with always? the same frustum settings?
I do not own a Rift myself, but I figure the projection matrix creation can be wrapped quite nicely.

[spasi]

ie. the only method I can find in LWJGL for creating a context is by referenceing a window.  I supose I could create a 1x1 borderless window to get around this, but just wanted to check that I didnt miss anything? and if that is maybe the best way, at least then I could still get some keyboard events too

See this reply. The alternative is pbuffers and LWJGL supports those too, but they are not cross-platform (e.g. WGL_ARB_pbuffer)

[BrickFarmer]

Thanks. I've decided to stick with the window.  As someone pointed out on the oculus forum, its the best way to know the rift is active.

Anyway I have updated my demo to use frustum for now, so at least the scene renders! if no rift is found it auto rotates on the y axis.

[BrickFarmer]

Hi,
your experimentation with the Rift seems quite cool.
If you manage to get the projection settings working, which looks to be quite complicated, involving some number of parameters, we could somehow wrap that in a nice method and add that to a coming JOML-LWJGL integration library, so that one would not have to fiddle around with all that native structs and buffer stuff, which looks like it can be hidden away quite effectively, especially because... well... the Rift is just the Rift with always? the same frustum settings?
I do not own a Rift myself, but I figure the projection matrix creation can be wrapped quite nicely.

Thanks.  Yes I hope to put a dependency on to JOML as soon as it makes maven   I want to try and put together another couple of demos to show what LWJGL 3 can do for the Rift community   Yesterday I was working on splitting out the Rift specific code, and the scene code.  So hopefully that will make it easier for people trying the library.  Code reviews always welcome!  I know my OpenGL is very rusty!  Once I have this GL11 version working I hope to do a shader based version, and that is all new for me.  Hopefully we will be able to do an ES2 version as well and maybe find a common platform between the Rift and GearVR for example.  Anyway I will need help on the math side for sure, the code seems ok with this frustum call, but I would prefer if I could fix the loadMatrix issue, since that might be why I'm also seeing some warping, and some strange effects on the model view stuff.

[BrickFarmer]

Updated my example with fixed Matrix code and now using JOML

I've checked in my code, which runs without a rift (assuming you have rift enabled build of LWJGL3). Based on JOML 1.2.6 (since I think the API has changed).  Ignore my maven stuff for now.

Full code here:

https://github.com/WhiteHexagon/example-lwjgl3-rift/tree/master/src/main/java/com/sunshineapps/rift/experimental

I have some issue with my colorBuffer, ie the globe changes color over time...

Switching method2 to method4 for my longitude/latitude calculations makes for an interesting optical illusion.

I do see some very slight movement in the scene geometry when moving my head around, but it's difficult to describe and only very slight.  It could just be a precision issue, or a slightly incorrect projection matrix from the Rift SDK.  I've no idea really, but something feels very slightly out.

here is the relevant matrix code for anyone trying to do this:

glMatrixMode(GL_MODELVIEW);
mat.identity();
Vector3f offsetPosition = new Vector3f(eyeRenderDesc[eye].getHmdToEyeViewOffsetX(), eyeRenderDesc[eye].getHmdToEyeViewOffsetY(), eyeRenderDesc[eye].getHmdToEyeViewOffsetZ());
mat.translate(offsetPosition);
Quaternion orientation = new Quaternion(eyePose.getOrientationX(), eyePose.getOrientationY(), eyePose.getOrientationZ(), eyePose.getOrientationW());
orientation.invert();
mat.rotate(orientation);
Vector3f position = new Vector3f(-eyePose.getPositionX(), -eyePose.getPositionY(), -eyePose.getPositionZ());
mat.translate(position);
mat.translate(playerEyePos); //back to 'floor' height