Oculus Rift SDK 1.3 released

[bluenote10]

Today Oculus has finally released the first official consumer version of their SDK:

https://developer.oculus.com/downloads/pc/1.3.0/Oculus_SDK_for_Windows/

I only had a brief look so far, but judging from the 0.8 to 1.3 migration guide it looks like the change isn't as big as the version jump suggests. Mainly some renamed/removed functionality. Not sure if this new VR Focus Management requires any bigger changes though. Will have a closer look in the next days, maybe I can help porting LWJGL to the new version.

[BrickFarmer]

I'm available for testing with my DK2.  Having tried some of the 1.3 demos I'm quite keen to see if we can still use Java for development

[BrickFarmer]

relating to this, is LWJGL3 supporting xbox controllers?  The Rift ships with one and also with a seperate 'remote' that we may want to support...

[spasi]

relating to this, is LWJGL3 supporting xbox controllers?

Yes, via GLFW. Actually, this issue was closed yesterday, so controller support should be great now.

On topic, I'll update the LibOVR bindings to 1.3.0 today or tomorrow. Sorry for the delay.

[BrickFarmer]

Awesome! always one step ahead of us

[spasi]

The latest nightly build (3.0.0 #64) supports Oculus SDK 1.3.0.

[bluenote10]

Wow, thanks a lot for solving this so quickly -- awesome job! I got hold back by some other tasks and was just about to dig into this.

I'm now experimenting with the new build and there is one thing that isn't quite clear to me. What has changed from 0.8 to 1.3 is that the texture swap chain is now an opaque pointer. So it is no longer possible to extract the textures / textures count / current index directly but by means of calling ovr_GetTextureSwapChainBufferGL / ovr_GetTextureSwapChainLength /     ovr_GetTextureSwapChainCurrentIndex. I can see the latter two in class OVR. ovr_GetTextureSwapChainBufferGL should probably live in class OVRGL, but I can't see it. Did I somehow overlook it or is there some other way to get texture IDs stored in the swap chain?

[spasi]

Did I somehow overlook it?

No, you didn't, I missed it in the 0.8-1.3.0 diff. Adding now, it will appear in the next build. Thanks!

[BrickFarmer]

Hi,  Thanks for the update, I'm also just porting my 'demo' over and it seems to be pulling in the old sources from maven nightly, for example ovrEye_Count which I think has been removed, but shows up in my sources, or do I need to manually place a new nightly sources jar into the .m2 local repo?  Sorry it's been a while since I touched this code.

[Kai]

I always use this dependency for the Java jar, which works just fine and pulls the latest with `mvn -U clean package`:

<dependency>
	<groupId>org.lwjgl</groupId>
	<artifactId>lwjgl</artifactId>
	<version>3.0.0-SNAPSHOT</version>
</dependency>

Make sure to not use any 'A' or 'B' suffix in the version.

[spasi]

Build #65 is up.

The latest snapshot sources look fine.

[BrickFarmer]

thanks! I got maven to pull the sources in, but the name of the jar had changed, so on with my migration and I'm going to hard code it with 2 eyes

[bluenote10]

That was again super quick, thanks a lot! Migration to 1.3.0 worked very well for me, and everything seems to be running perfectly so far.

@BrickFarmer: I had a look in your code and there is one potential bug when moving to 1.3: In this line you are iterating over "eyes", but the loop should rather go over the number of textures in the swap chain (so using ovr_GetTextureSwapChainLength for the loop length; and it would make more sense to rename "eye" in just "i"). In 0.8 the number of textures in the swap chain happened to be 2 like the number of eyes. With the 1.3 SDK I now get 3 textures, probably due to asynchronous timewarp.

[BrickFarmer]

@bluenote10 Thanks!!  It seems like they changed quite a bit more than I thought! but well done on getting yours working so fast do you DK2 or CV1?

[bluenote10]

If it is of any help, this is my Scala code (still WIP, not cleaned up -- but since it was inspired by your example you should get the idea):

import org.lwjgl.glfw.GLFW._
import org.lwjgl.opengl.GL
import org.lwjgl.opengl.GL11._
import org.lwjgl.opengl.GL12._
import org.lwjgl.opengl.GL21._
import org.lwjgl.opengl.GL14._
import org.lwjgl.opengl.GL20._
import org.lwjgl.opengl.GL30._

import org.lwjgl.ovr._
import org.lwjgl.ovr.OVR._
import org.lwjgl.ovr.OVRUtil._
import org.lwjgl.ovr.OVRGL._
import org.lwjgl.ovr.OVRErrorCode._
import org.lwjgl.system.MemoryUtil
import org.lwjgl.BufferUtils


trait OvrWrapper {
  
  def updateCamera(): CameraState
  def render(renderFunction: MatrixSet => Unit, clearColor: Color)
  
  def resetPosition()
  def toggleHud()

  def isCloseRequested(): Boolean
  def processMessages()
  
  def destroy()
}


class OvrWrapperWindows(Boolean, useSRGB: Boolean = false) extends OvrWrapper {

  private val logger = Logger(LoggerFactory getLogger getClass)
  
  OvrWrapperWindows.init()
  OvrWrapperWindows.detect()
  
  val (session, luid) = OvrWrapperWindows.createSession()

  // Usage of ovr_ConfigureTracking is no longer needed unless you want to disable tracking features.
  // ovr_ConfigureTracking(session, 0, 0)

  val window = OvrWrapperWindows.initGlContext()
  
  
  // get hmd desc
  val hmdDesc = OVRHmdDesc.malloc()
  ovr_GetHmdDesc(session, hmdDesc)
  val resolution = hmdDesc.Resolution()
  
  // prepare projection matrices
  val projections = Array.tabulate(2){ eye =>
    val m = OVRMatrix4f.calloc()
    OVRUtil.ovrMatrix4f_Projection(hmdDesc.DefaultEyeFov(eye), 0.05f, 1000f, ovrProjection_ClipRangeOpenGL, m)
    val valuesArray = Array.tabulate(16)(i => m.M(i))
    val mConverted = Mat4f.createFromRowMajorArray(valuesArray)
    m.free()
    mConverted
  }
  
  // determine backbuffer size
  val recommenedTex0Size = OVRSizei.calloc()
  val recommenedTex1Size = OVRSizei.calloc()
  ovr_GetFovTextureSize(session, ovrEye_Left, hmdDesc.DefaultEyeFov(0), 2.0f, recommenedTex0Size)    
  ovr_GetFovTextureSize(session, ovrEye_Right, hmdDesc.DefaultEyeFov(1), 2.0f, recommenedTex1Size)    
  
  val wTotal = recommenedTex0Size.w + recommenedTex1Size.w
  val hTotal = recommenedTex0Size.h max recommenedTex1Size.h
  logger.debug(f"recommenedTex0Size = ${recommenedTex0Size.w} x ${recommenedTex0Size.h}")
  logger.debug(f"recommenedTex1Size = ${recommenedTex1Size.w} x ${recommenedTex1Size.h}")
  logger.debug(f"total = $wTotal x $hTotal")
  
  // create fbo
  val fbo = glGenFramebuffers()
  GlWrapper.checkGlError("after glGenFramebuffers", false)

  val renderBuffer = glGenRenderbuffers()
  glBindRenderbuffer(GL_RENDERBUFFER, renderBuffer)
  glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT24, wTotal, hTotal)
  glBindRenderbuffer(GL_RENDERBUFFER, 0)
  GlWrapper.checkGlError("after glRenderbufferStorage", false)
  
  glBindFramebuffer(GL_FRAMEBUFFER, fbo)
  glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, renderBuffer)
  glBindFramebuffer(GL_FRAMEBUFFER, 0)
  GlWrapper.checkGlError("after glFramebufferRenderbuffer", false)
  
  // create swap texture set
  val textureSwapChainPB = BufferUtils.createPointerBuffer(1)
  // It looks like that in order to disable sRGB we have to
  // set the texture format to sRGB but omit the call the
  // enabling sRGB in the framebuffer. Info here:
  // https://forums.oculus.com/community/discussion/24347/srgb-and-sdk-0-6-0-0
  val colorSpace = ovrFORMAT_R8G8B8A8_UNORM_SRGB
  val textureSwapChainDesc = OVRTextureSwapChainDesc.calloc()
  textureSwapChainDesc.Type(ovrTexture_2D)
  textureSwapChainDesc.ArraySize(1)
  textureSwapChainDesc.Format(colorSpace)
  textureSwapChainDesc.Width(wTotal)
  textureSwapChainDesc.Height(hTotal)
  textureSwapChainDesc.MipLevels(1) // TODO: what do I need here -- looks like values > 1 result in black screen...
  textureSwapChainDesc.SampleCount(1)
  textureSwapChainDesc.StaticImage(false)
  textureSwapChainDesc.MiscFlags(ovrTextureMisc_None) // TODO: what do I need here
  if (ovr_CreateTextureSwapChainGL(session, textureSwapChainDesc, textureSwapChainPB) != ovrSuccess) {
    throw new IllegalStateException("Failed to create Swap Texture Set")
  }
  textureSwapChainDesc.free()
  
  val textureSwapChain = textureSwapChainPB.get(0)
  val texturesPerEyeCount = OvrWrapperWindows.getTextureSwapChainLength(session, textureSwapChain)
  logger.debug("texturesPerEyeCount = " + texturesPerEyeCount)
  
  val textures = Range(0, texturesPerEyeCount).map{ i =>
    val textureID = OvrWrapperWindows.getTextureSwapChainBufferGL(session, textureSwapChain, i)
    logger.debug(f"textureId = ${textureID}")
    glBindTexture(GL_TEXTURE_2D, textureID)
    //glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE)
    glGenerateMipmap(GL_TEXTURE_2D)
  }
  // important: we have to commit once before rendering the first frame
  ovr_CommitTextureSwapChain(session, textureSwapChain)
  
  
  // get eye render desc
  val eyeRenderDesc = Array.tabulate(2){ eye =>
    val eyeRenderDesc = OVREyeRenderDesc.malloc()
    ovr_GetRenderDesc(session, eye, hmdDesc.DefaultEyeFov(eye), eyeRenderDesc)
    eyeRenderDesc
  }
  
  // prepare layer
  val layer0 = OVRLayerEyeFov.calloc()
  layer0.Header().Type(ovrLayerType_EyeFov)
  layer0.Header().Flags(ovrLayerFlag_TextureOriginAtBottomLeft | ovrLayerFlag_HighQuality)
  layer0.ColorTexture(0, textureSwapChain)
  layer0.ColorTexture(1, textureSwapChain)
  layer0.Fov(0, eyeRenderDesc(0).Fov())
  layer0.Fov(1, eyeRenderDesc(1).Fov())
  layer0.Viewport(0).Size(recommenedTex0Size)
  layer0.Viewport(1).Size(recommenedTex1Size)
  layer0.Viewport(0).Pos.x(0)
  layer0.Viewport(0).Pos.y(0)
  layer0.Viewport(1).Pos.x(recommenedTex0Size.w)
  layer0.Viewport(1).Pos.y(0)
  
  val layers = BufferUtils.createPointerBuffer(1)
  layers.put(0, layer0.Header())
  
  hmdDesc.free()
  recommenedTex0Size.free()
  recommenedTex1Size.free()
  
  // prepare hmdToEyeViewOffsets (required for the call to ovr_CalcEyePoses)
  val hmdToEyeViewOffsets = OVRVector3f.calloc(2)
  hmdToEyeViewOffsets.put(0, eyeRenderDesc(ovrEye_Left).HmdToEyeOffset())
  hmdToEyeViewOffsets.put(1, eyeRenderDesc(ovrEye_Right).HmdToEyeOffset())    

  // prepare outEyePoses (required for the call to ovr_CalcEyePoses)
  val outEyePoses = OVRPosef.create(2)
  
  val tracking = OVRTrackingState.calloc()
  var headPose: OVRPosef = null
  
  val hudModes = Array(
    ovrPerfHud_Off,
    ovrPerfHud_PerfSummary,
    ovrPerfHud_LatencyTiming,
    ovrPerfHud_AppRenderTiming,
    ovrPerfHud_CompRenderTiming,
    ovrPerfHud_VersionInfo
  )
  var hudModeIndex = 0
  
  
  def updateCamera(): CameraState = {
    val timeDisplayMidpointSeconds = ovr_GetPredictedDisplayTime(session, 0)
    ovr_GetTrackingState(session, timeDisplayMidpointSeconds, true, tracking)
    
    headPose = tracking.HeadPose.ThePose
    
    val worldToCamera = OvrWrapperWindows.convertPoseToMatrix(headPose)
    val yaw = 0f
    val pitch = 0f
    val roll = 0f
    CameraState(worldToCamera, yaw, pitch, roll)
  }
  

  def render(renderFunction: MatrixSet => Unit, clearColor: Color) {
    
    GlWrapper.checkGlError("render start", false)
    
    val textureCurrentIndex = OvrWrapperWindows.getTextureSwapChainCurrentIndex(session, textureSwapChain)
    val textureID = OvrWrapperWindows.getTextureSwapChainBufferGL(session, textureSwapChain, textureCurrentIndex)
    // println(f"Rendering to index = ${textureCurrentIndex}, textureID = ${textureID}")
    
    OVRUtil.ovr_CalcEyePoses(headPose, hmdToEyeViewOffsets, outEyePoses)

    // Bind framebuffer and switch the binding to the current textureID.
    // Note that the renderbuffer binding does not have to be changed, i.e.,
    // we can bind it statically above. This is because LibOVR does not do
    // anything with the depth buffer in ATW.
    glBindFramebuffer(GL_DRAW_FRAMEBUFFER, fbo)
    glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureID, 0)
    glDrawBuffers(GL_COLOR_ATTACHMENT0)
    if (useSRGB) {
      glEnable(GL_FRAMEBUFFER_SRGB)
    } else {
      glDisable(GL_FRAMEBUFFER_SRGB)
    }
    GlWrapper.checkGlError("after binding fbo", false)

    GlWrapper.ClearColor.set(clearColor)
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)    
    GlWrapper.checkGlError("after clear", false)
    
    // loop over eyes
    cforRange(0 until 2){ i =>
      // update the eye pose in the layer
      val eyePose = outEyePoses.get(i)
      layer0.RenderPose(i, eyePose)
      
      // set the viewport -- since we have stored the required information already
      // in the layer description we can use it from there...
      glViewport(
        layer0.Viewport(i).Pos().x(),
        layer0.Viewport(i).Pos().y(),
        layer0.Viewport(i).Size().w(),
        layer0.Viewport(i).Size().h()
      )

      val V = OvrWrapperWindows.convertPoseToMatrix(eyePose)
      val P = projections(i)
      val mset = new MatrixSet(P, V)
      renderFunction(mset)
    }
    GlWrapper.checkGlError("after rendering", false)
    
    // we finally have to regenerate the mipmaps of the framebuffer texture
    glGenerateMipmap(GL_TEXTURE_2D)
    
    // cleanup: un-attach the texture from the framebuffer and unbind the framebuffer
    glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, 0, 0)
    glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0)
    GlWrapper.checkGlError("after unbinding fbo", false)
    
    // submit frame
    val result = ovr_SubmitFrame(session, 0, null, layers)
    if (result == ovrSuccess_NotVisible) {
      System.out.println("TODO not vis!!");
    } else if (result != ovrSuccess) {
      System.out.println("TODO failed submit");
    }
    GlWrapper.checkGlError("after ovr_SubmitFrame", false)
    
    // this switches the current index of the texture swap chain
    ovr_CommitTextureSwapChain(session, textureSwapChain)
  }

  def resetPosition() {
    ovr_RecenterTrackingOrigin(session)
  }
  
  def toggleHud() {
    hudModeIndex = (hudModeIndex + 1) % hudModes.length
    val hudMode = hudModes(hudModeIndex)
    ovr_SetInt(session, "PerfHudMode", hudMode)
  }
  
  def isCloseRequested(): Boolean = {
    glfwWindowShouldClose(window) == GL_TRUE
  }

  def processMessages() {
    glfwPollEvents()
  }
  
  def destroy() {
    // terminating GLFW before ovr_DestroyTextureSwapChain leads to a crash -- looks like we need this order...
    logger.debug("calling ovr_DestroyTextureSwapChain")
    ovr_DestroyTextureSwapChain(session, textureSwapChain)
    logger.debug("calling ovr_Destroy")
    ovr_Destroy(session)
    logger.debug("calling ovr_Shutdown")
    ovr_Shutdown()

    glfwDestroyWindow(window)
    glfwTerminate()      
    
    logger.debug("freeing eyeRenderDesc")
    eyeRenderDesc.foreach(_.free())
    
    logger.debug("freeing layer")
    layer0.free()
    logger.debug("freeing tracking")
    tracking.free()
    logger.debug("freeing luid")
    luid.free()
    
    logger.debug("freeing callback")
    OvrWrapperWindows.freeCallback()
  }
}


object OvrWrapperWindows {
  
  private val logger = Logger(LoggerFactory getLogger getClass)

  val callback = new OVRLogCallback() {
    override def invoke(userData: Long, level: Int, message: Long) {
      val strOrig = OVRLogCallback.getMessage(message)
      val strWithoutNewline = strOrig.filter(_ != '\n')
      logger.debug("LibOVR [" + level + "] " + strWithoutNewline)
    }
  }
  
  def freeCallback() {
    // the callback needs to be free'd -- last step in the destroy...
    callback.free()
  }
  
  def init() {
    val initParams = OVRInitParams.calloc()
    initParams.LogCallback(callback)
    initParams.Flags(ovrInit_Debug)

    logger.debug("ovr_Initialize = " + ovr_Initialize(initParams))
    initParams.free()
    
    logger.debug("ovr_GetVersionString = " + ovr_GetVersionString());
  }
  
  def detect() {
    val detect = OVRDetectResult.calloc()
    ovr_Detect(0, detect)

    logger.debug("OVRDetectResult.IsOculusHMDConnected = " + detect.IsOculusHMDConnected())
    logger.debug("OVRDetectResult.IsOculusServiceRunning = " + detect.IsOculusServiceRunning())
    // TODO: fail for both cases with appropriate messages boxes

    detect.free()
  }
  
  def createSession(): (Long, OVRGraphicsLuid) = {
    val sessionPointerBuffer = MemoryUtil.memAllocPointer(1)
    val luid = OVRGraphicsLuid.calloc()
    val createResult = ovr_Create(sessionPointerBuffer, luid)
    logger.debug("ovr_Create = " + createResult)
  
    val session = sessionPointerBuffer.get(0)
    MemoryUtil.memFree(sessionPointerBuffer)
    return (session, luid)
  }
  
  
  def initGlContext(): Long = {
    // init GL context
    logger.debug(" *** Initializing GL context")
    glfwInit()
    glfwDefaultWindowHints()
    glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3)
    glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 3)
    glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE)
    //glfwWindowHint(GLFW_VISIBLE, GL11.GL_FALSE)
    val window = glfwCreateWindow(640, 480, "Guitar Geeks VR", 0, 0)
    glfwSetKeyCallback(window, Keyboard.KeyCallback)
    glfwSetCharCallback(window, Keyboard.CharCallback)
    
    glfwMakeContextCurrent(window)
    GL.createCapabilities()
    
    logger.debug("     Initializing GL context [done]")  
    GlWrapper.checkGlError("after initializing context", false)
    
    return window
  }
  
  def getTextureSwapChainCurrentIndex(session: Long, chain: Long): Int = {
    val buffer = BufferUtils.createIntBuffer(1)
    ovr_GetTextureSwapChainCurrentIndex(session, chain, buffer)
    val currentIndex = buffer.get(0)
    return currentIndex
  }
  def getTextureSwapChainLength(session: Long, chain: Long): Int = {
    val buffer = BufferUtils.createIntBuffer(1)
    ovr_GetTextureSwapChainLength(session, chain, buffer)
    val textureCount = buffer.get(0)
    return textureCount
  }  
  def getTextureSwapChainBufferGL(session: Long, chain: Long, index: Int): Int = {
    val buffer = BufferUtils.createIntBuffer(1)
    ovr_GetTextureSwapChainBufferGL(session, chain, index, buffer)
    val textureID = buffer.get(0)
    return textureID
  }
  
  
  def convertPoseToMatrix(pose: OVRPosef): Mat4f = {
    val PcurInv = Mat4f.translate(-pose.Position.x, -pose.Position.y, -pose.Position.z)
    val QcurInv = new Quaternion(-pose.Orientation.x, -pose.Orientation.y, -pose.Orientation.z, pose.Orientation.w).castToOrientationMatrix
    QcurInv * PcurInv
  }
  
}