Trackball implementation using JOML

[Automaton]

Hey guys I've been beating my head against a trackball implementation issue using JOML and was hoping I could get some assistance in understanding my issue.

Before we take a look at my code let me explain my paradigm: my thought was that since I plan on adding several objects to my "scene" I want to rotate the "camera" about the origin of the scene using a trackball, as opposed to translating+rotating each object within the scene in order to achieve the same effect. Also, while I'm currently implementing the trackball functionality in Java using the CPU, I plan on migrating the associated computations into the GPU using the shaders once my understanding is firmed up. Let me know if this is the wrong way to go here!

I've created some sample code which should give a sense of what I'm trying to achieve. The code assumes SWT, LWJGL 3, and JOML without a fixed pipe. In the code I have two cubes which I've placed on either side of the yz-plane. One cube is stationary and centered on the x-axis, the other is spinning while also orbiting the x-axis. Using the trackball technique I'd like to be able to move the camera about my scene while continuing to look at the origin and maintaining a constant distance from the origin (I'll implement zoom, later). Further, I'd like for there to be no gimbal-lock.

I've been able to get really close to achieving this, but there is still some unexpected behavior that I'm having trouble tracking down. Specifically, when the user drags quickly the camera appears to get closer to the origin thus not meeting my "constant distance" criterion. I suspect the issue lies in my understanding of quaternions, which is elementary at best, but I may be doing something stupid. I'm hoping someone can spot the problem.

Here's a complete example:

TrackballTest Class

import static org.lwjgl.opengl.GL20.glGetUniformLocation;
import static org.lwjgl.opengl.GL20.glUniformMatrix4fv;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.nio.FloatBuffer;

import org.eclipse.swt.SWT;
import org.eclipse.swt.SWTException;
import org.eclipse.swt.events.MouseEvent;
import org.eclipse.swt.events.MouseListener;
import org.eclipse.swt.events.MouseMoveListener;
import org.eclipse.swt.graphics.Color;
import org.eclipse.swt.layout.FillLayout;
import org.eclipse.swt.opengl.GLCanvas;
import org.eclipse.swt.opengl.GLData;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Shell;
import org.joml.AxisAngle4f;
import org.joml.Matrix3f;
import org.joml.Matrix4f;
import org.joml.Quaternionf;
import org.joml.Vector3f;
import org.lwjgl.BufferUtils;
import org.lwjgl.opengl.GL;
import org.lwjgl.opengl.GL11;
import org.lwjgl.opengl.GL20;
import org.lwjgl.opengl.GLUtil;

import shape.Cube;

public class TrackballTest extends Composite
{
  private static final String VERTEX_ATTRIBUTE_POSITION = "position";

  private static final String VERTEX_ATTRIBUTE_COLOR    = "color";

  private static final String VERTEX_FIELD_MODEL        = "model";

  private static final String VERTEX_FIELD_VIEW         = "view";

  private static final String VERTEX_FIELD_PROJECTION   = "projection";

  // Canvas stuff
  private GLCanvas            canvas;

  private float               width, height;

  // Shader stuff
  private int                 vertexShaderID, fragmentShaderID, programID;

  private int                 modelMatrixID, projectionMatrixID, viewMatrixID;

  // Math stuff
  private FloatBuffer         buffer                    = BufferUtils.createFloatBuffer(16);

  private Matrix4f            projectionMatrix, viewMatrix;

  private Vector3f            eye, look, up, start, end;

  private boolean             isDragging                = false;

  // Model objects
  private Cube                cube1, cube2;

  private double              theta                     = 0;

  public TrackballTest(Composite parent, int style)
  {
    super(parent, style);
    this.setLayout(new FillLayout());

    initializeCanvas();
    initializeListeners();
    initializeShaderProgram();
    initializeScene();

    // Initialize our objects
    cube1 = new Cube();
    cube2 = new Cube();

    start();
  }

  private void initializeCanvas()
  {
    // Create GL data with double buffering
    GLData data = new GLData();
    data.doubleBuffer = true;

    // Create the SWT context
    canvas = new GLCanvas(this, SWT.NONE, data);

    // Set the context as current
    canvas.setCurrent();
    GL.createCapabilities(true);

    // Initialize the default clear color as SWT's default background color
    Color bg = this.getDisplay().getSystemColor(SWT.COLOR_WIDGET_BACKGROUND);
    GL11.glClearColor(bg.getRed() / (float) 255, bg.getGreen() / (float) 255,
        bg.getBlue() / (float) 255, 1f);
    bg.dispose();

    GLUtil.setupDebugMessageCallback();
  }

  private void initializeListeners()
  {
    // Click listener (for trackball events)
    canvas.addMouseListener(new MouseListener()
    {

      @Override
      public void mouseUp(MouseEvent e)
      {
        isDragging = false;
        start = null;
        end = null;
      }

      @Override
      public void mouseDown(MouseEvent e)
      {
        // If this is the first mouse down we record the mouse start location
        if (!isDragging)
        {
          start = getArcballVector(e.x, e.y);
        }
        isDragging = true;
      }

      @Override
      public void mouseDoubleClick(MouseEvent e)
      {
        // Nothing
      }
    });

    // Move listener (for trackball drag)
    canvas.addMouseMoveListener(new MouseMoveListener()
    {

      @Override
      public void mouseMove(MouseEvent e)
      {
        if (isDragging)
        {
          end = getArcballVector(e.x, e.y);
        }
      }
    });
  }

  private void initializeShaderProgram()
  {
    int errorCheckValue = GL11.glGetError();

    // Load the shaders
    vertexShaderID = this.loadShader("src/plot/vertexShader.glsl", GL20.GL_VERTEX_SHADER);
    fragmentShaderID = this.loadShader("src/plot/fragmentShader.glsl", GL20.GL_FRAGMENT_SHADER);

    // Create a new shader program that links both shaders
    programID = GL20.glCreateProgram();
    GL20.glAttachShader(programID, vertexShaderID);
    GL20.glAttachShader(programID, fragmentShaderID);

    // Position information will be attribute 0
    GL20.glBindAttribLocation(programID, 0, VERTEX_ATTRIBUTE_POSITION);

    // Color information will be attribute 1
    GL20.glBindAttribLocation(programID, 1, VERTEX_ATTRIBUTE_COLOR);

    GL20.glLinkProgram(programID);
    GL20.glValidateProgram(programID);

    errorCheckValue = GL11.glGetError();
    if (errorCheckValue != GL11.GL_NO_ERROR)
    {
      System.out.println("ERROR - Could not create the shaders!");
      System.exit(-1);
    }
  }

  private void initializeScene()
  {
    // Model matrix
    modelMatrixID = glGetUniformLocation(programID, VERTEX_FIELD_MODEL);

    // Perspective matrix (closer objects should appear larger)
    viewMatrixID = glGetUniformLocation(programID, VERTEX_FIELD_VIEW);
    viewMatrix = new Matrix4f();
    viewMatrix.identity();

    // Perspective matrix (closer objects should appear larger)
    projectionMatrixID = glGetUniformLocation(programID, VERTEX_FIELD_PROJECTION);
    projectionMatrix = new Matrix4f();
    projectionMatrix.identity();

    eye = new Vector3f(0f, 0f, 10f);
    look = new Vector3f(0f, 0f, 0f);
    up = new Vector3f(0f, 1f, 0f);
  }

  private void updateScene()
  {
    canvas.setCurrent();

    int w = canvas.getBounds().width;
    int h = canvas.getBounds().height;
    width = (float) w;
    height = (float) h;
    float aspect = width / height;

    // Map the internal OpenGL coordinate system to the entire screen. Since the
    // user can resize the window, we do this here.
    GL11.glViewport(0, 0, w, h);

    // Make sure to include depth testing and depth bit or we will see weird
    // things
    GL11.glClear(GL11.GL_COLOR_BUFFER_BIT | GL11.GL_DEPTH_BUFFER_BIT);
    GL11.glEnable(GL11.GL_DEPTH_TEST);

    GL20.glUseProgram(programID);
    updateViewPerspective(aspect);

    // Shift cubes off the origin
    cube1.getModelMatrix().setTranslation(-1f, 0f, 0f);
    cube2.getModelMatrix().setTranslation(1f, 0f, 0f);

    // Let's make cube 2 do crazy stuff like spin while orbiting the x axis
    cube2.getModelMatrix().rotate(0.03f, 1f, 0f, 0f);
    cube2.getModelMatrix().rotate(0.01f, 0f, 1f, 0f);

    theta += 1 % 360;
    cube2.getModelMatrix().setTranslation(1f, (float) Math.cos(Math.toRadians(theta)) * 2f,
        (float) Math.sin(Math.toRadians(theta)) * 2f);

    // Draw the cubes
    cube1.draw(modelMatrixID);
    cube2.draw(modelMatrixID);

    canvas.swapBuffers();
  }

  private void updateViewPerspective(float aspect)
  {
    if (isDragging)
    {
      if (end != null && start != null)
      {
        float angle = -10 * (float) Math.min(Math.acos(start.dot(end)), 1f);
        Vector3f axis = new Vector3f(start);
        axis.cross(end);

        Matrix3f vi = new Matrix3f(viewMatrix);
        vi.invert();
        axis.mul(vi);

        if (!Float.isNaN(angle))
        {
          Quaternionf q = new Quaternionf(new AxisAngle4f(angle, axis));
          eye.rotate(q);
          up.rotate(q);
          start = new Vector3f(end);
        }
      }
    }

    viewMatrix.setLookAt(eye, look, up);
    viewMatrix.get(buffer);
    glUniformMatrix4fv(viewMatrixID, false, buffer);

    projectionMatrix.setPerspective((float) Math.toRadians(45), aspect, 0.1f, 100f);
    projectionMatrix.get(buffer);
    glUniformMatrix4fv(projectionMatrixID, false, buffer);
  }

  private Vector3f getArcballVector(int x, int y)
  {
    Vector3f p = new Vector3f();
    p.x = 2f * x / width - 1f;
    p.y = 1f - 2f * y / height;

    float squared = p.x * p.x + p.y * p.y;
    if (squared <= 1)
    {
      p.z = (float) Math.sqrt(1 - squared);
    }
    else
    {
      p.normalize();
    }
    return p;
  }

  public int loadShader(String filename, int type)
  {
    StringBuilder shaderSource = new StringBuilder();
    int shaderID = 0;

    try
    {
      BufferedReader reader = new BufferedReader(new FileReader(filename));
      String line;
      while ( ( line = reader.readLine() ) != null)
      {
        shaderSource.append(line).append("\n");
      }
      reader.close();
    }
    catch (IOException e)
    {
      System.err.println("Could not read file.");
      e.printStackTrace();
      System.exit(-1);
    }

    shaderID = GL20.glCreateShader(type);
    GL20.glShaderSource(shaderID, shaderSource);
    GL20.glCompileShader(shaderID);

    return shaderID;
  }

  private void start()
  {
    Thread t = new Thread(new Runnable()
    {

      @Override
      public void run()
      {
        while (!TrackballTest.this.isDisposed())
        {
          TrackballTest.this.getDisplay().asyncExec(new Runnable()
          {

            @Override
            public void run()
            {
              updateScene();
            }
          });

          try
          {
            // Approximate a 30 FPS update rate for now. We'll get more fancy
            // later on.
            Thread.sleep(1000 / 30);
          }
          catch (InterruptedException e)
          {
            e.printStackTrace();
          }
        }
      }
    });
    t.start();
  }

  public static void main(String[] args)
  {
    Display d = new Display();

    Shell s = new Shell(d);
    int width = 1024;
    int height = 720;
    s.setSize(width, height);
    s.setLayout(new FillLayout());

    new TrackballTest(s, SWT.NONE);

    s.open();
    while (!s.isDisposed())
    {
      if (!d.readAndDispatch())
      {
        d.sleep();
      }
    }

    try
    {
      d.dispose();
    }
    catch (SWTException e)
    {
      // Do nothing
      System.exit(0);
    }
  }
}

Cube Class

package shape;

import static org.lwjgl.opengl.GL20.glUniformMatrix4fv;

import java.nio.ByteBuffer;
import java.nio.FloatBuffer;

import org.joml.Matrix4f;
import org.lwjgl.BufferUtils;
import org.lwjgl.opengl.GL11;
import org.lwjgl.opengl.GL15;
import org.lwjgl.opengl.GL20;
import org.lwjgl.opengl.GL30;

public class Cube
{
  // Quad variables
  private int         vaoId                   = 0;

  private int         verticesBufferId        = 0;

  private int         verticesColorBufferId   = 0;

  private int         verticesIndicesBufferId = 0;

  private int         numberOfIndices         = 0;

  private Matrix4f    modelMatrix             = new Matrix4f();

  private FloatBuffer buffer                  = BufferUtils.createFloatBuffer(16);

  public Cube()
  {
    float[] vertices = { -0.5f, -0.5f, 0.5f, 1f, // 0
        0.5f, -0.5f, 0.5f, 1f, // 1
        0.5f, 0.5f, 0.5f, 1f, // 2
        -0.5f, 0.5f, 0.5f, 1f, // 3
        -0.5f, -0.5f, -0.5f, 1f, // 4
        0.5f, -0.5f, -0.5f, 1f, // 5
        0.5f, 0.5f, -0.5f, 1f, // 6
        -0.5f, 0.5f, -0.5f, 1f, // 7
    };
    FloatBuffer verticesBuffer = BufferUtils.createFloatBuffer(vertices.length);
    verticesBuffer.put(vertices);
    verticesBuffer.flip();

    float[] colors = { 1f, 0f, 0f, 1f, // 1
        0f, 1f, 0f, 1f, // 2
        0f, 0f, 1f, 1f, // 3
        1f, 1f, 1f, 1f, // 7
        1f, 0f, 0f, 1f, // 4
        0f, 1f, 0f, 1f, // 5
        0f, 0f, 1f, 1f, // 6
        1f, 1f, 1f, 1f, // 7
    };
    FloatBuffer colorsBuffer = BufferUtils.createFloatBuffer(colors.length);
    colorsBuffer.put(colors);
    colorsBuffer.flip();

    byte[] indices = { 0, 1, 2, // front1
        2, 3, 0,// front2
        3, 2, 6, // top 1
        6, 7, 3,// top 2
        7, 6, 5, // back 1
        5, 4, 7,// back 2
        4, 5, 1, // bottom 1
        1, 0, 4, // bottom 2
        4, 0, 3, // left 1
        3, 7, 4,// left 2
        1, 5, 6, // right 1
        6, 2, 1, // right 2
    };
    numberOfIndices = indices.length;
    ByteBuffer indicesBuffer = BufferUtils.createByteBuffer(numberOfIndices);
    indicesBuffer.put(indices);
    indicesBuffer.flip();

    // Create and bind a new VAO
    vaoId = GL30.glGenVertexArrays();
    GL30.glBindVertexArray(vaoId);

    // Create and bind a VBO for the vertices
    verticesBufferId = GL15.glGenBuffers();
    GL15.glBindBuffer(GL15.GL_ARRAY_BUFFER, verticesBufferId);
    GL15.glBufferData(GL15.GL_ARRAY_BUFFER, verticesBuffer, GL15.GL_STATIC_DRAW);
    GL20.glVertexAttribPointer(0, 4, GL11.GL_FLOAT, false, 0, 0);
    GL15.glBindBuffer(GL15.GL_ARRAY_BUFFER, 0);

    // Create and bind a VBO for the colors
    verticesColorBufferId = GL15.glGenBuffers();
    GL15.glBindBuffer(GL15.GL_ARRAY_BUFFER, verticesColorBufferId);
    GL15.glBufferData(GL15.GL_ARRAY_BUFFER, colorsBuffer, GL15.GL_STATIC_DRAW);
    GL20.glVertexAttribPointer(1, 4, GL11.GL_FLOAT, false, 0, 0);
    GL15.glBindBuffer(GL15.GL_ARRAY_BUFFER, 0);

    // Create and bind a VBO for the indices
    verticesIndicesBufferId = GL15.glGenBuffers();
    GL15.glBindBuffer(GL15.GL_ELEMENT_ARRAY_BUFFER, verticesIndicesBufferId);
    GL15.glBufferData(GL15.GL_ELEMENT_ARRAY_BUFFER, indicesBuffer, GL15.GL_STATIC_DRAW);
    GL15.glBindBuffer(GL15.GL_ELEMENT_ARRAY_BUFFER, 0);
  }

  public void draw(int modelMatrixID)
  {
    // Bind the VAO
    GL30.glBindVertexArray(vaoId);
    GL20.glEnableVertexAttribArray(0);
    GL20.glEnableVertexAttribArray(1);

    // Bind the indices VBO
    GL15.glBindBuffer(GL15.GL_ELEMENT_ARRAY_BUFFER, verticesIndicesBufferId);

    // Draw the vertices
    GL11.glDrawElements(GL11.GL_TRIANGLES, numberOfIndices, GL11.GL_UNSIGNED_BYTE, 0);

    modelMatrix.get(buffer);
    glUniformMatrix4fv(modelMatrixID, false, buffer);
  }

  public Matrix4f getModelMatrix()
  {
    return modelMatrix;
  }
}

Vertex Shader

#version 150 core

in vec4 position;
in vec4 color;

out vec4 vertexColor;

uniform mat4 model;
uniform mat4 view;
uniform mat4 projection;

void main() {
	mat4 mvp =  projection * view * model;
    gl_Position = mvp * position;
    vertexColor = color;
}

Fragment shader

#version 150 core

in vec4 vertexColor;

out vec4 fragColor;

void main() {
    fragColor = vertexColor;
}

[Kai]

I've already done this in the joml-camera project. See: ArcBallCamera.java
A nice additional feature is the lazy/momentum-based movement. But you can easily get that out, if not desired.
Demo is in the joml-lwjgl3-demos project. See: ArcBallCameraDemo.java

Essentially, what you are looking for is the following matrix:

m.translate(0, 0, zoom)
 .rotateX(rotationUpDown)
 .rotateY(rotationLeftRight)
 .translate(negatedCenterPosition);

This applies the arcball view transformation to the matrix 'm'.

[Kai]

...I want to rotate the "camera" about the origin of the scene using a trackball, as opposed to translating+rotating each object within the scene in order to achieve the same effect.

Well, strictly and mathematically speaking there is only "one" possible method to rotate/translate/transform something in OpenGL.
And that is: you always transform the objects, you are rendering, "into" the camera view. There is no other way to transform something in OpenGL. Methodically you always start from the model-space vertices and transform them into camera-space by matrix-vector multiplications.
It's just that functions like gluLookAt make us think that it's the other way around and we are dealing with some kind of "camera." To achieve this, this function makes us think that the camera is transformed in the world so that it views the objects within the world. So, it is the opposite of what OpenGL really does - not transforming the objects into view space, but the view "into" object space.
Both transformations are indeed the "inverse" of each other, as in "matrix inverse."
And this is exactly what gluLookAt/Matrix4f.lookAt does: It interfaces us with the "intuitive" camera parameters (position, center, up), but internally it builds the inverse of that transformation to have it the way OpenGL works, by transforming the objects.

[Automaton]

Hey Kai, thanks for the explanation and for the code. I really like your ArcBall implementation. I was able to hack it in really easily and it works as expected (plus some cool momentum)!

Just to make sure I'm interpreting things correctly, you say "lazy", is this 'cause you're approximating the angle as opposed to calculating it:

      cam.setAlpha(cam.getAlpha() + Math.toRadians( ( x - mouseX ) * 0.1f));
      cam.setBeta(cam.getBeta() + Math.toRadians( ( mouseY - y ) * 0.1f));

[Kai]

I really like your ArcBall implementation.

Thank you! That's nice to hear. Someone finally making use of the arcball camera, instead of asking for the promised "free look" camera.
You can always ask me/tell me if you need anything changed with that. Currently you are the first one I know of using it, so we could change anything from behaviour to API.
However, I will make the "movement/rotation integrator" swappable.
The existing with "momentum" and one with "direct"/choppy/instant movement, and maybe also an option to restrict the rotation to discrete steps of X degrees, for exact controlled rotation, as is desirable with 3D content creation tools.

Just to make sure I'm interpreting things correctly, you say "lazy", is this 'cause you're approximating the angle as opposed to calculating it

The quoted code is also "exact" and does not approximate. It just increments the angles by the difference of the mouse positions. Dunno what you mean by "approximating."

[Automaton]

The quoted code is also "exact" and does not approximate. It just increments the angles by the difference of the mouse positions. Dunno what you mean by "approximating."

Sorry that it took me a while to respond. I needed to understand exactly what is different from what I was expecting and what I was observing and it's not the calculation of the angle, but the axis about which the rotation is occurring that was throwing me off.

I was thinking in terms of projecting onto an imaginary sphere as opposed to simple increments proportional to mouse movement. I actually like your method better. It's more intuitive. It is different from most 3D cad systems I have worked in, however.

As an example, here is what ArcBallCamera does:


Mouse motion in the window-y-axis of the screen rotates the cube around the model-x-axis until you are looking down from the top at which point gimbal lock is encountered and continued motion in the window-y-axis instead rotates the cube around the model-y-axis.

Here is what Solidworks does and what I was initially trying to achieve:


The gimbal doesn't lock at the top and continued mouse motion in the window-y-axis continues to rotate the cube about the model-x-axis.

As I said, for most general users with little 3D experience (my users), I think your implementation is actually more intuitive, but I was initially expecting what I had become used to using solidworks.

I'm not actually sure what mathematically causes the gimbal lock. Perhaps a preservation of what "up" is? If you are considering changes to your ArcBall implementation it may be convenient to be able to switch/specify which type of motion to use as well as toggling acceleration/momentum.

[Kai]

...until you are looking down from the top at which point gimbal lock is encountered.
I'm not actually sure what mathematically causes the gimbal lock. Perhaps a preservation of what "up" is?

This one is really easy to "fix." I intended the motion to stop when the "poles" were reached.
This is achieved by the range check in the setBeta(float) method.
Remove that and you have a continuous rotation. However, then the Y axis is inverted.
That's why I added the angle restriction.