[Solved] 3D Orientation vectors

[Andrew_3ds]

If I understand it correctly, the AL orientation uses two vectors known as "At" and "Up" to get the listener's head rotation. However, my camera doesn't use up and forward vector for rotation. Is there a way to change the way it understands orientation, or if not how would I convert my rotation to vectors? My camera uses simple XYZ rotation, with the X rotation being bound between 90 and -90, and the Y rotation being bound between 180 and -180. Is there a function in JOML to convert them to vectors?

[Kai]

This one is easy.

How it works the long way
Once you've built the camera matrix, you can transform the default orientation vectors for at=(0, 0, -1) and up=(0, 1, 0) with the inverse of that matrix to obtain the real 'at' and 'up' vectors for OpenAL, like so:

Matrix4f cam = ...; // <- some camera transformation
Matrix4f invCam = new Matrix4f(cam).invert(); // <- inverse of camera
Vector3f at = new Vector3f(0, 0, -1);
invCam.transformDirection(at);
Vector3f up = new Vector3f(0, 1, 0);
invCam.transformDirection(up);

After this, at and up contain the transformed vectors for OpenAL.
The method Matrix4f.transformDirection() takes care that only the rotational part of the camera transformation is used when transforming a direction vector as opposed to a position.

Why do we need to invert the camera matrix?
The reason for that is that the uninverted camera matrix transforms from some "world" coordinate system into eye/view/camera space.
In the latter space the direction along which the camera is looking is always (0, 0, -1) and up is always (0, 1, 0).
But we need the opposite transformation to get from that view/eye/camera-space vectors to the "world" vectors for that particular camera transformation. And the opposite transformation in matrix-speak is the matrix inverse.

The shortcut for this in JOML
Now that you know how it works in theory, let's do it more efficiently.  Because obtaining the world-space vectors from a (camera) transformation matrix for the base axes is such a frequent task, JOML provides a faster shortcut method for this that does not require to calculate the matrix inverse and does not require to perform a full matrix/vector multiplication:

Matrix4f cam = ...; // <- some camera transformation
Vector3f at = new Vector3f();
cam.positiveZ(at).negate();
Vector3f up = new Vector3f();
cam.positiveY(up);

This is equivalent to the above code.

[Andrew_3ds]

Wow, thank you. I didn't know it would actually be that easy. It's working now and it's cool to hear the 3D sound working .

[orange451]

@Kai

What's the at, if your up is (0, 0, 1)?

[Kai]

Okay, I think there might be a confusion here, about what I meant with the "up" vector above.
Correct me if I'm wrong, but your up=(0, 0, 1) vector is likely in world coordinates that you use for your own camera transformation (probably via a lookAt operation) to define which vector (in world coordinates!) should map to (0, 1, 0) in the local camera frame.

The "up"=(0, 1, 0) I meant above however is not in world coordinates but in OpenAL's local "listener" coordinate system, or likewise think OpenGL's view/eye/camera coordinate system.
It is always by definition (0, 1, 0) because it's the way OpenGL's conventional view/eye/camera and OpenAL's "listener" coordinate systems are defined.
Likewise "at" in those "right handed" coordinate systems is always by definition (0, 0, -1) (i.e. pointing "into" the screen, away from the viewer).

So, you can choose whichever world-space "up" vector or "lookat" direction you want for your camera matrix. That all works with arbitrary camera matrices.

[orange451]

Ah alright, thanks for the clarification.