[2D] Getting 'absolute' world coordinates based off of mouse coordinates

[DGK]

Hello.

I am working on a 2D game engine / prototype and I am having trouble with something - getting the 'absolute' world coordinates based off mouse coordinates.

My 2D camera, or 'Viewport' is identified by its position, a vector, and a zoom factor, that is between .5F - 1.5F (50% - 150%). In order to render objects on the screen, I subtract the camera's coordinates with the object's coordinates. For zoom, I use GL11's glScalef(zoomFactor, zoomFactor, 0) to accomplish the zooming effect.

How would I go about getting the "world coordinates' based on where I click with the mouse? I have tried numerous methods to do this when the camera is zoomed in/out and it always gives an incorrect world coordinate.

I would like to get the exact world coordinates from where a mouse is clicked relate to the 2D camera and its zoom/scale factor. Without zoom/scale in consideration, it works, but when I apply a zoom/scale and try to do something like:

double worldX = (mX + worldCamera.getPosition().getX()) * worldCamera.getZoom();
double worldY = (mY + worldCamera.getPosition().getY()) * worldCamera.getZoom();

Or any other methods suggested that I found on StackerOverflow (from other people's questions) it does not work

When testing this, I would click on a 'Block' or game object in my 2D world and print out the mouse coordinates and the calculated world coordinates. However, when I apply a zoom factor to it and move the camera around, the calculated world coordinates are way off.

And just in-case anyone is curious, here is the 2D camera code I have so far:

package dgk.prototype.game;

import dgk.prototype.util.Vec2D;

import static org.lwjgl.glfw.GLFW.*;

public class Viewport {

    public static final int WIDTH = 800;
    public static final int HEIGHT = 600;

    /*
    The tween for the Viewport, if it is locked onto a moving target, or simply a GameObject.
     */
    public static final double TWEEN = .5D;

    private Vec2D position;

    private IEntity target = null;

    /*
    Zoom value between .1 and 1.0f. (1 to 100%)
     */
    private float zoom = 1.0f;

    public Viewport(double x, double y) {
        this.position = new Vec2D(x, y);
    }

    public Vec2D getPosition() {
        return this.position;
    }

    public boolean hasTarget() {
        return (target == null) ? false : true;
    }

    public IEntity getTarget() {
        return target;
    }

    public float getZoom() {
        return zoom;
    }

    public void setZoom(float zoom) {
        this.zoom = zoom;
    }

    public void onUpdate(GameWindow gameWindow) {
        if(hasTarget()) {
            return;
        }

        if(gameWindow.isKeyPressed(GLFW_KEY_A)) {
            getPosition().add(new Vec2D(-1, 0));
        }
        if(gameWindow.isKeyPressed(GLFW_KEY_D)) {
            getPosition().add(new Vec2D(1, 0));
        }
        if(gameWindow.isKeyPressed(GLFW_KEY_W)) {
            getPosition().add(new Vec2D(0, -1));
        }
        if(gameWindow.isKeyPressed(GLFW_KEY_S)) {
            getPosition().add(new Vec2D(0, 1));
        }

        if(gameWindow.isKeyPressed(GLFW_KEY_UP)) {
            zoom += .01f;

            if(zoom >= 1.5f) {
                zoom = 1.5f;
            }

            System.out.println(this);
        }
        if(gameWindow.isKeyPressed(GLFW_KEY_DOWN)) {
            zoom -= .01f;

            if(zoom <= .25f) {
                zoom = .25f;
                return;
            }

            System.out.println(this);
        }

        //System.out.println(this);
    }

    @Override
    public String toString() {
        return "Viewport - (x: " + getPosition().getX() + ", y: " + getPosition().getY() + ", zoom: " + (100 * zoom) +  "%, hasTarget: " + hasTarget() + ", target: " + getTarget() + ")";
    }
}

The specific code I use for the mouse position / world coordinates calculations:

GLFW.glfwSetMouseButtonCallback(handle, (window, button, action, mods) -> {
            if(button == GLFW_MOUSE_BUTTON_1 && action == GLFW_PRESS) {
                DoubleBuffer xBuffer = BufferUtils.createDoubleBuffer(1);
                DoubleBuffer yBuffer = BufferUtils.createDoubleBuffer(1);

                glfwGetCursorPos(handle, xBuffer, yBuffer);

                double mX = xBuffer.get(0);
                double mY = yBuffer.get(0);

                double worldX = (mX + worldCamera.getPosition().getX()) * worldCamera.getZoom();
                double worldY = (mY + worldCamera.getPosition().getY()) * worldCamera.getZoom();

                System.out.println("Mouse X: " + mX + ", Mouse Y: " + mY);
                System.out.println("World X: " + worldX + ", World Y: " + worldY);
                //System.out.println("World X: " + (800 - Math.abs((mX + worldCamera.getPosition().getX()) / worldCamera.getZoom()) - (800 - worldCamera.getPosition().getX())) + ", World Y: " + (600 - Math.abs((mY + worldCamera.getPosition().getY()) / worldCamera.getZoom()) - (600 - worldCamera.getPosition().getY())));
            }
        });

[KaiHH]

Research how the OpenGL transformation pipeline works exactly. Which steps are involved and what these steps are doing exactly on a mathematical level. Then research what glScalef is doing exactly. And you probably also have a glOrtho() somewhere in your code which you did not show, since otherwise a zoom factor of 1.0 would also not work. So research what glOrtho() does exactly. Then you will understand how you need to reverse this process to get from window coordinates to world coordinates.

Hint: Multiplying by your zoom factor to obtain the world coordinates is wrong. You need to multiply by the reciprocal.

[DGK]

Research how the OpenGL transformation pipeline works exactly. Which steps are involved and what these steps are doing exactly on a mathematical level. Then research what glScalef is doing exactly. And you probably also have a glOrtho() somewhere in your code which you did not show, since otherwise a zoom factor of 1.0 would also not work. So research what glOrtho() does exactly. Then you will understand how you need to reverse this process to get from window coordinates to world coordinates.

Hint: Multiplying by your zoom factor to obtain the world coordinates is wrong. You need to multiply by the reciprocal.

I know how the OpenGL pipeline works... it has to deal with the three matrices that are multiplied with each other in order to get the outcome and you have to do the inverse of that. I don't understand how your reply is remotely helping me, when I previously stated that I already looked at other people's solutions on this problem and it did not help. I am not specifically using all of LWJGL's tools to do camera work as I have my own camera class and I just scale all rendered objects with glScalef().

My previous method was to divide the mouse coordinates by the zoom factor. And yes, I do have glOrtho() in my code, which basically sets the orthographic view to 0,0, 800, 600 in the top left of the screen.

glOrtho(0f, 800, 600, 0f, 0f, 1f);

My attempt at the solution was this, which I saw being used by another user on a different website but it fails to work when I translate the camera around:

System.out.println("World X: " + (800 - Math.abs((mX + worldCamera.getPosition().getX()) / worldCamera.getZoom()) - (800 - worldCamera.getPosition().getX())) + ", World Y: " + (600 - Math.abs((mY + worldCamera.getPosition().getY()) / worldCamera.getZoom()) - (600 - worldCamera.getPosition().getY())));

800 being Screen width, 600 being Screen height

If you could gladly point me to the direction of some useful tips regarding this, I would appreciate it a lot.

[KaiHH]

Well then you should perfectly know how to invert the transformation.
Just do the math in matrix form and then transform that to a system of linear equations for which most of the terms will just be zero and cancel each other out.
This will give you EXACTLY the necessary transformation steps.

And since you just said you also tried dividing by the scale factor (and now you multiply by it), you are obviously not following any formula but "trial and error", and therefore I assumed that you in fact do not know how the transformation steps work. Therefore I requested that you research that.
And just saying "all other stackoverflow questions did not help" do not rule out this.

And just knowing that the pipeline means that "some matrices are being multiplied together" is certainly not enough.
You need to know what the matrices look like exactly. How does a glOrtho() matrix look like... How does a glScalef() matrix look like. What exactly does a glTranslatef() matrix do? This is IMPORTANT when you want to invert the operations.

[DGK]

I'll look into it. Still getting used to matrices.

[KaiHH]

If you maybe want to offload that work to another library and keep on doing more important stuff then you can use JOML and its Matrix3x2f for all those 2D transformations, like so:

// Create a camera matrix:
Matrix3x2f m = new Matrix3x2f()
  .view(0, viewportWidth, 0, viewportHeight) // <- just like glOrtho(..., -1, +1)
  .scale(zoomFactor) // <- just like glScalef(); greather than 1.0f means zooming in
  .translate(-camX, -camY); // <- just like glTranslatef(); translate by negative camera position

// Load matrix into OpenGL's matrix stack.
// We just use the GL_PROJECTION stack here:
glMatrixMode(GL_PROJECTION);
try (MemoryStack frame = MemoryStack.stackPush()) {
  // Call glLoadMatrixf with a column-major buffer holding the 4x4 matrix
  glLoadMatrixf(m.get4x4(frame.mallocFloat(16)));
}
// GL_MODELVIEW not used here, so set to identity:
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();

// Unproject a position in window coordinates to world coordinates:
Vector2f worldPos = m.unproject(mouseX, viewportHeight - mouseY,
    new int[] {0, 0, viewportWidth, viewportHeight}, new Vector2f());