I've recently started working on a game engine for java using lwjgl. I created vector and matrix classes that I can multiply. I want to check if the mouse is within a certain range in world space even though glfw gives me the pixel it's on. The problem is that opengl uses 4 dimensions and the mouse is 2 dimensions. I know that the fourth dimension represents perspective but I don't know how I should multiply this.
How does the modelview projection matrix store information?
What you want is called "unprojecting". You want to unproject window coordinates to world coordinates.
Google for "OpenGL unproject", and you'll find a lot of resources explaining this.