How to decide rendering order of sprites?

[Sylph]

I'd need some help with my current project which has to do with the rendering order of my sprites.

I noticed most of the 2d games are able to decide the rendering order of their sprites (as in, having a player drawn over another player depending on it's position on the screen) I will show you an example of what I mean:

In this image I have set up two quads. Player 1 is rendered in front of player 2 since player 1 would be closer to the screen in a sense. It uses the same logic as old school games such as displayed in this example:


The whole logic behind it is as follows, the more an entity's "Y" position increases, the more it is rendered in front of other entities. Basically, the "Y" position of entities represent their rendering priority. In the second example, our little link is drawn in front of the tree even though he is colliding with it.

Now here is where I need your help since I am at a loss. Currently, my algorithm is the following: I calculate a zBuffer variable of type float by using the entity's "Y" position as follows:

zBuffer = ( ( ( position.y / Display.getHeight() ) * 2 ) - 1 ) / 2;

This basically give me a value of range [-0.5, 0.5] used during the rendering of the quads which works perfectly when rendering basic quads. Here is how the variable is used:

glColor3f( 1.0f, 1.0f, 1.0f );

glBegin( GL_QUADS );
    glVertex3f( position.x, position.y, zBuffer );
    glVertex3f( position.x + size.x, position.y, zBuffer );
    glVertex3f( position.x + size.x, position.y + size.y, zBuffer );
    glVertex3f( position.x, position.y + size.y, zBuffer );
glEnd();

So, if player 1 steps in front of player 2 he will be rendered on-top of player 2 and vice versa.

The problem I am facing occurs as soon as I bind a texture to the quad or in other words, a sprite. The order is still taken into account and the quads still render over each other depending on their Z priority. The texture's alpha though seems to be screwing up. Here is what I mean by that:


On the left, player 1 is rendered over player 2. On the right, player 2 is rendered over player 1. I don't fully understand the reason why it is acting this way since it works with regular quads and works when player 1 is in front of player 2. Here is how I draw the quads with the texture (I am simply using the TextureLoader to load the textures.)

glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST );

glColor3f( 1.0f, 1.0f, 1.0f );
			
playerSprite.bind();
	
glBegin( GL_QUADS );
	glTexCoord2f( 0.0f, 0.0f );
	glVertex3f( position.x, position.y, zBuffer );
	glTexCoord2f( 1.0f, 0.0f );
	glVertex3f( position.x + size.x, position.y, zBuffer );
	glTexCoord2f( 1.0f, 1.0f );
	glVertex3f( position.x + size.x, position.y + size.y, zBuffer );	
	glTexCoord2f( 0.0f, 1.0f );
	glVertex3f( position.x, position.y + size.y, zBuffer );
glEnd();

I would like to have you input on this. Much appreciated.

Here is some more of my code in case you need to see:

void setupOpenGL() {

		glMatrixMode( GL_PROJECTION );
		glLoadIdentity();
		
		glOrtho( 0, Display.getWidth(), Display.getHeight(), 0, -1, 1 );
		
		glMatrixMode( GL_MODELVIEW );
		glLoadIdentity();
		
		glEnable( GL_TEXTURE_2D );
		glEnable( GL_BLEND );
		glEnable( GL_DEPTH_TEST );
		glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
		
	}

[quew8]

So your problem is to do with the alpha channels of the textures and not the order?
I suspect that your issue actually has nothing to do with the values in your depth buffer (which seem to be correct) and it simply arises because you are drawing player 2 after player 1. When player 2 is drawn behind, any overlap is ignored because the depth test fails in for those overlapping pixels, but if player 2 is on top, there is nothing to stop overlap. You could test this by swapping the order of rendering (player 2 first) and the effect should happen for player 1 ontop instead.
I would suggest that you try changing the alpha test using:
    glEnable(GL_ALPHA_TEST);
and then
    glAlphaFunc(GL_NOT_EQUAL, 0);
This assumes your textures actually have alpha channels? If not you could add them in with a program like Paint.net or Gimp (or Photoshop if you want to be fancy). I think this may be what your trying to do with glBlendFunc but that isn't the way to do it. Here's the specification for glAlphaFunc: http://www.opengl.org/sdk/docs/man2/xhtml/glAlphaFunc.xml

[Sylph]

You sir just saved the day, I had ALPHA_TEST enabled before but it never changed anything, but the line you added about the glAlphaFunc saved the day.

Thank you so much. Also, just one little fix, it is GL_NOTEQUAL rather than GL_NOT_EQUAL.
Thank you again.

[quew8]

Happy to help, and sorry about the incorrect enum, I hadn't realized that it was different in lwjgl to the specification. (I was getting it straight from the linked page)