Stuck with Shadow Mapping

[Neoptolemus]

Hi everyone,

I've been trying to implement a simple shadow mapping example with a directional light, but have so far been unable to get it working even though I'm pretty sure I followed the instructions correctly.

This is the tutorial I am trying to follow: http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-16-shadow-mapping/

I am using a deferred renderer, so initially I have a pass which generates textures for the world position, normal, specular and diffuse, and the directional light is rendered as a full screen quad which samples from the geometry buffer. I also have an initial pass which updates the shadow map, which looks like this:



I then run the lighting pass. For my directional light, I define the bias matrix as so:

biasMatrix = new Matrix4f(0.5f, 0.0f, 0.0f, 0.5f, 0.0f, 0.5f, 0.0f, 0.5f, 0.0f, 0.0f, 0.5f, 0.5f, 0.0f, 0.0f, 0.0f, 1.0f);

To try and keep things as simple as possible, I am doing most of the work in the shaders, though once I have it working I will of course move things back onto the CPU where appropriate.

VERTEX

#version 440

layout(location = 0) in vec3 Position;

uniform mat4 pMatrix;
uniform mat4 vMatrix;
uniform mat4 mMatrix;
uniform mat4 biasMatrix;

out mat4 depthBiasMatrix;

void main()
{    
	mat4 depthMatrix = pMatrix * vMatrix * mMatrix;
	depthBiasMatrix = biasMatrix * depthMatrix;
	
    gl_Position = vec4(Position, 1.0);
}

FRAGMENT

#version 440

uniform vec2 gScreenSize;
uniform sampler2D gColorMap;
uniform sampler2D gNormalMap;
uniform sampler2D gShadowMap;
uniform sampler2D gPositionMap;

uniform vec3 lightColor;
uniform vec3 lightAngle;
uniform float lightBrightness;

in mat4 depthBiasMatrix;

out vec4 out_Color;

vec2 CalcTexCoord()
{
    return gl_FragCoord.xy / gScreenSize;
}

void main()
{   
	vec2 texCoords = CalcTexCoord();
	vec4 diffuse = texture(gColorMap, texCoords);
	
	vec3 fPosition = texture(gPositionMap, texCoords).xyz;
	vec4 ShadowCoord = depthBiasMatrix * vec4(fPosition,1.0);
	
	vec3 vNormal = normalize(texture(gNormalMap, texCoords).xyz);
	
	float brightness = max(dot(vNormal, lightAngle), 0);
	
	vec3 DiffuseColor = lightColor * brightness * lightBrightness;
	
	float visibility = 1.0;
	if ( texture( gShadowMap, ShadowCoord.xy ).z  <  ShadowCoord.z){
		visibility = 0.5;
	}

    out_Color = diffuse * vec4(DiffuseColor,1.0) * visibility;
	
}

As you can see, I first construct the depthBiasMatrix by using the same projection matrix, view matrix and model matrix I used to create the shadow map, and my bias matrix. I then pass this over to the fragment shader.

In the fragment shader I sample the position map to get the world position, multiply it by my depthBiasMatrix to create ShadowCoord, and then do a straight copy/paste of the tutorial to determine if that fragment is in shadow or not.

My problem is that it seems to always fail the condition, so everything is always fully visible. The directional lighting itself is correct (i.e. walls facing away from the light direction are dark, and the normal mapping works fine), but no shadows are cast.

I have no idea what I have done wrong here, I am certain my position map is correct as I populate it in the GBuffer as being modelMatrix * vertexPosition, i.e. its world position rather than local position. I also know my mvp matrix is correct as it is the same formula I use everywhere else.

Any pointers would be much appreciated!

[Kai]

It looks like you are missing out on the perspective-divide to take your ShadowCoord.z from clip-space into NDC-space, so that you can actually compare it with the depth value from the depth texture (which is in non-linear NDC-space, and simply interval-mapped from [-1..+1] to [0..1]). Have a look at OpenGL transformations
Currently you are comparing Apples with Oranges (i.e. clip-space ShadowCoord.z with NDC-space depth), so to say
Additionally, if your positions are already in world-space, you should not transform them again by the model matrix, which also kinda makes no sense with deferred shading, since you are already in screen space.

[Neoptolemus]

It looks like you are missing out on the perspective-divide to take your ShadowCoord.z from clip-space into NDC-space, so that you can actually compare it with the depth value from the depth texture (which is in non-linear NDC-space, and simply interval-mapped from [-1..+1] to [0..1]). Have a look at OpenGL transformations
Currently you are comparing Apples with Oranges (i.e. clip-space ShadowCoord.z with NDC-space depth), so to say
Additionally, if your positions are already in world-space, you should not transform them again by the model matrix, which also kinda makes no sense with deferred shading, since you are already in screen space.

Thanks Kai. I've modified the code for defining the depthBiasMatrix as so:

mat4 depthMatrix = pMatrix * vMatrix;
	depthBiasMatrix = biasMatrix * depthMatrix;

As you can see, I've remove the model matrix multiplication. That was an oversight on my part :p

As far as the other issue goes, how would I go about transforming ShadowCoord.z to NDC coordinates? I didn't see anything in the tutorial mentioning this, so I can only assume they achieve this through some other means. I assumed this was what the bias matrix was achieving, but apparently not...

[Kai]

The bias matrix is not used to transform into "Normalized Device Coordinates" (NDC) space.
It is rather used to map your x,y coordinates to [0..1], which I like to call "texture space" (but it does not have an official name, though).

The problem that the bias matrix is trying to solve is actually two-fold:
- Like I said, convert your x,y coordinates to [0..1] in order to be able to sample the depth texture, since texture coordinates in OpenGL are in [0..1]
- The second thing is the most tricky part: The bias matrix is also used to map your z coordinate to [0..1], because the sampler2D gives you texel values in [0..1] and only then can you compare both depth values with each other.

The interesting part now is that the bias matrix is being applied BEFORE perspective divide (dividing by the 'w' coordinate), which is kind of non-intuitive, but necessary since clip coordinates are not normalized to [-1..+1] but only so after perspective divide.
So without this perspective divide, your clip-space coordinates may range wildly in (-inf..+inf).
It is only after this perspective divide that x, y and z are within [-1..+1]. But you need them to be in [0..1] and that's why we use the bias matrix.

So there is some tricky matrix multiplication going on, which is explained in here:
https://www.opengl.org/discussion_boards/showthread.php/173890-Shadow-mapping-bias-before-the-w-divide-!

I also had a look at some other tutorials and they all do the perspective divide (look at their fragment shaders):

http://www.codinglabs.net/tutorial_opengl_deferred_rendering_shadow_mapping.aspx

http://www.fabiensanglard.net/shadowmapping/index.php

http://www.3dcpptutorials.sk/index.php?id=11

I am about to say that the fragment shader text in your mentioned tutorial is wrong, since it absolutely HAS to do perspective divide.

EDIT: If I find some time, I am going to write a tutorial in the LWJGL wiki trying to explain all of this in more depth, since I find that none of the tutorials out there REALLY really explains every single bit and detail and math of it. And also they do not get into using the predefined sampler2DShadow, which you can use to let OpenGL do the perspective divide and also the depth comparison.

[Neoptolemus]

Thanks again Kai, I've looked through those tutorials. I do like Fabien Sanglard actually, his dissections of older game engines are always really good reads. I find some of the tutorials a little confusing as they often use deprecated code which makes things muddy for someone still getting to grips with OpenGL like I am.

You're absolutely right though, I can see now that they're all doing the perspective divide. I chose that particular tutorial as I found it the easiest to follow.

I have done the perspective divide in my code now, so the fragment shader looks like this:

vec4 ShadowCoord = depthBiasMatrix * vec4(fPosition,1.0);
	vec3 finalShadowCoord = ShadowCoord.xyz / ShadowCoord.w;
	finalShadowCoord.z += 0.0005;
	
	float visibility = 1.0;
	if ( texture( gShadowMap, finalShadowCoord.xy ).z  <  finalShadowCoord.z){
		visibility = 0.0;
	}

Where of course the depthBiasMatrix is the projection * view matrix, which is then multiplied by the bias, and fPosition is the world position of the fragment. I'm still getting the same issue (everything is always visible) however.

The tutorial I'm following does appear to do a perspective divide, however it's present in the source code rather than the tutorial (maybe they missed it out by accident during the write-up?). You can see it here:

https://code.google.com/p/opengl-tutorial-org/source/browse/tutorial16_shadowmaps/ShadowMapping_SimpleVersion.fragmentshader

However, even converting gShadowMap to a sampler2DShadow and using the method described here still doesn't produce correct results.

[Kai]

Hey Neoptolemus,

I just hacked down a simple shadow mapping demo from scratch. Works splendidly.
It uses a single directional cone/splot light and the bias matrix with perspective divide.
Additionally, to combat shadow acne it uses a small depth offset.

Will add some comments and push it to the LWJGL repo in a couple of minutes.

[Neoptolemus]

One day, I too will be able to hack together a shadow mapping example in a few minutes. I've been banging my head against this particular wall for around a week now (spread over a month).

[Kai]

It's up.
See this commit.
Hang into it, you can do it!

EDIT: Just one thing about that perspective divide, I just noticed: Your mentioned tutorial did not do perspective divide because it did not need to. It was using an orthogonal projection... and with those 'w' is 1.0 anyways I guess (but am not completely sure), because there is no "perspective". The actual fragment shader does include that perspective divide now, because there is the option of switching between orthogonal light and cone/spot light now. The latter is a perspective projection and hence would need that divide.
So the problem with your code could lie somewhere else.

[Neoptolemus]

Thank you for all your efforts Kai. I had a quick look at your shaders and I don't see what you're doing that's any different from me other than your calculation of the bias light matrix. You do it as bias * mvp * position, but I have to do it as bias * vp * position since my vertex positions are already in world space. Could this be an issue? Perhaps I'm not doing the multiplication correctly?

I'm going to guess for now that it has to be a mistake elsewhere in the code, on the CPU side. I'll take another look when I can and hopefully crack this once and for all.