Parallax Occlusion Mapping issue

[Neoptolemus]

Hi everyone,

I've been trying to implement parallax occlusion mapping into my renderer, using a tutorial found here:

http://www.sunandblackcat.com/tipFullView.php?l=eng&topicid=28

I have tried each of the variations of parallax mapping (simple, steep, relief, POM) but each time the results I get come out distorted. For now I am just looking to achieve correct displacement, so I am skipping the self-shadowing section and you will see some inefficiencies in my GLSL (such as calculating the normal matrix in the shader). I'll obviously go back and tidy up the code once I'm happy it works. This is currently what my results look like:



The textures also appear to shift and slide around as I move the camera (but not if I change the viewing angle), which suggests an issue with the relative position of the camera.

I've tried to copy the tutorial exactly to reduce the risk of the issue arising from where I've deviated from the code, so it is almost entirely a copy/paste job for now:


VERTEX:

#version 440 core

layout(location = 0) in vec3 vPosition;
layout(location = 1) in vec3 vNormal;
layout(location = 2) in vec2 vTextureCoordinates;
layout(location = 3) in vec3 vTangent;
layout(location = 4) in vec3 vBinormal;

layout(location = 0) out vec4 fPosition;
layout(location = 1) out vec3 fNormal;
layout(location = 2) out vec2 fTextureCoordinates;
layout(location = 3) out vec3 fTangent;

uniform mat4 pMatrix;
uniform mat4 vMatrix;
uniform mat4 mMatrix;

uniform vec3 cameraPosition;

out vec3 cameraInTangentSpace;

void main(void) {
	
	mat4 nMatrix = transpose(inverse(mMatrix));
	vec4 worldPosition = (mMatrix * vec4(vPosition,1.0));
	vec3 worldNormal = normalize((nMatrix * vec4(vNormal,0.0)).xyz);
	vec3 worldTangent = normalize((nMatrix * vec4(vTangent,0.0)).xyz);
	
	fTextureCoordinates = vTextureCoordinates;
	fPosition = worldPosition;
	fNormal = worldNormal;
	fTangent = worldTangent;

	vec3 worldBitangent = normalize(cross(worldTangent, worldNormal));
	
	vec3 worldDirectionToCamera	= normalize(cameraPosition - worldPosition.xyz);

   cameraInTangentSpace = vec3(
         dot(worldDirectionToCamera, worldTangent),
         dot(worldDirectionToCamera, worldBitangent),
         dot(worldDirectionToCamera, worldNormal)
      );
	
	mat4 mvpMatrix = pMatrix * vMatrix * mMatrix;
	gl_Position = mvpMatrix * vec4(vPosition,1.0);
}

FRAGMENT:

#version 440 core

layout(location = 0) in vec4 fPosition;
layout(location = 1) in vec3 fNormal;
layout(location = 2) in vec2 fTextureCoordinates;
layout(location = 3) in vec3 fTangent;

layout(location = 0) out vec4 cPosition;
layout(location = 1) out vec4 cDiffuse;
layout(location = 2) out vec4 cNormal;
layout(location = 3) out vec4 cSpecular;

uniform sampler2D texture_diffuse;
uniform sampler2D texture_normal;
uniform sampler2D texture_specular;
uniform sampler2D texture_height;
uniform float parallaxScale;

in vec3 cameraInTangentSpace;



vec4 calcBumpedNormal() {
	vec3 Normal = normalize(fNormal);
	vec3 Tangent = normalize(fTangent);
	Tangent = normalize(Tangent - dot(Tangent, Normal) * Normal);
	vec3 Bitangent = cross(Tangent, Normal);
	vec3 BumpMapNormal = texture(texture_normal, fTextureCoordinates).xyz;
	BumpMapNormal = 2.0 * BumpMapNormal - vec3(1.0, 1.0, 1.0);
	mat3 TBN = mat3(Tangent, Bitangent, Normal);
	vec3 NewNormal = TBN * BumpMapNormal;
	NewNormal = normalize(NewNormal);
	return vec4(NewNormal,0);
}

vec2 parallaxMapping(in vec3 V, in vec2 T, out float parallaxHeight) {

   // determine optimal number of layers
   const float minLayers = 10;
   const float maxLayers = 15;
   float numLayers = mix(maxLayers, minLayers, abs(dot(vec3(0, 0, 1), V)));

   // height of each layer
   float layerHeight = 1.0 / numLayers;
   // current depth of the layer
   float curLayerHeight = 0;
   // shift of texture coordinates for each layer
   vec2 dtex = parallaxScale * V.xy / V.z / numLayers;

   // current texture coordinates
   vec2 currentTextureCoords = T;

   // depth from heightmap
   float heightFromTexture = texture(texture_height, currentTextureCoords).r;

   // while point is above the surface
   while(heightFromTexture > curLayerHeight) 
   {
      // to the next layer
      curLayerHeight += layerHeight; 
      // shift of texture coordinates
      currentTextureCoords -= dtex;
      // new depth from heightmap
      heightFromTexture = texture(texture_height, currentTextureCoords).r;
   }

   ///////////////////////////////////////////////////////////

   
   // previous texture coordinates
   vec2 prevTCoords = currentTextureCoords + dtex;

   // heights for linear interpolation
   float nextH	= heightFromTexture - curLayerHeight;
   float prevH	= texture(texture_height, prevTCoords).r
                           - curLayerHeight + layerHeight;

   // proportions for linear interpolation
   float weight = nextH / (nextH - prevH);

   // interpolation of texture coordinates
   vec2 finalTexCoords = prevTCoords * weight + currentTextureCoords * (1.0-weight);

   // interpolation of depth values
   parallaxHeight = curLayerHeight + prevH * weight + nextH * (1.0 - weight);

   // return result
   return finalTexCoords;

}

void main(void) {

    cPosition = fPosition;
	cNormal = calcBumpedNormal();
	cSpecular = texture(texture_specular, fTextureCoordinates);

	float parallaxHeight;
	vec3 V = normalize(cameraInTangentSpace);
	vec2 pTextureCoordinates = parallaxMapping(V, fTextureCoordinates, parallaxHeight);
	
	cDiffuse = texture(texture_diffuse, pTextureCoordinates);
}

Sorry for the code dump, but since I have no idea where I've gone wrong, I don't want to leave anything out. My normal mapping and specular highlights work perfectly, so I'm pretty sure it's not an issue with my normal matrix, tangents or normals, but I'm happy to consider any possibility. For the record, the default value for parallaxScale is 0.1 (as suggested in the tutorial), but modifying it doesn't help (other than to remove the parallax effect entirely once it reaches 0).

Thanks!

[Cornix]

Can you not ask the author of the article if it may be possible that there is an error on his/her side?
If you copied the tutorial 1:1 and your graphics card drivers are updated to a stable version then perhaps the tutorial is just wrong.

[Neoptolemus]

I hadn't even considered the possibility that the tutorial itself would be wrong! I'll get in touch and maybe try another tutorial as well. Unfortunately my mathematical understanding is weak at the moment so it's difficult for me to independently verify if the algorithm is correct.

In the meantime, if anyone does spot the problem then do let me know

[Neoptolemus]

Ok I tried a completely different tutorial which uses a very different method, available here:

http://www.gamedev.net/page/resources/_/technical/graphics-programming-and-theory/a-closer-look-at-parallax-occlusion-mapping-r3262

I downloaded the resources and literally copy and pasted directly from the HLSL shader, and obviously translated it into GLSL. I came up with this as my POM code:

VERTEX:

#version 440 core

layout(location = 0) in vec3 vPosition;
layout(location = 1) in vec3 vNormal;
layout(location = 2) in vec2 vTextureCoordinates;
layout(location = 3) in vec3 vTangent;
layout(location = 4) in vec3 vBinormal;

layout(location = 0) out vec4 fPosition;
layout(location = 1) out vec3 fNormal;
layout(location = 2) out vec2 fTextureCoordinates;
layout(location = 3) out vec3 fTangent;

uniform mat4 pMatrix;
uniform mat4 vMatrix;
uniform mat4 mMatrix;

uniform vec3 cameraPosition;

out vec3 pEye;
out vec3 pNormal;

void main(void) {

	mat4 nMatrix = transpose(inverse(mMatrix));

	vec4 worldPosition = (mMatrix * vec4(vPosition,1.0));

	fTextureCoordinates = vTextureCoordinates;
	fPosition = worldPosition;
	fNormal = normalize((nMatrix * vec4(vNormal,0.0)).xyz);
	fTangent = normalize((nMatrix * vec4(vTangent,0.0)).xyz);

	vec3 Bitangent = normalize(cross(vTangent, vNormal));

	vec3 pNorm = (mMatrix * normalize(vec4(vNormal, 0.0))).xyz;
	vec3 pTan = (mMatrix * normalize(vec4(vTangent, 0.0))).xyz;
	vec3 pBTan = (mMatrix * normalize(vec4(Bitangent, 0.0))).xyz;
	
	mat3 worldToTangentSpace = mat3(pTan, pBTan, pNorm);
	
	vec3 vertexToEye = worldPosition.xyz - cameraPosition;
	pEye = vertexToEye * worldToTangentSpace;
	pNormal = vNormal * worldToTangentSpace;
	
	mat4 mvpMatrix = pMatrix * vMatrix * mMatrix;
	gl_Position = mvpMatrix * vec4(vPosition,1.0);
}

FRAGMENT (Parallax code only as no other processing is done on these elements)

vec2 parallaxMapping() {

	float fParallaxLimit = -length( pEye.xy ) / pEye.z;

	fParallaxLimit = fParallaxLimit * parallaxScale;
	
	vec2 vOffsetDir = normalize( pEye.xy );
	vec2 vMaxOffset = vOffsetDir * fParallaxLimit;
	
	vec3 N = normalize( pNormal );
	vec3 E = normalize( pEye );
	
	int nNumSamples =  int( mix( 20, 4, dot( E, N ) ) );
	
	float fStepSize = 1.0 / float(nNumSamples);
	
	vec2 dx = dFdx( fTextureCoordinates );
	vec2 dy = dFdy( fTextureCoordinates );
	
	float fCurrRayHeight = 1.0;	
	vec2 vCurrOffset = vec2( 0, 0 );
	vec2 vLastOffset = vec2( 0, 0 );
	
	float fLastSampledHeight = 1;
	float fCurrSampledHeight = 1;

	int nCurrSample = 0;
	
	while ( nCurrSample < nNumSamples )
	{
		// Sample the heightmap at the current texcoord offset.  The heightmap 
		// is stored in the alpha channel of the height/normal map.
		//fCurrSampledHeight = tex2Dgrad( NH_Sampler, IN.texcoord + vCurrOffset, dx, dy ).a;
		vec2 currTexCoords = fTextureCoordinates + vCurrOffset;
		
		fCurrSampledHeight = textureGrad( texture_height, currTexCoords, dx, dy ).r;

		// Test if the view ray has intersected the surface.
		if ( fCurrSampledHeight > fCurrRayHeight )
		{
			// Find the relative height delta before and after the intersection.
			// This provides a measure of how close the intersection is to 
			// the final sample location.
			float delta1 = fCurrSampledHeight - fCurrRayHeight;
			float delta2 = ( fCurrRayHeight + fStepSize ) - fLastSampledHeight;
			float ratio = delta1/(delta1+delta2);

			// Interpolate between the final two segments to 
			// find the true intersection point offset.
			vCurrOffset = (ratio) * vLastOffset + (1.0-ratio) * vCurrOffset;
			
			// Force the exit of the while loop
			nCurrSample = nNumSamples + 1;	
		}
		else
		{
			// The intersection was not found.  Now set up the loop for the next
			// iteration by incrementing the sample count,
			nCurrSample++;

			// take the next view ray height step,
			fCurrRayHeight -= fStepSize;
			
			// save the current texture coordinate offset and increment
			// to the next sample location, 
			vLastOffset = vCurrOffset;
			vCurrOffset += fStepSize * vMaxOffset;

			// and finally save the current heightmap height.
			fLastSampledHeight = fCurrSampledHeight;
		}
	}
	
	vec2 finalCoords = fTextureCoordinates + vCurrOffset;
	return finalCoords;
}

The only change I made is on the textureGrad line, I sample the texture's red channel rather than alpha (as I am using a more conventional black and white map rather than using the normal map's alpha channel as he is). I still get the same result, albeit somewhat improved. It still distorts while I'm moving around though and it still looks all wobbly and ugly.

Here are the textures I'm using, they're from crytek's sponza model:



I've resized it for the purposes of showing it on this forum, but the original texture is untouched.

If anyone can help me at all, I would be really grateful as this is extremely frustrating.

[Cornix]

Okay.
You used 2 different sources with 2 different implementations but you get the same kind of erronous behavior. The chances for that being the fault of the sources is very slim.
I would personally suggest the problem is either:
1) The way you initialize the OpenGL context
2) Your hardware / driver
3) God hates you

I would first suggest trying the same code on a different machine with a different graphics card. If that miraculously works, try to test on your own hardware with a different driver version.
If it isnt working on a different machine try to play around with the way you initialize your context. Do some internet research for any similar problems other people might have had in the past.

[Neoptolemus]

Thanks Cornix,

I'll discard the suggestion that God hates me just for now, until I've explored other options I'm running a laptop with a Nvidia NVS 4200M with the latest drivers (downloaded 2 days ago from the Nvidia site, as my POM code was previously crashing the display driver). I will try my code on another machine with a "proper" video card in when I have a chance and see if that helps.

As far as context initialisation goes, do you have any pointers on what might cause the POM code to produce incorrect results, but allow normal mapping and specular highlighting to work fine? I'll do some investigation now, but if you have any idea on a good place to start that would be great

[Cornix]

As far as context initialisation goes, do you have any pointers on what might cause the POM code to produce incorrect results, but allow normal mapping and specular highlighting to work fine? I'll do some investigation now, but if you have any idea on a good place to start that would be great

I am sorry, I have no idea. I have never tried to implement POM myself, I am just trying to help you with general purpose debugging help. It is what I would personally do, but I dont have any experience in this regard. If there is nobody here on this forum you should definitely try to post on the semi-official OpenGL support forums, there are very experienced and very intelligent people there:
https://www.opengl.org/discussion_boards/forumdisplay.php/6-OpenGL-coding-beginners

[Neoptolemus]

Thanks for your efforts Cornix, I'll do some more investigation for a few days then post on the OpenGL forums if I still cannot resolve the issue.

[abcdef]

Does your laptop also have an integrated graphics card?

If you run glxinfo what renderer does it say is being used and is direct rendering set to yes?

[Neoptolemus]

I believe glxinfo is Linux only? I'm running Windows 7. Is there a way through LWJGL to determine which adapter is being used? I couldn't find an equivalent to glfwGetWin32Adapter in the LWJGL GLFW package. I'm using the latest version of LWJGL (I just downloaded the latest version from the site now to see if anything had been added).