[SOLVED] GLSL error with smoothstep

[pdid]

When I use the uniform function smoothstep in my GLSL shader code the JVM crashes with this error message:

An unrecoverable stack overflow has occurred.
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_STACK_OVERFLOW (0xc00000fd) at pc=0x57d612e7, pid=5176, tid=0x00001ba4
#
# JRE version: Java(TM) SE Runtime Environment (8.0_111-b14) (build 1.8.0_111-b14)
# Java VM: Java HotSpot(TM) Client VM (25.111-b14 mixed mode windows-x86 )
# Problematic frame:
# C  [ig7icd32.dll+0x3f12e7]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\Acer Computer\Desktop\Ian\Programming\Eclipse\Workspaces\Hones Game Engine\Test Game (Key Chase)\hs_err_pid5176.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

[abcdef]

I'm not sure what you expect anyone to do? Just posting an error is not useful at all.

Post your shader code, the version of opengl you are using, some sample values you are passing to smoothstep which cause the crash

[Kai]

Attaching the full content of this file:

  C:\Users\Acer Computer\Desktop\Ian\Programming\Eclipse\Workspaces\Hones Game Engine\Test Game (Key Chase)\hs_err_pid5176.log

would be nice.

[pdid]

Here is the shader code that I use to render text:
#version 330

in vec2 passTextureCoords;

out vec4 fragColour;

uniform vec3 colour;
uniform sampler2D fontAtlas;

const float width = 0.5;
const float edge = 0.1;

void main(){
   float distance = 1.0 - texture(fontAtlas, passTextureCoords).a;

   float alpha = smoothstep(width, edge + width, distance);
   fragColour = vec4(colour, texture(fontAtlas, passTextureCoords).a);
}

And the error file is one of the attachments.

[abcdef]

Your crash is happening during the linking phase of your shader compile, do you check for compile erorrs? This will hopefully tell you what is wrong.

This page has example code for the error checking process

https://www.khronos.org/opengl/wiki/Shader_Compilation

[pdid]

I do have a way to check for errors but the JVM crash happens during the glLinkProgram method, so the code where I check for an error is never reached.

[pdid]

And if it helps at all, below is the code I use in the text shader.


#version 330

in vec2 passTextureCoords;

out vec4 fragColour;

uniform vec3 colour;
uniform sampler2D fontAtlas;

const float width = 0.5;
const float edge = 0.1;

void main(){
   float distance = 1.0 - texture(fontAtlas, passTextureCoords).a;

   float alpha = smoothstep(width, edge + width, distance);
   fragColour = vec4(colour, texture(fontAtlas, passTextureCoords).a);
}

[Kai]

...but the JVM crash happens during the glLinkProgram method, so the code where I check for an error is never reached.

You apparently did not read the article linked by @abcdef.
Please DO read it and make sure you took all error checking measures described in it.

[pdid]

I'm sorry abcdef, I don't understand. I read the entire article, but the thing is the JVM crash happens somewhere inside the glLinkProgram method, and before I can call glGetProgrami(programID, GL_LINK_STATUS). Because of this I have no way of checking to see what the error is.

[abcdef]

pdid, you didn't read what I sent. I will be a bit more explicit this time.

The linking is crashing because you are trying to link uncompilable code. The additional error checking I was asking you to do was in the article I sent, I will post the relevant code here to make it easy for you.

glCompileShader(shader);

GLint isCompiled = 0;
glGetShaderiv(shader, GL_COMPILE_STATUS, &isCompiled);
if(isCompiled == GL_FALSE)
{
	GLint maxLength = 0;
	glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> errorLog(maxLength);
	glGetShaderInfoLog(shader, maxLength, &maxLength, &errorLog[0]);

	// Provide the infolog in whatever manor you deem best.
	// Exit with failure.
	glDeleteShader(shader); // Don't leak the shader.
	return;
}

The code is in c but very easily converted in to Java using LWJGL. What I would like to see is verification that you actually do this for all your shaders (vertex, fagment etc). And I would like to see any log messages where the compilation failed.

[pdid]

I did read the article quite thoroughly. There is never an error when I compile the shader, only when I link the program. Below is my code for compiling a shader:

                int shaderID = glCreateShader(type.openGLAddress);

		if (shaderID == 0) {
			throw new ShaderException("Shader creation failed.");
		}

		glShaderSource(shaderID, text);
		glCompileShader(shaderID);

		if (glGetShaderi(shaderID, GL_COMPILE_STATUS) == GL_FALSE) {
			throw new ShaderException("Shader compilation failed. Error Log:\n" + glGetShaderInfoLog(shaderID));
		}

		shaders.add(shaderID);
		return shaderID;

And here is the code for linking a shader program:

                glLinkProgram(programID);

		if (glGetProgrami(programID, GL_LINK_STATUS) == GL_FALSE) {
			throw new ShaderException("Failed to compile shader. Error Log:\n" + glGetProgramInfoLog(programID));
		}

And for good measure here is the shader code that causes the JVM crash.

#version 330

in vec2 passTextureCoords;

out vec4 fragColour;

uniform vec3 colour;
uniform sampler2D fontAtlas;

const float width = 0.5;
const float edge = 0.1; 

void main(){
   float distance = 1.0 - texture(fontAtlas, passTextureCoords).a;

   float alpha = smoothstep(width, edge + width, distance);
   fragColour = vec4(colour, texture(fontAtlas, passTextureCoords).a);
}

[pdid]

I'm just throwing this in here in case someone working on LWJGL3 happens to see it. I don't believe that an absence of error checking should cause an entire JVM crash. It should cause some exception to be thrown, but an entire JVM crash leads to the programmer being unaware of what has happened.

[spasi]

I'm just throwing this in here in case someone working on LWJGL3 happens to see it. I don't believe that an absence of error checking should cause an entire JVM crash. It should cause some exception to be thrown, but an entire JVM crash leads to the programmer being unaware of what has happened.

You must realize that the nature of LWJGL makes what you say impossible. In this particular case for example, it's a native stack overflow that happens deep inside the Intel OpenGL driver. No amount of error checking is going to stop the process from crashing. And anyway, the error checking would have to be what... a GLSL parser and validator inside LWJGL? This is outside the scope of LWJGL.

Also, you should be glad that there's plenty of information in the crash log. You know exactly where your code has crashed (glLinkProgram, called from the mcclean.engine.rendering.Shader.compile() method) and the EXCEPTION_STACK_OVERFLOW is a good hint that you're probably dealing with an OpenGL driver bug. Should you be complaining to someone, that someone must be the Intel driver engineers.

A couple of ideas on how to resolve this issue:

- Simplify the shader code as much as you can. When it stops crashing, add features back one by one until you find which one is problematic.
- Use a debug context and register a debug callback. See here for more information.

[pdid]

I'm sorry to bother everyone I thought it was a bug with the lwjgl code that caused this error, but now it appears to be a driver error. Thanks for your help everyone, I'll try other solutions from here on in.

[Deornoth]

I see that a resolution was not reached.

I had the same problem: linking my shader caused this same stack overflow, and it only occurred on older machines.  It seems to be relatively common, as it occurred independently on most of the laptops I tested it on.

The issue was with the smoothstep function.  I was able to workaround the issue by implementing my own smoothstep function:

float happy_smoothstep(float a, float b, float x) {
    float t = clamp((x - a) / (b - a), 0.0, 1.0);
 
    return t * t * (3.0 - (2.0 * t));
}