Shader compilation bug with Intel 965 GM chipset.

[Gilbert Brisset]

We are developping a Java based GUI that renders a video stream.
We use OpenGL to process YUV planar to RGB color space conversion, image enhencements
and zooming.
The application used the Eclipse RCP framework, the LWJGL OpenGL wrapper for Java
and run on the Fedora 10 32 bits Linux platform.
The YUV planar to RGB conversion is performed with the following shader applied to the three
Y, U, V textures that are loaded in the GPU at the video frame rate :

/*******/
uniform sampler2DRect Ytex;
uniform sampler2DRect Utex;
uniform sampler2DRect Vtex;
uniform float         XFormat ; // 2 if 4:2:x format, 1 if 4:4:4
uniform float         YFormat ; // 2 if 4:2:0 format, 1 if 4:4:4 or 4:2:2

void main(void)
{
float r, g, b, y, u, v ;

y  = texture2DRect(Ytex,vec2(gl_TexCoord[0].x,           gl_TexCoord[0].y)).r;
u  = texture2DRect(Utex,vec2(gl_TexCoord[0].x / XFormat, gl_TexCoord[0].y / YFormat)).r;
v  = texture2DRect(Vtex,vec2(gl_TexCoord[0].x / XFormat, gl_TexCoord[0].y / YFormat)).r;

y  = 1.1643 * (y - 0.0625) ;
u  = u - 0.5 ;
v  = v - 0.5 ;

r  = y + 1.5958  * v ;
g  = y - 0.39173 * u - 0.81290 * v ;
b  = y + 2.017   * u ;

gl_FragColor = vec4(r,g,b,1.0) ;
}
/******/

This application has been tested and runs OK on different hardware platforms,
industrial PC, desktop PCs, laptop PCs, but all with Nvidia GeForce or Quadro GPUs and
all with the closed source Nvidia driver either loaded from the Fedora repositories or
directly from the Nvidia download website.

We plan to run this application on embedded low power industrial Compact PCI platform.
All these platforms come with an Intel chipset. We are trying now with the Intel 965 GM chipset.

OpenGL is fully allowed for Intel chipsets since the last (june 2009) Mesa 7.5 driver.

The application has been tested on a Lenovo R61 laptop and a Men F18 6U Compact PCI board
both with a 965 GM chipset.
In both cases the colors are rendered almost pink.

Investigations demonstrate that it is as if the GLSL compiler casts floating coefficients to ints,
and replaces 1.1643 by 1, 2.017 by 2 and so on.

Moreover the following shader that it is expected to render a grey color renders black :

void main(void)
{
float r, g, b ;
r = 0.5 ; g = 0.5 ; b = 0.5 ;
gl_FragColor = vec4(r, g, b, 1.0) ;
}

But the following shader renders grey :

void main(void)
{
float r, g, b ;
r = 500.0 / 1000.0 ; g = 500.0 / 1000.0 ; b = 500.0 / 1000.0 ;
gl_FragColor = vec4(r, g, b, 1.0) ;
}

So we have now a work around for the above bug : just replaces coefficients in the following way :
Replace 1.1643 by 1164.3/1000.0 and so on and the colors are perfect.

Someone could say that it is a Mesa problem and not a LWJGL problem.

BUT, the same shader has absolutely no problem with a similar application written in C
with the GLUT OpenGL wrapper or directly written with X Window and GLX APIs.

Moreover, the same bugs appears when using the gtlglext or gtkglarea OpenGL wrappers for GTK.

This proves that the Intel chipset run OK, the Mesa 7.5 driver runs OK, some wrappers run OK, some other run NOK.
And, I repeat, never problem with Nvidia chipset and driver.

I have posted Bug reports since few weeks on Mesa Bugzilla and gtkglext mailing list with absolute no answers.

Hope that LWJGL gurus could handle this problem.

Thank you.

[spasi]

It does sound like a driver issue. There's no interaction between LWJGL and the driver/GPU other than passing the shader text for compilation (which either fails and you get no rendering output or it succeeds and it's up to the driver after that), for the "grey" shaders you posted at least.

Long shot, but the following might have side-effects on the results you're getting:

- Do you use the same shader entry points with all wrappers? That is, do you use ARB_shader_objects/ARB_fragment_shader or GL20 and is it the same in all cases?
- Do you use the same context initialization with all wrappers? Have you checked if the GL version returned is the same in all cases?
- Do you null-terminate the shader texts?

[Gilbert Brisset]

Thank you for interesting to my issue.

I agree that it sounds like a driver issue, not a wrapper issue.
But it works :
OK :
- with Nvidia driver and all wrappers
- Intel / Mesa driver with GLUT and GLX
NOK :
- with Intel /Mesa driver, LWJGL, gtkglext and gtkglarea wrappers.

There is some side effect in the way the shader text is passed to the driver.
But the strange part is that we could assume that the way the wrapper pass the text to the shader is independent of the driver.
So it is a very weird and vicious side effect.

I am going to check all what you request, it requires some delay.

Just : what is GL20 ?

[spasi]

Just : what is GL20 ?

The entry points defined by the OpenGL 2.0 spec. If your driver supports OpenGL version 2.0 or higher, then you can use e.g. glShaderSource instead of glShaderSourceARB. GL20 is just the name of the LWJGL class that exposes the 2.0 functionality. Anyway, it should behave exactly the same way in a driver that works correctly, as I said it's a long shot.

[Gilbert Brisset]

Answer to the requested questions :

- Do you use the same shader entry points with all wrappers? That is, do you use ARB_shader_objects/ARB_fragment_shader or GL20 and is it the same in all cases?

It is ARB entry points in the Java code and GL20 in C code.

Here is our Java shader class :

public AbstractShader(String programCode_p) {

      ByteBuffer programByteBuffer = Utils.buildNullTerminatedByteBuffer(programCode_p);
      int fragmentShader = ARBShaderObjects.glCreateShaderObjectARB(ARBFragmentShader.GL_FRAGMENT_SHADER_ARB);
      ARBShaderObjects.glShaderSourceARB(fragmentShader, programByteBuffer);
      ARBShaderObjects.glCompileShaderARB(fragmentShader);

      _programId = ARBShaderObjects.glCreateProgramObjectARB();
      ARBShaderObjects.glAttachObjectARB(_programId, fragmentShader);

      ARBShaderObjects.glLinkProgramARB(_programId);
   }

I have a C code demo that uses the same code for the 4 wrappers GLUT, GLX, gtkglarea and gtkglext.
The only differences in code between the 4 wrappers is the way a drawing area is bound to an OpenGl context,
the primitive used to make the OpenGL contetx current and to swap buffers.
In the C code I can replace glCompileShader and glLinkProgram by glCompileShaderARB and glLinkProgramARB without differences in all cases.

I have to check the use of the GL20 entry points in the Java code, but I am not able to do it quickly now.

- Do you use the same context initialization with all wrappers? Have you checked if the GL version returned is the same in all cases?

The OpenGl version is 2.1 Mesa 7.6-devel for all tests. Same problem with a 7.5-devel, both got from the Fedora repositories.

- Do you null-terminate the shader texts?

Yes, explicitely.