I came across a strange problem while coding a fragment shader using LWJGL, and after much trial and error I made the following code to pinpoint the problems.

1. Runs fine:


void test(int n)
{
    float s=0.0;
    for (int i=0;i<100;i++)
    {
        s++;
    }
}

void main()
{
   test(100);
   gl_FragColor = vec4(1.0,1.0,1.0,1.0);
}


2. Hangs the program, black screen and unresponsive application:


void test(int n)
{
    float s=0.0;
    for (int i=0;i<n;i++)
    {
        s++;
    }
}

void main()
{
   test(100);
   gl_FragColor = vec4(1.0,1.0,1.0,1.0);
}


3. Fatal crash:


void test(int n)
{
    float s=0.0;
    for (int i=0;i<300;i++)
    {
        s++;
    }
}

void main()
{
   test(100);
   gl_FragColor = vec4(1.0,1.0,1.0,1.0);
}


#
# An unexpected error has been detected by Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x693533ee, pid=1224, tid=820
#
# Java VM: Java HotSpot(TM) Client VM (1.6.0-beta2-b86 mixed mode, sharing)
# Problematic frame:
# C  [atioglxx.dll+0x3533ee]
#
# An error report file with more information is saved as hs_err_pid1224.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#



4. But, the code equivalent of the above runs fine:


void test(int n)
{
    float s=0.0;
    for (int i=0;i<100;i++)
    {
        s++;
    }
}

void main()
{
   test(100);
   test(100);
   test(100);
   gl_FragColor = vec4(1.0,1.0,1.0,1.0);
}


What is going on here?? Driver bug? Does the above programs work on your computer? :?

I have an ATI Mobility Radeon x600, with the Omega 3.8.252 drivers, which is the most recent ones I found.

Any help or pointers would be greatly appreciated, I am helplessly stuck with my program without the above working properly...

When do these shaders fail? At compile, link or run time?

Anyway:

2. This one fails because the compiler does not understand that "int n" is a constant. Try "const int n" instead (probably won't work either).

3. This one probably fails because you're surpassing the maximum instruction count on R420 (512 math + 512 tex instructions iirc).

4. Probably the compiler figures out you're passing the same parameter and optimizes away the last two calls. Try doing this:

test(100);
test(101);
test(99);

:lol:

Btw, what you're doing is generally a bad idea. What for do you need such a big loop?

When do these shaders fail? At compile, link or run time?

At run time, they all compile fine.

Anyway:

2. This one fails because the compiler does not understand that "int n" is a constant. Try "const int n" instead (probably won't work either).

Notice how the examples where n is not actually used, only passed, works. Anyway, this is not really important, the loop problem is...

3. This one probably fails because you're surpassing the maximum instruction count on R420 (512 math + 512 tex instructions iirc).

Yes, this is what I thought too, so that's why I included test 4. Test 3 and 4 are both running the same number of instructions. However, now it seems like the compiler optimized this likeness away, so it looks like you are right...

Do you know if nVidia has similar problems, and if there are any known workarounds on ATI? My program really needs more instructions than that...

4. Probably the compiler figures out you're passing the same parameter and optimizes away the last two calls. Try doing this:

Code: [Select]

test(100);
test(101);
test(99);

That worked fine, but it looks like you are right. I made a more torough test, which failed.

Updated test 4:


void test1(int n)
{
    float s=0.0;
    for (int i=0;i<100;i++)
    {
        s++;
    }
}

void test2(int n)
{
    float s=0.0;
    for (int i=0;i<100;i++)
    {
        s++;
    }
}

void test3(int n)
{
    float s=0.0;
    for (int i=0;i<100;i++)
    {
        s++;
    }
}

void main()
{
   test1(100);
   test2(100);
   test3(100);
   gl_FragColor = vec4(1.0,1.0,1.0,1.0);
}


So, the compiler does some smart things, although not too smart, or it would also be able to omit the first call to a method that is not doing anything.

Btw, what you're doing is generally a bad idea. What for do you need such a big loop?

The examples are just constructed to highlight the problems I found. The actual code has less than ten loops, but with more instructions in each one. It runs at excellent framerates, until the above illustrated problem kills the shader...