Issue with tinyexr loading

[xsupermetroidx]

I've been experimenting for a few hours with EXR files, and I'm fairly sure I've spotted a bug that corrupts the final row of values for each color channel.

I've used the following code to load a simple solid-color EXR file which I generated and saved. I've used an EXR viewer to verify that there's nothing wrong with the input file itself.

    private static FloatBuffer load(String path) {
        EXRVersion exr_version = EXRVersion.create();

        int ret = ParseEXRVersionFromFile(exr_version, path);
        if (ret != TINYEXR_SUCCESS) {
            System.out.println("Invalid EXR file: " + path);
            return null;
        }

        if (exr_version.multipart()) {
            System.out.println("Multipart unsupported");
            return null;
        }

        EXRHeader exr_header = EXRHeader.create();
        InitEXRHeader(exr_header);

        PointerBuffer err = BufferUtils.createPointerBuffer(1);
        ret = ParseEXRHeaderFromFile(exr_header, exr_version, path, err);
        if (ret != TINYEXR_SUCCESS) {
            System.out.println("EXR parse error: " + err.get(0));
            return null;
        }

        // Read HALF channel as FLOAT. (Not needed, image is float)
        //for (int i = 0; i < exr_header.num_channels(); i++) {
        //    if (exr_header.pixel_types().get(i) == TINYEXR_PIXELTYPE_HALF) {
        //        exr_header.requested_pixel_types().put(i, TINYEXR_PIXELTYPE_FLOAT);
        //    }
        //}

        EXRImage exr_image = EXRImage.create();
        InitEXRImage(exr_image);

        ret = LoadEXRImageFromFile(exr_image, exr_header, path, err);
        if (ret != TINYEXR_SUCCESS) {
            System.out.println("EXR load error: " + err.get(0));
            return null;
        }

        PointerBuffer images = exr_image.images();
        int w = exr_image.width();
        int h = exr_image.height();
        int c = exr_image.num_channels();

        System.out.println(w + " x " + h + " x " + c);

        return images.getFloatBuffer(0, w * h); //BGR, returning blue channel
    }

In the above case I return the blue channel, but the same issue occurs in all 3 channels. The final row of values is always corrupted, containing 0s and some other values which aren't present in the image at all. In an 8x8 image, the first 58 pixels contain the correct value, whereas the final 8 contain garbage. In a 256x256 image, the first 65280 values are correct, and the final 256 are garbage, etc.

I have tried messing with the memory addresses in the PointerBuffer returned by exr_image.images(), and I'm seemingly able to return more correct values by manipulating it. This suggests it could be an issue with incorrect memory addresses.

EDIT: Further testing shows that the final row of values changes from run to run with no code changes, meaning it must be reading into something else's memory.

[spasi]

I cannot reproduce this. Could you share the test image? Are you sure the image is loaded with PIXELTYPE_FLOAT? What does exr_header.pixel_types() contain?

[xsupermetroidx]

I'm sure that they are saved and loaded as TINYEXR_PIXELTYPE_FLOAT. When I load a demo EXR file off the internet, it works fine. When I save and reload that same image, the corruption in the final row of pixels shows up. However, the re-saved image still shows up fine in EXR viewers. Perhaps the EXR viewers I tried are more tolerant of faulty files, or perhaps there's an issue with my save/load code, although it's copied as nearly as I can manage from tinyEXR's docs. I've assembled a self-sufficient test case for you to take a look at when you have time.

import org.lwjgl.BufferUtils;
import org.lwjgl.PointerBuffer;
import org.lwjgl.util.tinyexr.*;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;

import static org.lwjgl.util.tinyexr.TinyEXR.*;

public class EXRTest {

    public static void tinyEXRTest() {
        String path = ""; //insert file path here

        //create a simple raster
        float[] raster = new float[] {
            1f, 2f, 3f, 4f,    1f, 2f, 3f, 4f,    1f, 2f, 3f, 4f,
            1f, 2f, 3f, 4f,    1f, 2f, 3f, 4f,    1f, 2f, 3f, 4f,
            1f, 2f, 3f, 4f,    1f, 2f, 3f, 4f,    1f, 2f, 3f, 4f,
        };
        //save EXR to disk: no errors, looks fine in EXR viewer program
        save(pathExr, 3, 3, raster);

        //load EXR and get FloatBuffers for each channel
        FloatBuffer[] fbs = load(pathExr);

        for (int i = 0; i < 3*3; i++) {
            System.out.print(fbs[0].get(i) + ", ");
            System.out.print(fbs[1].get(i) + ", ");
            System.out.print(fbs[2].get(i) + ", ");
            System.out.println(fbs[3].get(i));
        }

//        Output:
//        1.0, 2.0, 3.0, 4.0
//        1.0, 2.0, 3.0, 4.0
//        1.0, 2.0, 3.0, 4.0
//        1.0, 2.0, 3.0, 4.0
//        1.0, 2.0, 3.0, 4.0
//        1.0, 2.0, 3.0, 4.0
//        133169.11, 4.611403E27, 2.626384E32, 1.172696E-39
//        1.18026E-40, 2.070413E-19, 2.4149792E8, 3.0353178E-31
//        1.1429877E33, 7.715514E31, 4.7423467E30, 8538626.0
    }

    private static ByteBuffer strBB(String s) {
        byte[] bytes = s.getBytes();
        ByteBuffer bb = BufferUtils.createByteBuffer(bytes.length);
        bb.put(bytes);
        bb.flip();
        return bb;
    }

    private static void save(String path, int width, int height, float[] rgba) {
        EXRHeader exr_header = EXRHeader.create();
        InitEXRHeader(exr_header);

        EXRImage exr_image = EXRImage.create();
        InitEXRImage(exr_image);

        exr_image.num_channels(4);
        exr_header.num_channels(4);

        float[][] images = new float[4][width * height];

        int x = 0;
        int y = 0;
        for (int i = 0; i < width * height; i++) {
            images[0][i] = rgba[4*i+0];
            images[1][i] = rgba[4*i+1];
            images[2][i] = rgba[4*i+2];
            images[3][i] = rgba[4*i+3];
        }

        FloatBuffer red = BufferUtils.createFloatBuffer(width * height);
        red.put(images[0]);
        red.flip();
        FloatBuffer green = BufferUtils.createFloatBuffer(width * height);
        green.put(images[1]);
        green.flip();
        FloatBuffer blue = BufferUtils.createFloatBuffer(width * height);
        blue.put(images[2]);
        blue.flip();
        FloatBuffer alpha = BufferUtils.createFloatBuffer(width * height);
        alpha.put(images[3]);
        alpha.flip();

        PointerBuffer imagesPtr = BufferUtils.createPointerBuffer(4);
        imagesPtr.put(red);
        imagesPtr.put(green);
        imagesPtr.put(blue);
        imagesPtr.put(alpha);
        imagesPtr.flip();

        exr_image.images(imagesPtr);
        exr_image.width(width);
        exr_image.height(height);

        EXRChannelInfo.Buffer channelInfos = EXRChannelInfo.create(exr_header.num_channels());
        exr_header.channels(channelInfos);
        channelInfos.get(0).name(strBB("R\0"));
        channelInfos.get(1).name(strBB("G\0"));
        channelInfos.get(2).name(strBB("B\0"));
        channelInfos.get(3).name(strBB("A\0"));

        IntBuffer pixelTypes = BufferUtils.createIntBuffer(exr_header.num_channels());
        IntBuffer requestedPixelTypes = BufferUtils.createIntBuffer(exr_header.num_channels());
        exr_header.pixel_types(pixelTypes);
        exr_header.requested_pixel_types(requestedPixelTypes);
        for (int i = 0; i < exr_header.num_channels(); i++) {
            pixelTypes.put(i, TINYEXR_PIXELTYPE_FLOAT);
            requestedPixelTypes.put(i, TINYEXR_PIXELTYPE_FLOAT);
        }

        PointerBuffer err = BufferUtils.createPointerBuffer(1);
        int ret = SaveEXRImageToFile(exr_image, exr_header, path, err);
        if (ret != TINYEXR_SUCCESS) {
            System.out.println("EXR err: " + err.get(0));
        }
        System.out.println("EXR saved at " + path);
    }

    private static FloatBuffer[] load(String path) {
        EXRVersion exr_version = EXRVersion.create();

        int ret = ParseEXRVersionFromFile(exr_version, path);
        if (ret != TINYEXR_SUCCESS) {
            System.out.println("Invalid EXR file: " + path);
            return null;
        }

        if (exr_version.multipart()) {
            System.out.println("Multipart unsupported");
            return null;
        }

        EXRHeader exr_header = EXRHeader.create();
        InitEXRHeader(exr_header);

        PointerBuffer err = BufferUtils.createPointerBuffer(1);
        ret = ParseEXRHeaderFromFile(exr_header, exr_version, path, err);
        if (ret != TINYEXR_SUCCESS) {
            System.out.println("EXR parse error: " + err.get(0));
            return null;
        }

        // Read HALF channel as FLOAT.
        for (int i = 0; i < exr_header.num_channels(); i++) {
            if (exr_header.pixel_types().get(i) == TINYEXR_PIXELTYPE_HALF) {
                exr_header.requested_pixel_types().put(i, TINYEXR_PIXELTYPE_FLOAT);
            }
        }

        EXRImage exr_image = EXRImage.create();
        InitEXRImage(exr_image);

        ret = LoadEXRImageFromFile(exr_image, exr_header, path, err);
        if (ret != TINYEXR_SUCCESS) {
            System.out.println("EXR load errror: " + err.get(0));
            return null;
        }

        PointerBuffer images = exr_image.images();
        int w = exr_image.width();
        int h = exr_image.height();

        return new FloatBuffer[] {
            images.getFloatBuffer(0, w * h),
            images.getFloatBuffer(1, w * h),
            images.getFloatBuffer(2, w * h),
            images.getFloatBuffer(3, w * h)
        };
    }

}

[spasi]

Still cannot reproduce this, the output is fine, without any random numbers. Could you share some details about your environment? (OS, LWJGL version, JVM version)

[xsupermetroidx]

Windows 7 64 bit, current release build (3.1.6 build 14), Java 10.0.1.

I just tried updating LWJGL to the latest "stable" build (3.1.7) and I'm getting a new error which is occuring at the SaveEXRImageToFile call in the demo code:

Exception in thread "main" java.lang.NullPointerException
	at org.lwjgl.system.Checks.check(Checks.java:100)
	at org.lwjgl.util.tinyexr.EXRHeader.validate(EXRHeader.java:622)
	at org.lwjgl.util.tinyexr.TinyEXR.SaveEXRImageToFile(TinyEXR.java:411)
	at glimdemo.main.EXRTest.save(EXRTest.java:130)
	at glimdemo.main.EXRTest.tinyEXRTest(EXRTest.java:32)
	at glimdemo.main.Main.main(Main.java:289)

Seems to be having an issue with the custom attributes of the exr_header.

[spasi]

Reproduced with 3.1.6. It doesn't happen with 3.1.7. The EXRHeader validation NPE will be fixed in the next 3.1.7 snapshot.

[xsupermetroidx]

Thanks once again.

[spasi]

LWJGL 3.1.7 build 9 is now available, could you please verify that it's fixed?

[xsupermetroidx]

Everything works fine now with both saving and loading. The demo example no longer produces random numbers. Thanks for the quick fix.