Slow texture loading

[lwjuggler]

Hi all,

   I'm working on a 2d game which requires lot of texture loadings and i'm facing a loading time problem.
I'm trying to load PNG's with Java's ImageIO class, then I put the raster data of the image in a byteBuffer to pass to glTexImage2D (I followed the most basic wayi've seen in this forum). I profiled my game and it seem's that the application wastes 2 seconds for the PNG decoding and 17 seconds in sending data to GPU. for the same images loading, i got 2 seconds loading in the c++ version of the game. So I'm wondering why there is a gap of performance.

Any of you have encountered this problem ?

Thanks by advance.

[Matthias]

Use the TWL PNGDecoder instead of ImageIO.

[lwjuggler]

Hallo Matthias, wie geht es dir ? ^^

Thanks for your quick reply Matthias. I have downloaded your jar, and used it like this :

class textureLoader {

....


	private static Texture loadFromProperties(String fileName) {
		Texture texture = null;

		String path = fileName.split("data/")[1].split("[.]")[0];

		String size = (String) ResourceLoader.imageProperties.get(path);
		String[] dims = size.split("x");
		int w = Integer.parseInt(dims[0]);
		int h = Integer.parseInt(dims[1]);
		texture = new Texture(w, h);
		texture.target = GL11.GL_TEXTURE_2D;

		return texture;
	}

	public static void createBuffer() {
		textureBuffer = ByteBuffer.allocateDirect(4 * decoder.getWidth() * decoder.getHeight());
		try {
			decoder.decode(textureBuffer, decoder.getWidth() * 4, de.matthiasmann.twl.utils.PNGDecoder.Format.RGBA);
		} catch (IOException e) {
			e.printStackTrace();
		}
		textureBuffer.flip();
	}

	public static void decodePNG(String fileName) {
		try {
			decoder = new PNGDecoder(new FileInputStream(fileName + ".png"));
		} catch (FileNotFoundException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

	public static void createGLTexture(Texture texture) {

		int srcPixelFormat = 0;
		// create the texture ID for this texture
		texture.textureID = createTextureID();

		// bind this texture

		GL11.glBindTexture(texture.target, texture.textureID);
		if (decoder.hasAlpha()) {
			srcPixelFormat = GL11.GL_RGBA;
		} else {
			srcPixelFormat = GL11.GL_RGB;
		}

		GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_MIN_FILTER, GL11.GL_NEAREST);
		GL11.glTexParameteri(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_MAG_FILTER, GL11.GL_NEAREST);

		GL11.glTexParameterf(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_WRAP_S, GL11.GL_CLAMP);
		GL11.glTexParameterf(GL11.GL_TEXTURE_2D, GL11.GL_TEXTURE_WRAP_T, GL11.GL_CLAMP);

		GL11.glTexImage2D(GL11.GL_TEXTURE_2D, 0, GL11.GL_RGBA, decoder.getWidth(), decoder.getHeight(), 0, srcPixelFormat, GL11.GL_UNSIGNED_BYTE, textureBuffer);

	}

	/**
	 * 
	 */

	public static Texture loadPNGTexture2D(String fileName) {
		Texture texture = loadFromProperties(fileName);
		decodePNG(fileName);
		createBuffer();
		createGLTexture(texture);
		return texture;
	}

}

Unfortunately, there are no improvements, I still have a loading taking around 20 seconds. Am i doing something wrong ?
Also, I observed that when I load my game the second time, It takes only 2 seconds. Does it matter with some caching ? If it does, when does it exactly happen and how does it work ?

[Matthias]

Hello,

first you seem to use a lot of static variables, and don't use try{}finally{} to close the InputStream. This could cause issue when you load a lot of files.

About the loading speed - 20sec for one texture is A LOT. What texture size do you use? Caching has of course a factor - depending on your hard disk and file size (also on fragmentation).

One thing you could try is to decode to BGRA format and use GL12.GL_BGRA as format - many OpenGL implementations use this as native format. You could also try to use ARB_pixel_buffer_objects to allocate a GL managed ByteBuffer (instead of ByteBuffer.allocateDirect).

You should try to profile you app to locate exactly where the time is spend.

Also you have a method "createTextureID()" which seems like you could use directly the overloaded version of GL11.glGenTextures() without parameters. It returns the texture ID as an int without the need to use an temporary IntBuffer.

Ciao
Matthias

[lwjuggler]

No no,

it is not only for one texture ^^ I'm loading a map :  16 background images (1024*1024), sprites (60 frames) and other images.
I'll begin with your advices, then i'll come back tto you !


Thanks a lot.

[lwjuggler]

I followed your advices, but i didn't manage to find informations on how to use ARB_pixel_buffer_objects with lwjgl.
Could you give some help ?

[Matthias]

It's in the class ARBPixelBufferObject. The basic steps are:

// create PBO
int id = ARBPixelBufferObject.glGenBuffersARB();
// bind it for texture upload
ARBPixelBufferObject.glBindBufferARB(ARBPixelBufferObject.GL_PIXEL_UNPACK_BUFFER_ARB, id);
// allocate uninitialized storage
ARBPixelBufferObject.glBufferDataARB(ARBPixelBufferObject.GL_PIXEL_UNPACK_BUFFER_ARB, size, ARBPixelBufferObject.GL_DYNAMIC_DRAW_ARB);
// map it
ByteBuffer bb = ARBPixelBufferObject.glMapBufferARB(ARBPixelBufferObject.GL_PIXEL_UNPACK_BUFFER_ARB, ARBPixelBufferObject.GL_WRITE_ONLY_ARB, size, null);
// unbind it
ARBPixelBufferObject.glBindBufferARB(ARBPixelBufferObject.GL_PIXEL_UNPACK_BUFFER_ARB, 0);

// now write into that ByteBuffer - this can also be done on another thread

// after writing to the ByteBuffer bind it again
ARBPixelBufferObject.glBindBufferARB(ARBPixelBufferObject.GL_PIXEL_UNPACK_BUFFER_ARB, id);
// unmap it
ARBPixelBufferObject.glUnmapBufferARB(ARBPixelBufferObject.GL_PIXEL_UNPACK_BUFFER_ARB);
// now upload to the texture
GL11.glTex(Sub)ImageXD(....., 0L);  // the offset 0L is an offset into the bound PBO
// unbind the PBO
ARBPixelBufferObject.glBindBufferARB(ARBPixelBufferObject.GL_PIXEL_UNPACK_BUFFER_ARB, 0);

// a PBO can be reused and must be deleted when no longer used
// make sure that it is no longer mapped or bound before deleteing
// use try/finally to ensure correct order of operations even when decoding of the image fails
ARBPixelBufferObject.glDeleteBuffersARB(id);

If you allocate a PBO large enough for the largest image then you can use a single PBO to upload all images. But it could be beneficial to use 2 alternating PBOs to give OpenGL time to upload the data before you try to map it again - otherwise map will block until the upload is finished.

Ciao
Matthias

[lwjuggler]

Wonderful !

But, maybe a last question. Which version of lwjgl do you use, because I don't have the Gl11.glGentextures(void) method, I must put an IntBuffer parameter.

[Matthias]

Use of course the latest - currently LWJGL 2.6

[lwjuggler]

Hi Matthias,
      this is me again ^^I followed your advices. I tried to use ARBPixelBufferObject, but i realized it may not work on all platforms (beginning with my chief's one, he got an invalid enum exception).

       However, I tried to use a single Bytebuffer, because before, i used to create one everytime I load a texture. So now, I use a single one, with a large capacity, and rewind it after i load a texture. But I can see no improvements.

Is it normal ?

[Matthias]

If you want help you should paste code - from what you told it's hard to say what can be improved.
Have you tried running the PNGDecoder in a separate thread from OGL? so that you can decode one image while you upload another? Use java.util.concurrent.* for this.

[lwjuggler]

Hallo Matthias,

     I left this part of the development for a moment, to go ahead in the game development. However, 3 days ago, i restarted trying to improve the texture loading. As you asked me, i Show my code :

package graphic.texture;

import graphic.Dimension;
import graphic.opengl.IGL;

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.IntBuffer;
import java.util.concurrent.CopyOnWriteArrayList;

import org.lwjgl.opengl.GL11;
import org.lwjgl.opengl.GL12;

import utils.ResourceLoader;
import de.matthiasmann.twl.utils.PNGDecoder;

public class FastTextureloader extends TextureLoader {

	public static CopyOnWriteArrayList<PNGTexture> decoders;

	static int bufferSize = 2048 * 2048;

	public TextureBuffer textureBuffer1;
	public TextureBuffer textureBuffer2;

	public FastTextureloader() {
		textureBuffer1 = new TextureBuffer();
		textureBuffer2 = new TextureBuffer();
		textureBuffer1.buffer = ByteBuffer.allocateDirect(4 * bufferSize).order(ByteOrder.nativeOrder());
		textureBuffer2.buffer = ByteBuffer.allocateDirect(4 * bufferSize).order(ByteOrder.nativeOrder());
		decoders = new CopyOnWriteArrayList<PNGTexture>();
	}

	private TextureBuffer getBuffer() {

		while (!textureBuffer1.ready && !textureBuffer2.ready) {
			//System.out.println("waiting for buffer");
		}
		if (!textureBuffer1.ready && textureBuffer2.ready) {
			return textureBuffer2;
		} else if (textureBuffer1.ready && !textureBuffer2.ready) {
			return textureBuffer1;
		} else
			return textureBuffer1;

	}

	private static Texture loadFromProperties(String fileName) {
		Texture texture = null;
		Dimension dim = ResourceLoader.getImageDimensions(fileName);
		texture = new Texture(dim.width, dim.height);
		texture.target = IGL.GL_TEXTURE_2D;

		return texture;
	}

	public void decodePNG(String fileName, Texture texture) {
		try {
			FileInputStream is = new FileInputStream(fileName);
			PNGTexture pngTex = new PNGTexture();
			pngTex.decoder = new PNGDecoder(is);
			int nbComponents;
			de.matthiasmann.twl.utils.PNGDecoder.Format format = null;
			if (pngTex.decoder.hasAlpha()) {
				nbComponents = 4;
				format = de.matthiasmann.twl.utils.PNGDecoder.Format.BGRA;
			} else {
				nbComponents = 3;
				format = de.matthiasmann.twl.utils.PNGDecoder.Format.RGB;
			}
			TextureBuffer textureBuffer = getBuffer();
			//System.out.println("decoding " + fileName);
			pngTex.decoder.decode(textureBuffer.buffer, pngTex.decoder.getWidth() * nbComponents, format);
			pngTex.fileName = fileName;
			pngTex.texture = texture;
			pngTex.textureBuffer = textureBuffer;
			textureBuffer.buffer.rewind();
			decoders.add(pngTex);
			is.close();
		} catch (FileNotFoundException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		}

	}

	/**
	 * 
	 */

	public Texture loadPNGTexture2D(String fileName) {
		Texture texture = loadFromProperties(fileName);
		decodePNG(fileName, texture);
		return texture;
	}

	public void createGLTexture(PNGTexture pngTexture) {

		int srcPixelFormat = 0;
		// create the texture ID for this texture
		int textureID = GL11.glGenTextures();

		pngTexture.texture.textureID = textureID;
		// bind this texture
		int nbComponents = 0;
		GL11.glBindTexture(pngTexture.texture.target, textureID);
		if (pngTexture.decoder.hasAlpha()) {
			srcPixelFormat = GL12.GL_BGRA;
			nbComponents = 4;
		} else {
			srcPixelFormat = GL11.GL_RGB;
			nbComponents = 3;
		}

		GL11.glTexParameteri(IGL.GL_TEXTURE_2D, GL11.GL_TEXTURE_MIN_FILTER, GL11.GL_NEAREST);
		GL11.glTexParameteri(IGL.GL_TEXTURE_2D, GL11.GL_TEXTURE_MAG_FILTER, GL11.GL_NEAREST);

		GL11.glTexParameterf(IGL.GL_TEXTURE_2D, GL11.GL_TEXTURE_WRAP_S, GL11.GL_CLAMP);
		GL11.glTexParameterf(IGL.GL_TEXTURE_2D, GL11.GL_TEXTURE_WRAP_T, GL11.GL_CLAMP);

		pngTexture.textureBuffer.ready = false;
		GL11.glTexImage2D(IGL.GL_TEXTURE_2D, 0, nbComponents, pngTexture.decoder.getWidth(), pngTexture.decoder.getHeight(), 0, srcPixelFormat, GL11.GL_UNSIGNED_BYTE, pngTexture.textureBuffer.buffer);
		pngTexture.textureBuffer.ready = true;
	}

	/**
	 * permet d'attacher la texture
	 */
	public static void bind(Texture texture) {
		GL11.glBindTexture(texture.target, texture.textureID);
	}

	public void unloadTexture(int id) {
		GL11.glDeleteTextures(id);
	}

	/**
	 * Creates an integer buffer to hold specified ints - strictly a utility
	 * method
	 * 
	 * @param size
	 *            how many int to contain
	 * @return created IntBuffer
	 */
	protected static IntBuffer createIntBuffer(int size) {
		ByteBuffer temp = ByteBuffer.allocateDirect(4 * size);
		temp.order(ByteOrder.nativeOrder());
		return temp.asIntBuffer();
	}

	public void clear() {
		textureBuffer1.buffer.clear();
		textureBuffer2.buffer.clear();
	}

	public void loadTextures() {
		for (PNGTexture pngTexture : decoders) {
			createGLTexture(pngTexture);
			decoders.remove(pngTexture);
		}
	}
}

assuming :

public class PNGTexture {
	public String fileName;
	public PNGDecoder decoder;
	public Texture texture;
	public boolean loaded = false;

	public TextureBuffer textureBuffer;

	public PNGTexture() {

	}
}

and :

public class TextureBuffer {
	public boolean ready = true;
	public ByteBuffer buffer;

}

I created a simple test class to test the texture laoding :

public class TextureLoadingTest {

	public ArrayList<Drawable> images;
	FastTextureloader fastTextureloader;
	private boolean isTxLoadingFinished = false;
	long t1, t2;

	public TextureLoadingTest() {
		images = new ArrayList<Drawable>();
		createWindow();
		mainLoop();
	}

	private void mainLoop() {

		while (!Display.isCloseRequested()) {
			render();
			eventsListener();

			Display.update();

			Display.setTitle("Test");
		}
	}

	private void createWindow() {

		int width = 800;
		int height = 600;
		int depth = 16;

		DisplayMode modes[];
		try {
			modes = Display.getAvailableDisplayModes();
			for (DisplayMode currentMode : modes) {
				if (currentMode.getWidth() == width && currentMode.getHeight() == height && currentMode.getBitsPerPixel() == depth) {
					Display.setDisplayMode(currentMode);
					break;
				}
			}
		} catch (LWJGLException e) {
			e.printStackTrace();
		}

		Display.setTitle("GameTest");
		Display.setLocation(10, 10);
		try {
			Display.create();
		} catch (LWJGLException e) {
			e.printStackTrace();
		}

		initGL();
		init();

		Display.sync(60);

	}

	private void loadMap() {
              // HERE I load all images
	}

	private void init() {
		WNGame.mainTimer = new LWJGLTimer();
		t1 = System.currentTimeMillis();

		new Thread() {

			public void run() {
				loadMap();
				isTxLoadingFinished = true;
				t2 = System.currentTimeMillis();
				long delta = t2 - t1;
				System.out.println("" + delta / 1000f);
			};
		}.start();

	}

	private void initGL() {
		LWJGL lwjgl = new LWJGL();
		fastTextureloader = new FastTextureloader();
		lwjgl.textureLoader = fastTextureloader;
		Scene.igl = lwjgl;

		Scene.igl.glEnable(IGL.GL_BLEND);
		Scene.igl.glEnable(IGL.GL_ALPHA_TEST);
		Scene.igl.glBlendFunc(IGL.GL_SRC_ALPHA, IGL.GL_ONE_MINUS_SRC_ALPHA);

		Scene.igl.glMatrixMode(IGL.GL_PROJECTION);
		Scene.igl.glLoadIdentity();
		Scene.igl.gluOrtho2D(0, 3800, 0, 3600);
		Scene.igl.glMatrixMode(IGL.GL_MODELVIEW);
		Scene.igl.glLoadIdentity();
	}

	private void eventsListener() {

	}

	private void render() {
		GL11.glClear(GL11.GL_COLOR_BUFFER_BIT);
		while (!isTxLoadingFinished) {
			fastTextureloader.loadTextures();
		}

		for (Drawable image : images) {
			image.draw();
		}

	}

	public static void main(String[] args) {
		new TextureLoadingTest();
	}

}

It seems to work, but, images are displayed strangely (some images are not complete etc). I guess I did something wrong in the order of calling the bytebuffer rewind() method.
But i still have a big difference between the first and second run. You didn't tell me exactly when caching occurs. I've searched on google and it seems you can disable it with the ImageIo set UseCache method, but you jars' classes doesn't use ImageIO i guess, so when does exactly this caching occur ?

An other problem is that for now, I'm using Non Power Of Two textures (for 2d Game), so I have png files which contain Power of Two textures, that's why my buffer size is so huge. Is there a way to transform a NPOT tpng texture into POT texture via opengl ? (with gltexImage2d, subImage or build2DMipmap ...)

Thanks for helping me. By the way, if you help me resolving this problem, i will put your name into the game's list of thanks