Help me make my game engine.

[sasha-san]

Since StackOverflow is a too confining medium for this topic. I beseech all of you, LWJGL users, to help me in building my on game engine and learn a bit of Java/OpenGL programming along the way.

I've actually built some functioning elements: an obj/mtl loader, non-POT map loader, other tools for loading data, some drawing methods that don't use a shader and more. I got stuck at skeleton transformations, too a 4 months break and forgot almost everything. Even looking at my code makes me confused. But maybe a fresh start is what I need.

Inspired by thebennybox LWJGL tutorials on YT I decided to use a shader for most of the work. And a shader-centered graphic engine is what I like to make.

So form time to time I'll be asking some questions regarding a part of my engine. I hope  you'll help me avoid some pitfalls and dead ends.

And so it begins...

BTW

This engine is supposed to be for all kind of games(mostly 3d), but I started learning java/opengl with one game in mind: a pc version of "Talisman". So atm it's a pc adaptation of a board game. If anyone here played the game and would like to on pc, maybe even online, please support my endevour.

[sasha-san]

I'm thinking on my mesh class drawing method. Is this an acceptable way of doing things?

public void draw()
	{
		glEnableVertexAttribArray(0); //position
		glEnableVertexAttribArray(1); //normal
		glEnableVertexAttribArray(2); //color ambient
		glEnableVertexAttribArray(3); //color diffused
		glEnableVertexAttribArray(4); //color specular
		glEnableVertexAttribArray(5); //color emission
		glEnableVertexAttribArray(6); //texture 1 coordinates
		
	    glActiveTexture(GL_TEXTURE0);
	    glBindTexture(GL_TEXTURE_2D, colorMap);
	    glActiveTexture(GL_TEXTURE1);
	    glBindTexture(GL_TEXTURE_2D, normalMap);
	    glActiveTexture(GL_TEXTURE2);
	    glBindTexture(GL_TEXTURE_2D, displacementMap);
	    glActiveTexture(GL_TEXTURE3);
	    glBindTexture(GL_TEXTURE_2D, colorMapMod);		

		glBindBuffer(GL_ARRAY_BUFFER, vbo);
		glVertexAttribPointer(0, 3, GL_FLOAT, false, Vertex.SIZE * 4, 0);
		glVertexAttribPointer(1, 3, GL_FLOAT, false, Vertex.SIZE * 4, 12);
		glVertexAttribPointer(2, 4, GL_FLOAT, false, Vertex.SIZE * 4, 24);
		glVertexAttribPointer(3, 4, GL_FLOAT, false, Vertex.SIZE * 4, 40);
		glVertexAttribPointer(4, 4, GL_FLOAT, false, Vertex.SIZE * 4, 56);
		glVertexAttribPointer(5, 4, GL_FLOAT, false, Vertex.SIZE * 4, 72);
		glVertexAttribPointer(6, 2, GL_FLOAT, false, Vertex.SIZE * 4, 88);

		glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, ibo);
		glDrawElements(GL_TRIANGLES, size, GL_UNSIGNED_INT, 0);

		glDisableVertexAttribArray(0);
		glDisableVertexAttribArray(1);
		glDisableVertexAttribArray(2);
		glDisableVertexAttribArray(3);
		glDisableVertexAttribArray(4);
		glDisableVertexAttribArray(5);
		glDisableVertexAttribArray(6);
		
	    glActiveTexture(GL_TEXTURE0);
	    glBindTexture(GL_TEXTURE_2D, 0);
	    glActiveTexture(GL_TEXTURE1);
	    glBindTexture(GL_TEXTURE_2D, 0);
	    glActiveTexture(GL_TEXTURE2);
	    glBindTexture(GL_TEXTURE_2D, 0);
	    glActiveTexture(GL_TEXTURE3);
	    glBindTexture(GL_TEXTURE_2D, 0);

	}

Am I missing something? Should I make room for more data like maps. The Idea is to send everything that doesn't change at once. Variables would be sent as uniforms.

Also, should I include bone index/weight data here? And how about the bone matrices themselves? Send each as a uniform matrix4f or is there a better way of sending a group of matrices at once?

[sasha-san]

Now that I think of it, I have to decide on how an object(3d object from Blender for example) will be imported, converted to a vbo & ibo, modified and sent to the shader.

Lets begin with color. It would be easier if I assume that all 3d object are made of other objects/groups which have their specific or shared material. This way the colors would be sent as mat4f uniforms for an object part. An example would be a 3d model of a car with some textures and various materials. While the whole car could be imported as one unified object, I would have to include the colors for every vertex. The textures would be inapplicable. So am I to assume that this approach is wrong? Is vertex-painted objects even necessary?

Or should I use an "Object" class containing a map of all its meshes and materials, with a vbo&ibo for each component separately?

[sasha-san]

Is this how I should go about sending vertex bone indices and weights?

public void draw()
	{
	glEnableVertexAttribArray(0); //position
	glEnableVertexAttribArray(1); //normal
	glEnableVertexAttribArray(2); //texture coordinates
	for(int i = 0; i < numberOfBones; i++) {
		glEnableVertexAttribArray(i+3);
	}

	glBindBuffer(GL_ARRAY_BUFFER, vbo);
	glVertexAttribPointer(0, 3, GL_FLOAT, false, Vertex.SIZE * 4, 0);
	glVertexAttribPointer(1, 3, GL_FLOAT, false, Vertex.SIZE * 4, 12);
	glVertexAttribPointer(2, 2, GL_FLOAT, false, Vertex.SIZE * 4, 24);
	for(int i = 0; i < numberOfBones; i++) {
		glVertexAttribPointer(i+3, 2, GL_FLOAT, false,  Vertex.SIZE * 4, 32+i*8);//first float is the bone index, the second is the weight	
	}

	glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, ibo);
	glDrawElements(GL_TRIANGLES, size, GL_UNSIGNED_INT, 0);

	glDisableVertexAttribArray(0);
	glDisableVertexAttribArray(1);
	glDisableVertexAttribArray(2);
	for(int i = 0; i < numberOfBones; i++) {
		glDisableVertexAttribArray(i+3);
	}

	}

Not sure if I wrote it correctly, but it's the idea that matters. But the variable numberOfBones would have to be a static value. From the get-go there would have to be an exact number of bones a mesh vertex in my engine is influenced by. I would be reserving slots in the vbo for bone data, whether there are any bones or not. If this is a correct approach, what number of bones should be the limit?

[quew8]

So I've got a couple of thoughts.

Firstly you're trying to do some quite complex stuff considering you're a bit of a novice with OpenGL (don't get me wrong you seem to be doing well but experience trumps all) and, you gave the impression, Java as well. Are you really using a displacement map and a moded colour map? I have been working with OpenGL for years, doing similar stuff to you, and I've never had cause to do either. Give it some thought and don't do anything you don't need to. It's a drain on performance and will come back to bite you. You asked:

Am I missing something? Should I make room for more data like maps.

And the answer is absolutely not. At least until you decide you need it.

I would have to include the colors for every vertex. The textures would be inapplicable. So am I to assume that this approach is wrong?

I'm sure there are some specialist situations where this is exactly what is necessary. But not for what you're doing. Seriously pick up any object in front of you and see if there is any gradual change (like what you would see in fragment interpolation) in the material properties (essentially how it reflects light). I guarantee you the answer is no. So yes, send material properties as uniforms. At most: send multiple materials and give an attribute for the index of the material to use, but I would avoid even this.

If this is a correct approach, what number of bones should be the limit?

You shouldn't use a static number of bones. The restrictions that would put on your engine would be too much. Sometimes my shader implementations require an upper limit, but I can set this at runtime to allow for any situation I need. All in all, it seems to me that your rendering engine seems very inflexible. I once had the same problem and it took a lot of refactoring to fix it but now I can replace almost any rendering code as needed without rewriting anything but what is different. I suggest you aim for a similar outcome.

That is all for now I think.

[sasha-san]

@quew8 Thank you so much for the reply. Right now I'm gathering ideas on how my graphic engine should work.

Am I missing something? Should I make room for more data like maps.

And the answer is absolutely not. At least until you decide you need it.

I might have gone too far on that one. It appears that instead of an engine I went straight for the game. Thinking of making a shader, instead of making tools for communication and management of shader programs.

If this is a correct approach, what number of bones should be the limit?

You shouldn't use a static number of bones. The restrictions that would put on your engine would be too much. Sometimes my shader implementations require an upper limit, but I can set this at runtime to allow for any situation I need. All in all, it seems to me that your rendering engine seems very inflexible. I once had the same problem and it took a lot of refactoring to fix it but now I can replace almost any rendering code as needed without rewriting anything but what is different. I suggest you aim for a similar outcome.

This the hardest part for me since my knowledge of how to communicate with shaders is basic at best. Could you give me example of skeleton animation/transformation using LWJGL? Especially the part where data is sent to the shader and how the shader works with it.

[quew8]

I'm afraid I can't give you an example. Purely coincidentally, I was working on my own skeletal animation when you posted, and all the code is in place however, I've encountered a bug in the "infrastructure" of the skeletal animation. By which I mean the bit which isn't actually specific to skeletal animation, which is odd since the whole purpose of the design means that that code is actually common. But hey. The point is right now I can only give theoretical help.

In terms of communicating with the shader, I presume you are talking about the variable number of skeletal matrices. The way I do variable number of variables, is to create a fixed sized uniform array. When I load the shader into OpenGL, I replace the size of the array with the maximum size I would need. Whenever you upload a set of matrices, you also have to set a uniform integer saying how many matrices are in use. Potentially (although I don't do this) you could recreate the shader if at any time your requirements changed.

Another method (which I don't particularly like and have never implemented) is to use a uniform sampler object. That way you can send the data to OpenGL via a texture which is resizeable. I don't see this as a "fast" method, if nothing else since one must interpret the texture data as a matrix at the shader level every single vertex of every single frame. But this is a more dynamic, if problematic, method.

As for what to do with the data, take a look at COLLADA's specification: http://www.khronos.org/files/collada_spec_1_4.pdf. More specifically the "Skinning a Skeleton in COLLADA" section of Chapter 4. Otherwise I would need a more specific information request.

Hope this helps.

[sasha-san]

Hmm. This might be something i should've started with... Skeletal transformations(lets leave animation for later) have two parts in the context of lwjgl. One is the weights and bone indices/names assigned to every effected vertex. The other are bone matrices. Right now I think we covered the matrices part, but what about the vertex data. I was kinda hoping for one draw() method in my Mesh class, but vbos with different amount of bones would confuse the attribute pointer. Would the code in my earlier posts be a good solution? Or am I just pulling things outta' my ass?

BTW. When I started, I used vertex/color/normal/texcoord pointers(since I could do it without a shader), but someone told me that these methods were deprecated and vertex attribute pointers were the way. True or not, which would you suggest?

[abcdef]

I have skeletal animation working in the old open gl (open gl 1, plan is to support it in all versions for backwards compatibility)

I have

• A skeleton object with a map of bones
• each bone has a transformation
• I have a set of meshes which have skin data (ie one or more bones influencing it)

To animate I

• first update the skeleton by going from the root bone to the end bones and working out the transformations on the way
• then apply these transformations to the mesh
• If there is more than one bone to skin on a mesh I work out the transformation that needs to be done which replicates the weighting of the various bones combined.
• I then use the original vertex data with this new transformation and pass this to the renderer. The plan is this could then be adapted to VBO / shader rendering later

[quew8]

@sasha-san I meant to use a similar method for vertex weights. For simple models I doubt you'll need more than 4 bone weights per vertex right? So an attribute ivec4 for the bone indices and an attribute vec4 for the associated bone weights. By the way, I said in the previous post that you should also use a uniform integer to see how many bones you've uploaded. But that's just stupid, not sure why I said it. It doesn't matter how many bones you've got since you're indexing the weights. Any way, I hope that helps.

As for a single draw call. I suggest that you make use of the central concept of Java: Object Orientated Programming. For my system I use  a couple of interfaces to determine how data is drawn. A little example:

GeometricDataInterpreter determines how a generic geometric object is converted into vertex and index data.

/**
 *
 * @param <T> 
 * @author Quew8
 */
public interface GeometricDataInterpreter<T> {
    
    public Vector[] toPositions(T[] t); 
    
    public float[] toVertexData(T[] t);
    
    public int[][] toIndexData(T[] t);
}

StaticRenderMode and DynamicRenderMode setup OpenGL ready for rendering a particular type of object and give a way to update transformation matrices. DynamicRenderMode can also act based on generic data contained within the drawable handle (In the case of skeletal animation, this would be the skeleton but its completely generic).

/**
 *
 * @author Quew8
 */
public abstract class StaticRenderMode {

    public void onPreRendering() {};

    public void onPostRendering() {};

    public abstract void updateProjectionMatrix(FloatBuffer matrix);
}

/**
 *
 * @param <T> 
 * @author Quew8
 */
public abstract class DynamicRenderMode<T> extends StaticRenderMode {
    
    public void onPreDraw(T data) {};

    public void onPostDraw(T data) {};
    
    public void bindGeneralTexture(Image image) {
        image.bind();
    }
    
    public abstract void updateModelMatrix(FloatBuffer matrix);
}

And an example usage for basic 3D, position, normal, tex-coord data:

/**
 *
 * @author Quew8
 * @param <T>
 */
public class FGeneralInstanceMode<T> extends DynamicRenderMode<T> {
    private final ShaderProgram shaderProgram;
    
    private FGeneralInstanceMode(ShaderProgram shaderProgram, Class<T> clazz) {
        this.shaderProgram = shaderProgram;
    }
    
    @Override
    public void onPreRendering() {
        shaderProgram.use();
        ShaderUtils.setVertexAttribPointer(0, 3, GL_FLOAT, false, 32, 0);
        ShaderUtils.setVertexAttribPointer(1, 3, GL_FLOAT, false, 32, 12);
        ShaderUtils.setVertexAttribPointer(2, 2, GL_FLOAT, false, 32, 24);
    }
    
    @Override
    public void updateModelMatrix(FloatBuffer matrix) {
        ShaderUtils.setUniformMatrix(shaderProgram.getId(), "modelMatrix", matrix);
    }
    
    @Override
    public void updateProjectionMatrix(FloatBuffer matrix) {
        ShaderUtils.setUniformMatrix(shaderProgram.getId(), "projectionViewingMatrix", matrix);
    }
}

Hope this helps.

@abcdef Nice post, good fundamentals and impressed you can get any speed for non-hardware-accelerated skeletal animation. However I would suggest that it really isn't necessary to maintain backward compatibility woth OpenGL 1.1. I do backward compatibility too but mine is with OpenGL 2.1 (Or at least it will be very soon). With shaders, 2.0 was a bi step forward, provides an excellent speed boost and (here's the thing) I've never encountered, or heard of, a computer without at least support OpenGL 2.1 (Discounting dodgy drivers). My humble opinion.

[sasha-san]

@abcdef I think I get the concept of skeletal animation. I made a Skeleton class to govern the bones and to easily attach a skeleton to a mesh. Bone classes with parent/child connections, calculating its end-matrix by multiplying its matrix with its parents end-matrix and so on up to the root bone. Even made some linear interpolated animations going, but only in the console since I didn't know hot to use a shader for that kind of thing. The idea is,I think, for the vbo to be unchanging and all the transformation made in the shader. One could make the transformations in java and send a new vertex array each frame, but it seams wasteful.

@quew8 Since render modes were some of the first things I learned, they were the first to go too... A rendering class... I just have a draw method in my gameloop and most of the work is done by the meshes draw(). It's something to think about.

Right now I'm working on sending material data to the shader program. Tell me what you think.

public class Material {
	
	private Map<String, Vector4f> colors;	
	private Map<String, Integer> maps;
	private Shader program;
	public void setMaterialUniformPointers(Shader program) {
		this.program = program;
		for(String colorName : colors.keySet()) {
			program.addUniform(colorName);			
		}
		for(String mapName : maps.keySet()) {
			program.addUniform(mapName);
		}
	}
	
	private void sendColors() {
		for (String colorName : colors.keySet()) {
			program.setUniformVec4f(colorName, colors.get(colorName));
		}
	}
	
	private void sendMaps() {	
		int activeTexture = 0;
		for (String mapName : maps.keySet()) {
			glActiveTexture(GL_TEXTURE0 + activeTexture);
			glBindTexture(GL_TEXTURE_2D, maps.get(mapName));
			program.setUniformi(mapName, activeTexture);	
			activeTexture++;
		}
		glActiveTexture(GL_TEXTURE0);
	}
	
	public void sendMaterial() {
		sendColors();
		sendMaps();
	}
}

Hmm.. Now that I think about it, the set uniform part may be buggy. I could make engine constants like COLORMAP, DIFFUSEDCOLOR that would than be used as uniform pointers for the shader. Any ideas? Maybe an enum with map and color types to make things clear?

BTW What file format are you using for the skeleton? I tried write my own export script for Blender, but there must be something useful already.

[quew8]

Sorry, I didn't make it clear. The idea is that you can give each Mesh (I presume the mesh represents a section of a VBO) its own RenderMode at initialization which you call at appropriate times in the draw() call. So any style of rendering can be accommodated with a simple loop through the geometry.

The Material class looks fine but all your ideas also sound good. I would tend towards Enums since they're like anti-bug remedies. However they may be too static for your needs. Try it, see what happens. (This next bit might not be relevant) If by pointer you mean the integer returned by glGetUniformLocation() (or whatever it is) I might warn you that if you re-link the shader program the position of uniforms and attributes might change. Just a warning.

As for file format, I use the COLLADA format (.dae) which is an open source, xml-based format (from Khronos, the same people as the OpenGL specs). It has support for all sorts of geometric data from meshes and skinning data to animations and shader effects. Of course you can ignore as much of that as you want. It's quite a complex format to get into (I feel) and I certainly wasn't able to find any outstanding tutorials. After my learning experience, I think I'll write my own tutorial on the JGO forums so if you want to learn maybe take a look there in the near future. In the mean time there is this: http://www.wazim.com/Collada_Tutorial_1.htm. Generally one would only use this as an intermediate format. When you deploy a game you use your own binary format with only the bare essentials to reduce file size.

[abcdef]

@quew8

You are completely right on the "everything" supports version 2, I'm a bit of a sucker for optimising my code as much as possible so I thought I would start with the least hardware accelerated method and explore which parts of the code are slow and which are fast and to try and make things as optimal as possible. I've made things multithreaded to do certain actions in parallel and to leave the render thread to only render, I've also found what the CPU sucks at at tried to move things to the GPU. Its kind of fun to learn things and experiment. The skeletal animation (bone transformations) I can do really fast on the CPU the mesh transformation is pretty slow (like 10fps for a single skeleton). I'm slowly moving more things to the GPU and eventually I'll start using shaders. I've designed things such that upgrading will be fairly easy

@sasha-san

I understand where you are coming from, the VBO method you mention is a good one. I currently do that with plain normal state machine opengl, I look forward to the speed ups when I have ironed out all the inefficient code

[sasha-san]

I'm getting no location from glunifromlocation(), the uniform exists and is used in the shader. What gives?

[quew8]

Make sure the shader is linked. If it is, it might even have to be in use (that's not specification defined behaviour but one can't always trust drivers). Failing that, I don't know.

Also, I just found the bug in my skeletal animation (It was in my COLLADA parser, surprise surprise). So I can give you some example code now if you'd like.