Simple way to do point sprites - without points

[RiverC]

Got a question for y'all. When drawing what are called 'sprites' in 3d, I often use points with textures. What is annoying to me about this (At least in how I've got it set up) is that I need two separate shader programs that I need to switch between to render a complete scene that involves some point sprites.

Is there a way to draw into projection view directly, i.e. calculate the position of the sprite itself, but draw the quad (or whatever works best) directly into projection? I had noticed an issue with ATI cards where if a point was cut off slightly in the projection space, it would disappear entirely. For some uses of sprites, I'd like to avoid that possibility. I'd also like to consolidate my render 'style'; even my shaders if possible.

My thoughts of possible solutions were:

1. Don't input into modelview, but into projection view (not sure what the heck this would do with a regular vertex shader present...!)
2. Write a second shader that is identical to the regular vertex shader but does not rotate the vertex positions, rather somehow always drawing the quad with equal 'z' values in projection space (perhaps another uniform would be helpful here.)

This sort of thing could open up the door to some other weird tricks I might like to try, since I found point sprites a bit.. limited.

[quew8]

A little off topic, but something I heard about a while back but never got around to using:
http://www.opengl.org/registry/specs/ARB/point_sprite.txt

Now, what you want is for the sprites to be translated to their position but to always be perpendicular to the view? Well the technique is called billboarding - there are several methods you can use. I seem to remember rather a good tutorial on several methods here somewhere: http://www.lighthouse3d.com/. Of coarse is you are actually using points, this is pointless but hey. The technique I used (not for particles) involved extracting the up and left viewing vectors from the viewing matrix and constructing the quad's four vertices from these.

[RiverC]

that would be the idea - so you can treat the billboarded 'sprites' as regular polygons, maybe do stuff like use four quads for some sprites and give them a slight 'roundness'...

yeah, points always 'face forward'; they're automatically billboarded. All you have to do is translate them into the right position in the Vertex Shader.

I'm trying some composite sprite stuff and points are kind of limited... maybe the first thing I need to do is create a little conceptual model regarding what I'm actually trying to achieve and if it would be easier to do it another way.

Clearly, it would be something like the following:

1. you need x,y,z for position of the 'sprite polygon' (p0)
2. you need relative x,y,z for each vertex of the 'sprite polygon' (relative to the x,y,z) - (p1)
3. all vertexes are 'set' to p0, translated using the matrix, then offset in projection view by p1. (Vertex Shader)

When you draw the sprite each piece would have a single p0 (it would be uniform) but p1 would be glVertex3f...

p1 would be determined by the 'size' of the sprite, its offset against its center, etc.

You could have the flexibility of roundness, whereby you could apply GL_LIGHT to create some dynamic contour on the so-called 'flat' sprites.

I wonder though if that's not going to be unduly inefficient.

[quew8]

Sure what you're saying is possible. In the vertex shader, multiply p0 by your modelview and projection matrices then add p1. The problem is that the projection matrix puts everything into perspective, so you would have to do something funky with p1 to get it proportional to how far away it is. You could of coarse multiply by the modelview, then add p0, then multiply by the projection matrix. I certainly don't see anything wrong with that. The problem with any approach like this is that it would be inefficient to do it in one shader (since you essentially need an if statement and uniform mode variable) so you need to switch between two shaders which is fine but as you said before annoying.
My approach was aimed at keeping the same shader for the whole lot. Out of interest, did you look at the point_sprite extension (It's fine if you didn't, I just think it could help you) I got the impression that it let you do the stuff with roundedness and textures that you want.

[RiverC]

I've used point sprites before, but some cards have a little bit odd behavior... I have a point sprite shader routine with textures that works, but I'm looking to do something 'better'.

As for distance attenuation, I'm a bit confused. I would assume the ModelViewProjectionMatrix embeds translation that creates the normal attentuation associated with perspective.

the initial vertex shader looks like this:

uniform vec3 p;
uniform mat4 q;
...
gl_Position=(gl_ModelViewProjectionMatrix*q*p)+gl_Vertex;

would be nice if the fourth part of p, say it's a vec4, could contain an attenuation value (where 1 = full size.)

Sometimes I wish there were frameShaders or vertexGroupShaders.

[RiverC]

I did the following

gl_Position=gl_ProjectionMatrix*((gl_ModelViewMatrix*q*p)+gl_Vertex);

I'll see how it works.

(q is the camera's translation matrix that I calculate in java; it needs only to be calculated once a frame.)

[RiverC]

Hmm, it's not working. I feel like my vertexes, which are now being used as offsets, need to have different values, as what they have right now is offsetting the poly off the screen or some such...

Any idea what the 'actual' size they would need to be? Mine currently should create a 1x1 billboarded polygon, but as soon as I move the math to apply the vertex transformation AFTERWARDS the polygon disappears.

Sort of disappointing, but I'm sure it's just a problem with my math.

UPDATE:

Ah, I get it. The poly I was drawing was reverse-facing because I was turning the camera 180 degrees. Those coordinates (the vertex coordinates I was providing) would show up in normal 3d, since the camera was rotated. But when you put them into the projection space, they would always be 'backside'... when I flipped the camera around and adjusted my vertex coordinates so the test poly would show up with the camera at 0,0,0, suddenly when I moved the gl_Vertex to be an offset, it worked.

Spacial thinking... is hard.

[quew8]

Try hyperspatial thinking. So is it working well now in your opinion. This is actually something I recon I'll do this way. The first time I did it I was a bit of a shader noob and going through a "I'll work everything out for myself" phase. I never even considered shaders for particles until now. So thanks for asking.

[RiverC]

the trick was that (also) I needed a second uniform for the camera position offset. Probably my 'q' matrix doesn't work right when you reverse the order of the operations for model rotation.

Thanks for the tip!

EDIT:

Also good to note: I'm going to need to be able to fix the roll position of the points even though I don't reposition them otherwise. this becomes obvious if you think what happens when you look at the top of a sprite that is a person - when you rotate around the view the sprite stays fixed against the direction the person is facing; thus the sprite quad should be able to rotate even though it doesn't do other transforms. I'm going to see if I can do this in the card or if I should calculate it as I send the points.

[RiverC]

So here's a question to follow up.

- I have the sprite-part's facing (yaw,pitch,roll)
- I have the camera's facing (yaw,pitch,roll)
- What is the best way to get the Normal direction of the sprite based on the camera's facing (I just need a 3-float normal; I can convert it into a nearest-index)
- What is the best way to get the resulting roll?

I would suppose I'm going to do some kind of transformation on the sprite-part's facing based on the camera's facing, but it's a bit of a chin scratcher to figure out what.

EDIT

Can I simply make a new transformation matrix for the object and multiply it by the camera's one to get a final transformation? How would I get a normal out of that, though?

[quew8]

I'm not sure what you mean.
So you have a person being rendered as a single quad and you want him to face in the direction he is looking? Surely that isn't billboarded and you should just render it as normal, not as a sprite?

EDIT:

Or is he being rendered as lots of different quads? But then I'm still not sure what the "Normal direction of the sprite based on the camera's facing" is. Perhaps a little background on what the game (or whatever) actually is. As in 1st person shooter, top down rpg etc.

[RiverC]

Normal is the vector going out of a plane. Sorry, I'm being a little confusing I think, since I don't have a formal background in Computer Graphics, though we did do Linear Algebra... so I know some terms and others I just 'make up'.

Essentially, what I need is to know the direction the sprite is facing after it is transformed for the camera. So for instance, If sprite is facing east and I'm facing east, I will see (more or less) the back of the sprite if it is in camera (setting aside perspective issues for the moment.)

Just billboarding it is one step; now I need to know what direction it faces so I can render the right texture based on the direction you're looking at it.

- Yes, I'll be using a few quads with different locations and orientations to render a person - this makes multitexture for equipment and items easier to parse.

- I say 'normal' because the normalized vector created by the orientation of the sprite is what I need, as if the sprite were a regular quad instead a billboarded one. Since the sprite will show different textures based on what direction it is viewed, I take the 'normal' (like -1,0,0, or 2*sqr(2),-2*sqr(2),0 ) and clamp it into a six bit bitmap, then that bitmap can be used to get the index of the sprite for the proper direction without any additional math.

- I also figured out I can determine the 'roll' by this normal; for instance, if viewed from the top, the sprite's yaw becomes the roll, and the percentage of each rotation you apply is provided by the normal.

Would be nice if I could to some degree account for perspective too, but maybe that will be later when I integrate this more fully into the shader.

EDIT:

As for the type of game, I'm working on an engine. I want the option to use composite sprites in a variety of camera types; we might want to lock to isometric for battle scenes, but return to first or third person for exploration.

When I worked before I ran into the problem of rendering complex objects - items, furniture, people, animals, monsters - and animating them in 3d being very time consuming. So when I returned to the project, I decided to rebuild the engine from scratch and integrate a model viewer, plus change the way I did the modelling; going for something that didn't require manipulation of 3d models. Sure, I will eventually be able to load them into the engine, but the priority for this operation is to create a unique style and be able to actually have some greater control over its feel. I don't get a lot with 3d models as the level of complexity required to get what I really want is prohibitive. I realized that the most difficult part was to get animated characters and objects in the world that didn't look like crap.

I did build a way to convert a png image, as slices, into a voxel model. The style though, felt very rigid.

This also integrates with an idea I have regarding assets for each character - including later hi-res sprites that have the quality of hi-quality flash (see Lauren Faust's reboot of My Little Pony for an example of how good the animation looks with hi quality sprites) that can be integrated into staged scenes, properly including details that normally would have to be manually included in the graphics.

[quew8]

What confused me was you talking about orientating these billboarded sprites. So the normal for them is always going to the negation of the forward vector of the camera. Example:
Matrix: a1  a2  a3  a4     
          b1  b2  b3  b4
          c1  c2  c3  c4
          d1  d2  d3  d4
The "right" vector is (a1, a2, a3), the "up" vector is (b1, b2, b3) and the foward vector is (c1, c2, c3). This becomes clearer is you look at the identity matrix which has no rotation - so the right vector is the x axis, the up vector is the y axis and the forward vector is the z axis.

1, 0, 0, 0
0, 1, 0, 0
0, 0, 1, 0
0, 0, 0, 1

So right is (1, 0, 0) ie x axis, up is (0, 1, 0) ie y axis forward is (0, 0, 1) ie z axis. You should note that the "right" vector is only the right vector is you use a left-handed coordinate system otherwise it is the "left" vector. Confusing right? Also be careful whether you are using column major or row major notation.

Now the orientated vector of the sprite is the negative of this vector. You can get the angle it has been rotated through by the arc-cosine of the dot product of this and the sprites un-orientated normal. acos(dot(origNormal, orientatedNormal)); I think the LWJGL vector classes have a dot product method. But this is not necessarily the roll (btw if it is what I think you mean it is - the rotation around the y axis - then we tend to call it yaw. Roll is about the z-axis). To get the yaw/roll you need to isolate the x and z components of the vectors. So set the y component to 0 and then re-normalize before doing the dot product.

I hope this is clear and what you are actually asking about (fingers crossed). Don't hesitate to question any part of it, I know I've spammed quite a lot.

[RiverC]

I've almost got it.

The rotations were pretty simple once I realized i could just invert the camera (as you suggested.)

However, how to simply handle the transparent parts of the sprite texture? I'm not intending to introduce translucency, so I shouldn't need depth peeling or other expensive techniques. Is it a matter of sending 0,0,0,0 instead of 255,255,255,0 to the fragment shader, or is there a way to make the fragment shader ignore a=0 fragments, perhaps?

---

AH hah! This is quite a find.

mediump vec4 basecolor = texture2D(sTexture, TexCoord);

    if(basecolor.a == 0.0){
        discard;
    }

    gl_FragColor = basecolor;

you can tell the fragment shader to discard fully transparent fragments. So.. useful.

[quew8]

I like it when people figure things out for themselves.   (That is the only time I've ever used a smiley here)