Any advice for writing your own windowing toolkit?

[Cornix]

Hi guys.

I am about to write my own windowing toolkit using lwjgl. I am doing this just for fun, for honor, and maybe for future projects.
But before I start there will be a lot of planning involved. And before I start to plan I would like to ask for some advice on various issues I might run into.
The very first issue I have noticed myself is, that I am not completely sure how I will render all the GUI components.

The easiest solution I came up with was to create a separate VBO for each single component. The component would take care of filling its VBO with vertex data and draw it as necessary. But I guess this would not come with good performance because I would have dozens of very small VBO's, almost like using immediate mode.
So should I rather have one very large VBO which I fill all the components into?

What are your thoughts on this?

Thank you for your help, I really appreciate!

[quew8]

Good luck. I've thought about this many times but always given up when I couldn't come up with an answer that I didn't detest the next day. The best solutions (probably) that I managed to come up with were:

1) Screw VBOs and use vertex arrays.
2) Have GUI components implement float[] getData(), int[] getIndices(), void preDraw() kind of methods, but the top level container being the only one which actually creates a VBO. It iterates recursively (if the two things aren't technically mutually exclusive) through its children filling up a buffer of data and indices. When you draw, you just go through the VBO in order. That way you preserve z-order and hopefully performance.

[Cornix]

Yeah, I was thinking the same way.

Having nested components and they each fill in their vertex data into some kind of GUI manager in the order they are supposed to be drawn. Thats not the biggest problem.
Much worse are things like Scissor-Areas and zooming / translation.
If I have some kind of scroll pane or something, and I add components to it, but the scroll pane is too small to show them completely, I have to somehow scissor out the parts which are outside of the bounds of the pane. If I was using immediate mode it would become pretty simple. But when I have everything in one big VBO I would need to split up draw calls for each scissor. Makes things much more difficult.

I was thinking of maybe emulating immediate mode through the use of a VBO in some kind of GUI-Renderer class.
I could push / pop translations, rotations and scaling of vertex data, and I could define scissor rects. And the GUI-Renderer class does some magic in the background to manage all this data and somehow translate it into VBO-rendering.

[quew8]

In terms of transformations, I think the vast majority are going to be static aren't they. Not much moves around in your average GUI, mostly its different states eg. tabs (which selected), buttons (is pressed). So for most transformations, I would pre-bake them and perform the transform before you put it into the VBO. For states I would probably do either draw this or draw this and just have completely different geometry in the VBO. The rest of it - pushing and popping and translating and rotating etc. - is what I was thinking to do in the preDraw() method.

As for as clipping goes - you could use the scissor box test thing, but I thought that using the stencil test might be a little more dynamic. You're already drawing the parents before the children right? So when you render the parent, draw a value to the stencil buffer. When you draw its children, set the stencil test to test for that value. It becomes a little more complex when you need to do this recursively - ie scroll panes inside scroll panes inside scroll panes (aren't I imaginative) - and you might have to clear the stencil buffer half way through the frame. It depends on the size (I can't think of the right word here - I mean the number of bytes per "pixel") of stencil buffer you're using.

Some food for thought. 

[Cornix]

With the transformation I was referring to things like, if I put a button into a pane the button needs to have the panes x, y coordinates added to its own position. So I thought I could do this as if I had some kind of modelview matrix and the pane would simply do a push(); translatef(x, y); drawAllChildren(); pop(); or something like that.
I guess I could also calculate that statically after components were added to the pane and then I dont have to push / pop stuff because it really shouldnt change that much, but this would be even more complicated. I am going to think some more about this.

I didnt even think about the stencil buffer, I might actually use that. This would cut down the number of draw calls immensely I guess. Thank you very much.

[quew8]

Don't thank me. You're the one who's going to have to implement it.

BTW: Please do post if you come up with a final solution. I'd be interested to know of any problems/solutions you came up with.

[Cornix]

I can tell you my concept so far; although not all of it is implemented at the moment I am fairly sure this will work out this way:

1) Components are structured in a tree. Each component can have multiple children (if appropriate) and each child can only have 1 parent (its a tree, duh!) and only 1 component has no parent, that one is called the root component. Some components (like labels for example) can not have children. They are always leafs.

2) For each component tree there is one GUI-Renderer which takes care of rendering all components in the component tree.

3) When a component has changed in some form or another (a button is pressed, a child was added/removed, textbox had input, etc) it will inform its parent of the change. The parent will inform its own parent and so on until we hit the root component.

4) When the root component is informed of a change in the component tree it will reset the GUI-renderer and have all components in the tree register at the GUI-renderer to refresh the rendering data.

5) As long as nothing has changed in the component tree the GUI-renderer will simply re-use the previous render data. The GUI-renderer will be implemented with a self-resizing VBO.

6) When components are registered at the GUI-renderer they will leave information about their "level" into the component tree (this is used for clipping) and the texture to be used for rendering (or a "no texture" object if no texture is used at all). When the GUI-renderer is rendering the GUI it will render each level of the component tree individually with one draw call per texture that was registered for that level.

7) When a component is updated it is supposed to update all of its children recursively.




Example code: (final product might look differently)

Window wnd = new Window();
wnd.setRelativeLocation(0, 0); // location is relative to the components viewport. Viewport is affected by all ancestors of a component.
wnd.setPreferredSize(640, 480); // actual size might differ because of layouts or viewport restrictions. The actual size of the root will be its preferred size.

Button btn = new Button();
btn.setRelativeLocation(32, 32);
btn.setPreferredSize(128, 32);

Label lbl = new Label();
lbl.setFont(someFontNotImportantForThisExample);
lbl.setText("This is some button");

btn.setContent(lbl); // a button can have labels, pictures, etc as its content

wnd.add(btn); // window has no layout, therefor the button will simply be put at its location with its preferred size

while (true) {
    wnd.update(); // all components will be updated, that will be scroll bars, sliders, buttons, and all that good gui stuff
    wnd.render(); // will render all components in the component tree. Will only upload new vertex data to the graphics card if anything has actually changed since the last render cycle.
}

[Cornix]

Damn. I played around with this a lot in the last couple of days and got something working, but I am not too pleased with the result.
No matter what methods of optimization I tried I always came down to having 1 render call per component. Damn clipping and textures just make it too complicated to batch things together.

Does anyone know of an openGL windowing toolkit that manages to draw the components more efficiently then that?

[CaleyM]

What are you trying to do that TWL doesn't?