SwingGL - Swing on top of OpenGL

[raft]

hello,

for a few days i've been working on a crazy idea: making Swing work on top of OpenGL. which i think will be very useful if we make it run smoothly. so i've downloaded the jdk source and started digging into it

i first tried rendering, extended awt.Graphics2D and implemented some methods of it. luckily swing seems to use only a small portion of that methods and most can be easily implemented with OpenGL commands. there are some differences though. anyway, below screenshot will give an idea.. the main trick is making top level component believe that it's placed in a container and is visible. than it happily painted itself and its children on given GL based Graphics implementation

second i tried forwarding mouse events from LWJGL display to Swing. this one was hard. all major classes and methods in this process are package private. i still couldnt find a 'normal way' but made a nasty hack: set a private field of some class with reflection. this will almost certainly wont work on a non-Sun JVM and will possibly break in the future. but still proofs, with a proper interleave point making swing event system work offline is possible. i believe, with implementing a custom awt.Tookit and a dummy peer of some component, making event system work without reflection hack is possible. i just couldnt figure out how. maybe some Swing/AWT guru may help ?

later turn will come to keyboard events and keyboard focus. similarly i believe this would be possible with customizing toolkit..

i'm not sure i will have time and continue this project. the code is a mess now, i will publish it when i clean it up. maybe somene will continue it ? if anyone is interested, i will happily go into details..

here is a a web startable link. i tried it on Ubuntu Hardy and Win XP, it worked ok altough there are some small rendering differences. i especially wonder if it will work ok on MacOSX

below is a screenshot, SwingGL running on HelloWorld example (GL one) of jPCT


cheers 
r a f t

note: i've also posted this to jPCT forum. yes, crosspost but i guess this may interest users of both project

[raft]

i've cleaned up the code and setup a Google-code project: http://code.google.com/p/swing-gl/
all source and a demo can be found there. demo still uses jPCT but rendering is completely on top of LWJGL now.

any contributions are welcome

[Fool Running]

Out of curiosity, what type of FPS do you get? I assume you won't ever get full rendering speed (like it was completely designed for OpenGL). There was a post a long time ago (I'll see if I can find it) where someone did a similar thing, but it only got a max of 30-40 FPS (on my machine at the time) and was unusable for rendering a game GUI.

EDIT: Found it (Swingle in one of the posts): http://lwjgl.org/forum/index.php/topic,2474.0.html

I also looked at it again and realized that the test app has v-sync enabled I'm getting over 1k FPS (on a newer machine) with it now.

[raft]

well i haven't made any FPS tests indeed but i dont think it will effect performance too much. i guess it will be as fast as FengGUI. there might be a very small handicap due to Swing's continuous object creation but again i dont think it will effect performance much

i will look into that Swingle think when i have time

[raft]

i've had a look at Swingle. very impressive

seems as we followed two different approachs: Swingle renders to an image and draws that image to screen by GL. i use a GL based Graphics2D implementation and directly render to it. mine is a more suitable/flexible approach IMHO. for example, without a background one cannot get semi transparent panels as in the below screenshot. it's my Swing/SW renderer app and one of my future goals is to port it to OpenGL



however Swingle seems to have solved the event dispatching problem. i guess combining both approaches will give the optimum result 

unfortunately Kev posted to that thread as a guest, is he here ? maybe he wants to contribute to SwingGL ?

[Schnitter]

Really interesting, but I get the following error when I want to run your demo:

java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.sun.javaws.Launcher.executeApplication(Launcher.java:1302)
	at com.sun.javaws.Launcher.executeMainClass(Launcher.java:1248)
	at com.sun.javaws.Launcher.doLaunchApp(Launcher.java:1066)
	at com.sun.javaws.Launcher.run(Launcher.java:116)
	at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.UnsatisfiedLinkError: /home/schnitter/.java/deployment/cache/6.0/57/625b9ef9-1eb9c5e1-n/liblwjgl.so: /home/schnitter/.java/deployment/cache/6.0/57/625b9ef9-1eb9c5e1-n/liblwjgl.so: wrong ELF class: ELFCLASS32 (Possible cause: architecture word width mismatch)
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1778)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1687)
	at java.lang.Runtime.loadLibrary0(Runtime.java:823)
	at java.lang.System.loadLibrary(System.java:1030)
	at org.lwjgl.Sys$1.run(Sys.java:72)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.lwjgl.Sys.doLoadLibrary(Sys.java:65)
	at org.lwjgl.Sys.loadLibrary(Sys.java:81)
	at org.lwjgl.Sys.<clinit>(Sys.java:98)
	at org.lwjgl.opengl.Display.<clinit>(Display.java:129)
	at com.threed.jpct.GLHelper.findMode(Unknown Source)
	at com.threed.jpct.GLHelper.init(Unknown Source)
	at com.threed.jpct.GLRenderer.init(Unknown Source)
	at com.threed.jpct.FrameBuffer.enableRenderer(Unknown Source)
	at com.threed.jpct.FrameBuffer.enableRenderer(Unknown Source)
	at com.threed.jpct.FrameBuffer.enableRenderer(Unknown Source)
	at demo.HelloWorldOGL.loop(HelloWorldOGL.java:72)
	at demo.HelloWorldOGL.main(HelloWorldOGL.java:50)
	... 9 more

Looks like a problem with my architecture(64bit), but shouldn't that be fixed with the current 64bit native libraries?(As other lwjgl programs also work)

[raft]

64bit linux natives weren't in the jar, that's why it failed. thank you for remider. it should work now

[Fool Running]

... i use a GL based Graphics2D implementation and directly render to it. mine is a more suitable/flexible approach IMHO. for example, without a background one cannot get semi transparent panels as in the below screenshot. it's my Swing/SW renderer app and one of my future goals is to port it to OpenGL

I agree that creating a GL based Graphics2D implementation is the way to go. It may not end out as fast as rendering to an image, but it is definitely more flexible. I hope you get it working well. It would save a lot of headache for people to just be able to render swing as a GUI for their games.

[raft]

in a static (not chaning much) gui rendering to image may perform better. but in dynamic gui, i doubt it, due to continuous texture uploading

anyway, i'm not sure i will have time for this project, that's why i shared it as an open source project. so feel free to contribute despite my 3d application i'm a newbea at Open GL. (jPCT hides all complexity) so you guys do very well help for implementing GL based Graphics2D 

[jpp]

This looks very impressive. raft, have you been working on this project lately?

I've tried the web start demo on my Mac, but it fails with this exception:


java.lang.UnsupportedOperationException
   at org.swinggl.GLGraphics.getFontMetrics(GLGraphics.java:368)
   at java.awt.Graphics.getFontMetrics(Graphics.java:240)
   at com.apple.laf.AquaButtonUI.layoutAndGetText(AquaButtonUI.java:327)
   at com.apple.laf.AquaButtonUI.paint(AquaButtonUI.java:291)
   at javax.swing.plaf.ComponentUI.update(ComponentUI.java:153)
   at javax.swing.JComponent.paintComponent(JComponent.java:752)
   at javax.swing.JComponent.paint(JComponent.java:1029)
   at javax.swing.JComponent.paintChildren(JComponent.java:862)
   at javax.swing.JComponent.paint(JComponent.java:1038)
   at javax.swing.JComponent.paintChildren(JComponent.java:862)
   at javax.swing.JComponent.paint(JComponent.java:1038)
   at javax.swing.JComponent.paintChildren(JComponent.java:862)
   at javax.swing.JComponent.paint(JComponent.java:1038)
   at org.swinggl.GLFrame.paint(GLFrame.java:90)
   at org.swinggl.GLFrame.paint(GLFrame.java:79)
   at demo.HelloWorldOGL.loop(HelloWorldOGL.java:86)
   at demo.HelloWorldOGL.main(HelloWorldOGL.java:50)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at com.sun.javaws.Launcher.executeApplication(Launcher.java:1812)
   at com.sun.javaws.Launcher.executeMainClass(Launcher.java:1750)
   at com.sun.javaws.Launcher.doLaunchApp(Launcher.java:1532)
   at com.sun.javaws.Launcher.run(Launcher.java:135)
   at java.lang.Thread.run(Thread.java:637)


How could this be fixed?

Cheers,
J.-P.

[raft]

no, i dont work on SwingGL and no future plans

on linux and windows, getFontMetrics() is not called in demo. but apperantly this is not the case for mac. FontMetrics is an abstract class but has no abstract methods. you may try to subclass it and return an instance..

[Matthias]

I suggest TWL as a 100% pure Java + OpenGL GUI system with excellent theme ability.