resigning applet JARs

[NateS]

I finally got around to trying to create an applet. I need my code signed, so I need to also resign the LWJGL applet JARs. I can sign and pack just fine, but how do I create the LZMA files?

I tried using all unpacked, signed JARs and this works up until it tries to find "windows_natives.jar.lzma" and fails. I tried using 7-zip on windows_natives.jar, but the applet runs out of heap space trying to decode it.

[kappa]

oh didn't know 7-zip could create lzma files thought it was mainly just 7z files (which is a container format for lzma).

However the tool used with LWJGL build system is called JLzma.jar you can get it here.

Its a command line tool but you can have a look at the lwjgl build.xml to get an idea on how to use it with ant.

more tools can be obtained here

hope that helps.

[NateS]

Yeah, 7-zip wasn't the right tool.

Your post was very helpful, thanks! 

Ant!? Pssshf!  I spent yesterday writing a new build tool to replace ant, once and for all.  It is called "Scar" and is, naturally, really sweet. I'll post it on Google Code and JGO for feedback soon, after adding a few more features. It can re-sign all the LWJGL applet JARs in a line of code or two.

[NateS]

Well, using the JLzma.jar from LWJGL... apparently it has a System.exit call!? Building my own JAR from the 7-zip SDK doesn't have this problem.

Scar is coming along nicely.   Check it out:

signLwjglApplet(".", keystore, alias, password);

That unLZMAs the files as needed, removes any existing signature from the JAR, signs the JARs, then LZMAs and/or pack200s the JARs as needed. Ah, you say I just hide all the nastiness behind a method? Nooo! Scar is awesome!  

static public void signLwjglApplet (String dir, String keystoreFile, String alias, String password) throws IOException {
	for (String jarFile : new Paths(dir, "*.jar")) {
		sign(normalize(unsign(jarFile)), keystoreFile, alias, password);
		if (fileExists(jarFile + ".lzma")) lzma(jarFile, jarFile + ".lzma");
	}
	for (String jarLzmaFile : new Paths(dir, "*.jar.lzma")) {
		if (fileExists(substring(jarLzmaFile, 0, -5))) continue;
		String jarFile = sign(normalize(unsign(unlzma(jarLzmaFile))), keystoreFile, alias, password);
		lzma(jarFile, jarFile + ".lzma");
		lzma(pack200(jarFile));
	}
}

[NateS]

Having a bit of trouble. Using my Scar stuff to sign my JARs works fine for JWS:
http://www.esotericsoftware.com/dragon/dragon.jnlp
But it doesn't work for LWJGL:
http://www.esotericsoftware.com/dragon-applet/applet.html
I get:

Exception in thread "AppletLoader.loaderThread" java.lang.ExceptionInInitializerError
	at org.lwjgl.opengl.Display.<clinit>(Display.java:128)
	at com.esotericsoftware.skorpios.opengl.DesktopGLView.setupDisplay(DesktopGLView.java:45)
	at com.esotericsoftware.skorpios.opengl.DesktopGLView.start(DesktopGLView.java:65)
	at com.esotericsoftware.dragon.DesktopDragon.<init>(DesktopDragon.java:14)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at java.lang.Class.newInstance0(Unknown Source)
	at java.lang.Class.newInstance(Unknown Source)
	at org.lwjgl.util.applet.AppletLoader.switchApplet(AppletLoader.java:944)
	at org.lwjgl.util.applet.AppletLoader.run(AppletLoader.java:751)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.SecurityException: SHA1 digest error for org/lwjgl/LWJGLUtil.class
	at sun.security.util.ManifestEntryVerifier.verify(Unknown Source)
	at java.util.jar.JarVerifier.processEntry(Unknown Source)
	at java.util.jar.JarVerifier.update(Unknown Source)
	at java.util.jar.JarVerifier$VerifierStream.read(Unknown Source)
	at sun.misc.Resource.getBytes(Unknown Source)
	at java.net.URLClassLoader.defineClass(Unknown Source)
	at java.net.URLClassLoader.access$000(Unknown Source)
	at java.net.URLClassLoader$1.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(Unknown Source)
	at java.lang.ClassLoader.loadClass(Unknown Source)
	at java.lang.ClassLoader.loadClass(Unknown Source)
	at org.lwjgl.Sys.createImplementation(Sys.java:109)
	at org.lwjgl.Sys.<clinit>(Sys.java:97)
	... 13 more

Any ideas what I am doing wrong?

The JARs aren't signed twice. If it were a signing problem, I'd think it would show up when I use the same signing code for JWS. Maybe it is an LZMA problem? As mentioned above, I had to get the Java LZMA stuff from the 7-zip SDK because the JLzma.jar from LWJGL's build seems to do a System.exit.

[bobjob]

I cant really tell how your stuff works, but judging by the information:
* works for JWS
* doesnt work applet
* error called in org.lwjgl.util.applet.AppletLoader.switchApplet


So Im guessing the applet loader cannot locate the classes in the LWJGL.jar, as they are possibly loaded into a different class loader (than expected). By that I mean, not the classloader located in the applet loader class.

its most likely a signing issue, as the appletloader uses a custom class loader to handle the non-loader applet packages.

For testing purposes maybe:
* make sure all packages are signed by the same certificate;
* make sure there is no unsigned jar files
* make sure not to add any mixed code information in the Manifest file (trusted-library...)
* when "packing" the jars make sure to repack them first before signing.
* try run the applet without the use of LZMA and .pack.

[NateS]

Thanks for your help bobjob!

For testing purposes maybe:
1) make sure all packages are signed by the same certificate;
2) make sure there is no unsigned jar files
3) make sure not to add any mixed code information in the Manifest file (trusted-library...)
4) when "packing" the jars make sure to repack them first before signing.
5) try run the applet without the use of LZMA and .pack.

1) Check.
2) Check.
3) Check.
4) Check.
5) Good idea. Tried without LZMA, same failure. Tried without pack, success!

I was doing this...

pack200" --repack jarFile
// sign
pack200 --no-gzip --segment-limit=-1 --no-keep-file-order --effort=7 --modification-time=latest packedFile jarFile

I changed to just pack200 then unpack200 and it works. Cool!

New question... exactly what files do I need? Eg, the LWJGL applet download comes with natives*.jar and natives*.jar.lzma. Do I really need both in the directory if I am using LZMA in my HTML? Maybe a better way to ask... here are all my files (I got by following the LWJGL applet download as an example), which do I not need if I'm using pack and LZMA?

dragon.jar.lzma
dragon.jar.pack.lzma
lwjgl.jar.lzma
lwjgl.jar.pack.lzma
lwjgl_util_applet.jar
lzma.jar
natives-linux.jar
natives-linux.jar.lzma
natives-mac.jar
natives-mac.jar.lzma
natives-solaris.jar
natives-solaris.jar.lzma
natives-win.jar
natives-win.jar.lzma

[kappa]

You should always repack when using pack200.

The LWJGL Applet download package is a bit misleading as it includes lots of extra files that you don't need, I'll see if I can sort it out before the 2.6 release.

Anyway you only need one version of a file either .jar, .jar.pack, .jar.lzma or .jar.pack.lzma. No point including every type since in the end they are all extracted down to just .jar.

The following should be fine, if your using lzma and pack200.

dragon.jar.pack.lzma
lwjgl.jar.pack.lzma
lwjgl_util_applet.jar
lzma.jar
natives-linux.jar.lzma
natives-mac.jar.lzma
natives-solaris.jar.lzma
natives-win.jar.lzma

If you don't want to use LZMA and Pack200 (I usually don't when initally creating applets as its faster to just make jar's) then you'd need the following:(note lzma.jar isn't needed)

dragon.jar
lwjgl.jar
lwjgl_util_applet.jar
natives-linux.jar
natives-mac.jar
natives-solaris.jar
natives-win.jar

[NateS]

Awesome, that is what I thought but wanted to double check since other people can use Scar.

Here is my final Scar build script for making an applet:

main: com.esotericsoftware.dragon.AppletDragon
---
clean(project);
compile(project);
jar(project);
dist(project);
lwjglApplet(project, "Esoteric Software", "Dragon");

[NateS]

To be perfectly clear, I don't need to keep the *.jar.pack files, only the *.jar.pack.lzma files? Everyone has pack everywhere? Why have I gone to the trouble to have Apache serve either packed or unpacked JARs for my JWS based on a header? Is that not needed anymore?

[kappa]

To be perfectly clear, I don't need to keep the *.jar.pack files, only the *.jar.pack.lzma files? Everyone has pack everywhere? Why have I gone to the trouble to have Apache serve either packed or unpacked JARs for my JWS based on a header? Is that not needed anymore?

grab the latest lwjgl 2.6 applet download package from https://www.newdawnsoftware.com/hudson/view/LWJGL/job/LWJGL/
it should make things much clearer and easier to pick up.

and the answer is no, you don't need to keep *.jar.pack

[NateS]

Gah, I asked the question wrong... meant to ask if I needed both *.jar.lzma and *.jar.pack.lzma. Anyway, you are right, it is much easier to understand after seeing the latest LWJGL applet package. Scar has been updated to do the right thing when making an LWJGL applet.

[kappa]

don't let the extension parts confuse you. All the files are essentially uncompressed to just jar. so you only need one format out of .jar, .jar.pack, .jar.lzma, .jar.pack.lzma and not multiple copies of it.