Executable JAR creation with LWJGL

[Vrunk]

Hey everyone,

I've been looking for a way to create an executable JAR file with LWJGL for a while now, in order to be able to run my game without a batch script (without the -Djava.library.path option). I have come across a method that works, although it is a bit of a hack. (uses Reflection and requires you to know the internals of java.lang.ClassLoader)

Add this to your application's Main-Class (the application's entry point, as specified by the JAR's manifest)

    static {
		String s = File.separator;
		// Modify this to point to the location of the native libraries.
		String newLibPath = System.getProperty("user.dir") + s + "lib" + s + "native";
		System.setProperty("java.library.path", newLibPath);

		Field fieldSysPath = null;
		try {
		    fieldSysPath = ClassLoader.class.getDeclaredField("sys_paths");
		} catch (SecurityException e) {
		    e.printStackTrace();
		} catch (NoSuchFieldException e) {
		    e.printStackTrace();
		}

		if (fieldSysPath != null) {
		    try {
			fieldSysPath.setAccessible(true);
			fieldSysPath.set(System.class.getClassLoader(), null);
		    } catch (IllegalArgumentException e) {
			e.printStackTrace();
		    } catch (IllegalAccessException e) {
			e.printStackTrace();
		    }
		}
    }

An ant script to create your JAR file with the proper manifest.

<project name="game" basedir="." default="jar">
	<!-- Change these to be compatible with your project. -->
	<property name="dir.library" location="lib" />
	<property name="dir.bin" location="bin" />
	<property name="jar.name" value="game.jar" />
	<property name="main.class" value="com.Game" />
	<!-- Set this to wherever your libraries will be located, relative to ${jar.name} -->
	<property name="classpath.prefix" value="lib" />

	<path id="CLASSPATH">
		<fileset dir="${dir.library}" includes="**/*.jar" />
	</path>

	<target name="jar">
		<!-- Convert project class path to a string property -->
		<pathconvert property="CONVERTED_CLASSPATH" pathsep=" ">
			<path refid="CLASSPATH" />
			<chainedmapper>
				<flattenmapper />
				<!-- For example: converts "lwjgl.jar" to "lib/lwjgl.jar" -->
				<globmapper from="*" to="${classpath.prefix}/*" />
			</chainedmapper>
		</pathconvert>
		<!-- Make the jar -->
		<jar destfile="${dir.bin}/${jar.name}">
			<manifest>
				<attribute name="Built-By" value="${user.name}" />
				<attribute name="Main-Class" value="${main.class}" />
				<attribute name="Class-Path" value="${CONVERTED_CLASSPATH}" />
			</manifest>
			<fileset dir="${dir.bin}" />
		</jar>
	</target>

</project>

I'll be happy to answer any questions people have about this, but it's nice to have finally found a way for JAR deployment without a batch script (just double click the jar to start your application)

Cheers

[wolf_m]

What I don't like about your method is that you're going via user.dir.

I'm going to tell you how I do it (LWJGL 1.4 here) so you have an alternative without that.

I use the FatJar extension in Eclipse.

Only my code goes into the JAR.

I tell FatJar via the interface in what relative path outside of the app JAR the lwjgl JARs are, that path is called 3rdparty. It writes that path as Class-Path into the manifest.

This is what I write for Class-Path:

. 3rdparty/ 3rdparty/lwjgl.jar 3rdparty/lwjgl_util.jar

FatJar has a settings file and can generate ant scripts. That's really nice because I hate configuration details.

I have my native libraries in a folder outside of the app JAR called "3rdparty". If you instead put them into the same folder your app JAR is in though, you're already done, LWJGL finds everything. But that's kind of ugly.

If you put 'em into a separate folder, you have to tell LWJGL where to look for them.

Matthias gave me some pointers on how to do that. This is the code for it, run upon startup:

// Check whether I'm running from inside a jar
ClassLoader mCL = ClassLoader.getSystemClassLoader();
if( mCL instanceof URLClassLoader ){
  URL[] urls = ((URLClassLoader)mCL).getURLs();
  for ( URL url: urls ) {
      // APP_TITLE has to be set to the filename of your app JAR minus extension beforehand
      if(url.getFile().endsWith( APP_TITLE+".jar" )){
        // If we end up here, we're running inside a JAR, so we have special lib paths
        File thirdp = new File("3rdparty"); // Or some other folder
        System.setProperty( "org.lwjgl.librarypath", thirdp.getAbsolutePath() );
        break;
      }
    }
}

The system property there is used by LWJGL upon first use of native methods, a fact I realized after looking at the source.

The beauty of this method is that your app only looks for those paths if it's running from inside a JAR, which gives you the opportunity to have differing paths in your code environment.

It appears this doesn't work for libIL though, I'm not sure why that is because it DOES include the property's value in its search path. A workaround would be putting libIL stuff into the same folder the app JAR resides in.
Upon switching to LWJGL2, that doesn't matter though. If you use Java's methods for loading images, it's also not a problem.

What I also haven't tried is adding the IL paths explicitly to the Class-Path, I'll do that now, maybe that already does the trick.

[wolf_m]

Mentioning IL stuff explicitly in Class-Path does in fact work!

So my Class-Path has become way bigger, but there's no problem with that in my opinion - you only set it once anyway and then save it to your fatjar settings file, it's not redundant, results in painless deployment, ...:

. 3rdparty/ 3rdparty/lwjgl.jar 3rdparty/lwjgl_util.jar 3rdparty/lwjgl_devil.jar 3rdparty/libILUT.so 3rdparty/liblwjgl-devil.so 3rdparty/libIL.so 3rdparty/libILU.so 3rdparty/libILU.dylib 3rdparty/liblwjgl-devil.jnilib 3rdparty/libILUT.dylib 3rdparty/libIL.dylib 3rdparty/lwjgl-devil.dll 3rdparty/ILU.dll 3rdparty/DevIL.dll 3rdparty/ILUT.dll

[Vrunk]

FatJar is one of the first things that I tried, but I didn't like it very much. I use ant scripts to compile, jar, obfuscate, and deploy my application. There are simply too many steps in my build process for me to consider FatJar as an adequate solution to the problem (and I find it quite ugly to package jar files inside of jar files). Is there a specific disadvantage to using user.dir? If there is, there must be a better way to get the directory that the jar you just double clicked on is located in. If anything, the user could always point to the location of the libraries in a config file deployed with the application.

Also, I know that LWJGL takes care of loading the appropriate library during startup, based on the OS.
Is the property that you mentioned (org.lwjgl.librarypath) the first place that LWJGL looks for the libraries?
If so, that would make for a much simpler solution than the one I listed above.

[wolf_m]

FatJar is one of the first things that I tried, but I didn't like it very much. I use ant scripts to compile, jar, obfuscate, and deploy my application.

You can generate and then alter ant scripts with FatJar. There's basically a "Export ANT..." button.
I just use it for packaging my own source and for putting the Class-Path into the manifest, really. If your app is way more complicated than that, FatJar might not be for you, agreed. It's just a usability enhancement for me, really.

There are simply too many steps in my build process for me to consider FatJar as an adequate solution to the problem (and I find it quite ugly to package jar files inside of jar files).

I also find it quite ugly to do this, hence I don't do it.I realize one could use FatJar for doing this, but I find it wrong. And some licenses don't allow repackaging, I think.
So we're on the same page there.

Is there a specific disadvantage to using user.dir? If there is, there must be a better way to get the directory that the jar you just double clicked on is located in. If anything, the user could always point to the location of the libraries in a config file deployed with the application.

The thing is it's kind of hard to rely on the accuracy of user.dir for all platforms, at least the way I see it. And you can't guarantee that this property is here to stay concerning Java's future. Maybe there are other disadvantages I haven't thought of right now.

Also, I know that LWJGL takes care of loading the appropriate library during startup, based on the OS.
Is the property that you mentioned (org.lwjgl.librarypath) the first place that LWJGL looks for the libraries?
If so, that would make for a much simpler solution than the one I listed above.

I think so.. Mind you, I only looked at 1.4 source. To confirm for releases 2.x, you'd have to check the source there.

Actually, I just did, it's still there, in String[] LWJGLUtil.getLibraryPaths(String libname, String platform_lib_name, ClassLoader classloader)
http://java-game-lib.svn.sourceforge.net/viewvc/java-game-lib/trunk/LWJGL/src/java/org/lwjgl/LWJGLUtil.java?view=markup
Line 355

Edit: Okay, it's not necessarily the first place, actually. But it's one of the places, which should be good enough.

[wolf_m]

Oh, and if you look at line 370, you can see that lwjgl actually includes user.dir itself, that's probably why you can just place the native libs into the same dir as your JAR and it magically works.

[Vrunk]

Oh, and if you look at line 370, you can see that lwjgl actually includes user.dir itself, that's probably why you can just place the native libs into the same dir as your JAR and it magically works.

Wasn't aware of that one either -that actually makes my solution somewhat redundant, haha. (unless you want the native libraries in a separate folder than the jar).

So thanks for the responses, and the clarification on the org.lwjgl.librarypath property. It's always nice to get a different perspectives on how things can be done. But, I think I'll stick with my motto for now: "If it ain't broke, don't fix it." So until somebody finds a way to break my app, I'll stick with my ANT script .

[wolf_m]

Oh, and if you look at line 370, you can see that lwjgl actually includes user.dir itself, that's probably why you can just place the native libs into the same dir as your JAR and it magically works.

Wasn't aware of that one either -that actually makes my solution somewhat redundant, haha. (unless you want the native libraries in a separate folder than the jar).

So thanks for the responses, and the clarification on the org.lwjgl.librarypath property. It's always nice to get a different perspectives on how things can be done. But, I think I'll stick with my motto for now: "If it ain't broke, don't fix it." So until somebody finds a way to break my app, I'll stick with my ANT script .

I'd do just the same, actually Just wanted to give you a valid alternative.

[Ciardhubh]

I use the "org.lwjgl.librarypath" property and it works fine so far. Here's my code in case somebody is interested:

        /* Set lwjgl library path so that LWJGL finds the natives depending on the OS. */
        String osName = System.getProperty("os.name");
        // Get .jar dir. new File(".") and property "user.dir" will not work if .jar is called from
        // a different directory, e.g. java -jar /someOtherDirectory/myApp.jar
        String nativeDir = "";
        try {
            nativeDir = new File(this.getClass().getProtectionDomain().getCodeSource().getLocation().
                    toURI()).getParent();
        } catch (URISyntaxException uriEx) {
            try {
                // Try to resort to current dir. May still fail later due to bad start dir.
                uriEx.printStackTrace();
                nativeDir = new File(".").getCanonicalPath();
            } catch (IOException ioEx) {
                // Completely failed
                System.out.println("Failed to locate native library directory. Error:\n" + ioEx.toString());
                ioEx.printStackTrace();
                System.exit(-1);
            }
        }
        // Append library subdir
        nativeDir += File.separator + "lib" + File.separator + "native" + File.separator;
        if (osName.startsWith("Windows")) {
            nativeDir += "win32";
        } else if (osName.startsWith("Linux") || osName.startsWith("FreeBSD")) {
            nativeDir += "linux";
        } else if (osName.startsWith("Mac OS X")) {
            nativeDir += "macosx";
        } else if (osName.startsWith("Solaris") || osName.startsWith("SunOS")) {
            nativeDir += "solaris";
        } else {
            System.out.println("Unsupported OS: " + osName + ". Exiting.");
            System.exit(-1);
        }
        System.setProperty("org.lwjgl.librarypath", nativeDir);

This code assumes that the natives are in subdirectories in the same directory as the .jar-file, e.g. <jar-file-dir>/lib/native/win32. The property "user.dir" and new File(".") can point to other directories if you start the .jar from another directory, so I've used this.getClass().getProtectionDomain().getCodeSource().getLocation().

[Vrunk]

Interesting code that you have there. Although, the following code really comes down to preference:

        // Append library subdir
        nativeDir += File.separator + "lib" + File.separator + "native" + File.separator;
        if (osName.startsWith("Windows")) {
            nativeDir += "win32";
        } else if (osName.startsWith("Linux") || osName.startsWith("FreeBSD")) {
            nativeDir += "linux";
        } else if (osName.startsWith("Mac OS X")) {
            nativeDir += "macosx";
        } else if (osName.startsWith("Solaris") || osName.startsWith("SunOS")) {
            nativeDir += "solaris";
        } else {
            System.out.println("Unsupported OS: " + osName + ". Exiting.");
            System.exit(-1);
        }

Personally, I would either
1) Build a separate jar for each OS and only distribute the appropriate native libraries for the OS, or
2) Put all of the native libraries in a common folder (lib/native)

this.getClass().getProtectionDomain().getCodeSource().getLocation();

Never tried using that before. You would think that Sun would provide an easy way to determine the directory where the currently executed jar is located. I have yet to find a way that is both easy and 100% accurate for all operating systems.

Other than that, it's good to know that someone is using org.lwjgl.librarypath with success.

[wolf_m]

Other than that, it's good to know that someone is using org.lwjgl.librarypath with success.

You mean someone who doesn't carry the forum handle wolf_m, eh? 

[Vrunk]

Other than that, it's good to know that someone is using org.lwjgl.librarypath with success.

You mean someone who doesn't carry the forum handle wolf_m, eh? 

Someone other than wolf_m, if you will

[broumbroum]

I've been looking for a way to create an executable JAR file with LWJGL for a while now, in order to be able to run my game without a batch script (without the -Djava.library.path option). I have come across a method that works, although it is a bit of a hack. (uses Reflection and requires you to know the internals of java.lang.ClassLoader)
(...)
I'll be happy to answer any questions people have about this, but it's nice to have finally found a way for JAR deployment without a batch script (just double click the jar to start your application)

Cheers

Here's my method to redirect the libpath property at runtime, which works well with self-executable files :

/** updates the current {@link System#getProperties() java.library.path}
    @param path the path to add
    @see System#setProperty(String, String)*/
    private void updateLP(String path) {
        boolean found = false;
        for (String regPath : libraryPath) {
            if (regPath instanceof String) {
                if (regPath.equals(path)) {
                    found = true;
                    break;
                }
            }
        }
        if (!found) {
            System.setProperty("java.library.path", System.getProperty("java.library.path") + File.pathSeparator + path);
            libraryPath = java.lang.management.ManagementFactory.getRuntimeMXBean().getLibraryPath().split(File.pathSeparator);
            System.out.println("library.path updated to : " + java.lang.management.ManagementFactory.getRuntimeMXBean().getLibraryPath());
        }
    }

BTW, why this manner of changing the libpath CAN'T WORK WITH APPLET ON SAFARI AND WINDOWS AS WELL AS WITH JNLP ? It has worked for some time ago with Java 1.5 and now that doesn't work with applet.