LWJGL Forum

Hi, I am trying to get lwjgl deployed to the maven central repository, this would allow people using maven to have easy access to working with lwjgl, but also as a lot of libraries use lwjgl as a dependency (slick2d, nifty, twl, jmonkeyengine, ardor3d, libgdx, etc) it would make it possible for those to be deployed to the maven central repository as well.

What I am trying to accomplish doesn't involve migrating the lwjgl build to maven or anything like that, the proposed changes try to be the as minimal as possible.

In order to do that there are some requirements imposed by the people who manage the central repo.

   
• All the project dependencies should be in the Maven Central Repository
• For each jar with java classes there should be an attached jar with javadocs
• For each jar with java classes there should be an attached jar with sources
• All project artifacts are signed using GPG

To be able to fulfill these requirements I need to apply a patch to the current build in order to make some definitions in build-definitions.xml a little more fine grained. (for example to be able to tell which natives are for lwjgl + openal and which are for jinput)

In a local git repo I already have an extra build-maven.xml that makes all the extra artifacts, copies the poms, and signs everything, but that still needs to be validated depending on how is the lwjgl release process.

I am attaching the patch for the build-definitions.xml I tested it and it seems to not break anything but please check it out, and if everything is ok it would be great if you could apply it so I can move on to the rest of the steps.

EDIT: This is a link to the instructions and requirements for uploading to maven central just for reference -  https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

thanks

+100

I'd really like to have LWJGL available in the central maven repo! It would allow so much easier access to LWJGL for all Maven users and it will allow third party libraries that use LWJGL to be available in the Maven Central Repo as well! That would be really really really great! :)

patch looks fine and a build test didn't blow up :) - patch applied. Thanks.

Hi, I want to share how this task is advancing, I made an extension to the build that generates a zip file that contains the needed artifacts to make the deployment together with an extra build.xml file that executes the deployment into a maven repository. In the current version it just deploys to a /tmp/m2repo file repository.

In order to generate the zip you just need to apply this patch to the latest version from svn.
This will add two ant targets:
   ant maven-full:  makes a build from scratch
   ant maven: it is supposed to be used after the regular lwjgl build

in both cases you need to download the natives from the hudson builds (I will attach a little script to do that in linux)

When you generate the zip, it is placed in the dist directory as lwjgl-maven-2.6.zip

To perform the distribution you need:

• To install the maven ant tasks (http://maven.apache.org/ant-tasks/installation.html): just save the jar into your ant lib dir, or in your home .ant/lib
• Generate a gpg sign key: using "gpg --gen-key"
• Add a profile to your maven .settings file in ${home}/.m2/ directory declaring the passphrase used for the gpg key
  
  
  
  • If you know how to do that look at the sample settings.xml and copy the profile declaration changing the passphrase to match the one used to generate the key
  • If you don't know just use the example settings.xml changing the passphrase to match the one used to generate the key

Then to perform the distribution just unzip the lwjgl-maven zip somewhere and inside there is a maven dir with a build.xml file, in that directory, just run "ant deploy" this will sign with your gpg key and deploy everything to a local repo in your /tmp/m2repo dir.

If you have any question, or if anything fails don't hesitate to ask me for help, here or in irc.

thanks

nice. I will look into it this weekend. In between some LoL sessions, it would be nice to get this out with a 2.7 release - if time permits.
What about the jinput version issue?
How are the lwjgl dependencies such as lzma.jar and AppleJavaExtensions.jar handled ?

Hi, I made another patch in order to add jinput to the maven distribution.

With this patch jinput will be deployed as a subproject of lwjgl it is not an ideal solution, but for now is the easiest one, and gives us a fully functional lwjgl maven distribution.

The new patch will create the native jars for jinput and has the poms to deploy all the jinput artifacts to a repo. Also jinput is added as a dependency of lwjgl in the lwjgl pom.

Also the new patch adds support to install the distribution in the current user local maven repository as well as deploying to a repo.

If you want to make some test and generate a custom version of lwjgl there is a property called "lwjgl-maven-version" in platform_build/build-maven.xml you have to set that property to the version you want before generating the maven distribution, currently there is no way to change that after the distribution is generated but before installing or deploying (other than manual edit of the generated poms). In the future I want to make this property calculated from the "lwjgl.version" property (like transform from 2.7 to 2.7.0 and adding -SNAPTSHOT to the nightly releases).

To answer Matzon questions in his last post:

• The AppleJavaExtensions.jar is used only at build time, so it doesn't have an impact in the maven distribution.
• The lzma.jar is used as a deployment tool with the appletloader, but not as a transitive dependency to other projects that use lwjgl so for now it is not handled in the maven distribution.
• About jinput, I have been unable to find any info about version numbers for jinput, or a release process that allows to match the jinput.jar used in lwjgl to a stable jinput release. This fact, and that I have been unable to get in touch with endolf made me add jinput as part of the lwjgl release, instead of an external dependency. This will work fine for now, but is not an ideal solution at least according to maven best practices.

Extra github info.

In order to make this development a lot easier we made (with arielsan) a lwjgl github repo that is synched to the latest svn version (https://github.com/gemserk/lwjgl) and the latest work with the maven distribution can be seen in my fork in the branch maven (https://github.com/rgarat/lwjgl/tree/maven). There is also a page where you can see the difference between the latest svn version I merged to my master branch on my repo (doesn't have to be the actual latest svn version) and the latest work on the maven branch (https://github.com/rgarat/lwjgl/compare/master...maven).

I am a complete git noob so I can't promise that the gemserk/lwjgl repo will stay in sync without any hiccups, or that it won't have to be deleted and rebuild (I already screwed up once, and had to start from scratch).

Ruben

Looks good. Will apply tonight.

At some point we need to look into pushing this into the central repo.

postponing until 2.7.1 is out - dont want other changes in

no problem, good luck with the openal fix

Hi

I replied to your DM on twitter last week, not sure if you got it. If memory serves, to get in to the central repo you have to control the domain that is the same as your package structure, in the case of jinput, that is java.net, which I have no access/control over, which is why I gave up on sticking jinput in the central repository when I was looking at this a year or so ago.

I tweaked JInput so that the results of a normal build can be shoved in to a maven repository, and I set up one on newdawnsoftware.com for jinput

http://www.newdawnsoftware.com/maven2/net/java/games/jinput/

Not sure this helps much, as I can't get jinput into the central repo.

Endolf

patch applied

doh, with attribution to arielsan instead of you - doh - for that commit, still in the credits file :)

sorry for the delay, had some family visiting over

endolf: about the requirements to get a project into maven, the groupId should be a valid domain, but you don't have to own it for example you can have com.googlecode.myproyect or in jinput case net.java.jinput or something like that.

here are the requirements and process to get a project into maven central https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

about getting jinput to maven central, I have identified a couple of issues:

• jutils is added to the jinput jar, should it be an external dependency?
• a sources.jar and javadoc.jar is required to get into maven central
• we are using a similar but different nomenclature for the natives jars
  
  
  
  • they are jars not zips
  • we are using the format natives-${OS} like natives-win, natives-mac, natives-linux for the maven classifier, then you can use our maven plugin (http://code.google.com/p/mavennatives/) (coming soon to maven central) to extract the natives content automatically
• all the jars have to be signed with gpg

another issue with how lwjgl uses jinput, is that right now lwjgl can take a nightly that has a needed fix, and update the lwjgl distribution to use that, by having lwjgl as an external dependency they would need a formal jinput release.

We can rip jutils out during a maven build and add that to the central maven repo too

Endolf

I added support for generating the maven version from the lwjgl.version property.

This is needed because maven versions should be like <major version>.<minor version>.<incremental version>-<qualifier> (with the qualifier being optional) and currently what would be for example the 2.7.0 version in maven land is 2.7 in lwjgl.

The patch uses a task made in javascript to add the extra ".0" to the version if it is Number+.Number+. if the lwjgl.version has another format it leaves it untouched (this makes it easier to make temporal builds like 2.7.1-test1 or stuff like that and being able to install it locally or in a personal repository).

In this patch I could not use antcall to invoke the custom task because antcall generates a new project environment to execute the task, so any changes made to properties in the called task are not seen by the caller. So I had to make the maven task depend on the custom one.

Hi endolf I made a patch for the jinput build that generates everything needed for deployment into maven central

The patch is made of

• build-maven.xml
• maven/jinput.pom
• maven/build.xml
• maven/jinput-parent.pom
• maven/jinput-platform.pom

In order to make the maven build you need to download the natives and jutil.lib (as is done now in the hudson build) execute the normal build with the dist and javadoc targets to generate the jutils.jar and the javadoc.
Then running "ant -f build-maven.xml mavendist" a directory dist-maven will be created and in there it will create:

• jinput-sources.jar
• jinput-javadoc.jar
• copy of the jinput poms with the current version
• build.xml a copy from maven/build.xml this supports local repo install and signing and deployment to a repo
• copy of the maven-ant-tasks-2.1.0.jar so that dist-maven is a complete unit of work

In order to be able to deploy to maven central you have to request access to the oss sonatype people as explained here (https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide) using as groupId "net.java.jinput"

I got a lib into maven central that is hosted in google code using the groupId "com.googlecode.mavennatives" here (https://issues.sonatype.org/browse/OSSRH-1327) you can see the request as a reference

This is a first attempt at this so there are still some details to see, like if you want this as a separate build or something more integrated into the normal build, about jutils I think that perhaps leaving it like it is, as just classes inside the jutils jar would be the easiest and fastest way.

If there is any problem please let me know

thanks.

Rubén

Hi

Seeing as JInput already has a maven target in it's build, would it not have been better to modify that?

Endolf

yes if you want I will modify that target, seeing that there was some differences like classifier of the natives and using jars instead of zips and as I didn't know if you were using the maven stuff in your projects right now I didn't want to break anything.

any other comment?

Nope, not looked at it other than that yet :), I would suggest adding the jutils stuff to the jutils project, then removing those classes from the jinput.jar file. Rather than mixing the two. The jutils jar only ends up in jinput to make distribution easier.

Endolf

I am using the jinput maven build in a project of mine, but I can change things if that makes sense. I would probably use net.java.games.input as the groupid as that is the package name.

I uses the other groupId because it matches the domain of the project in the rules for inclusion to maven central they say

the groupId of your Maven project, it must match your domain name, so com.googlecode.myprj is valid one but myprj is not

not sure if it won't be a problem to use another groupId but I have no personal problem with either one

Ok, that makes sense.

Well this is attempt number 2 at the jinput patch

now the build is integrated into build.xml

the targets are
mvninstall - to install to the local repo
mvndeploy - to sign (with gpg) and deploy to a repo (currently to /tmp/m2repo)
mvndeploy-old - this allows to deploy using scp (couldn't replicate this with the new method) shouldn't be needed for the sync to maven central

Endolf it would be great if you could start creating a user and requesting access for the oss sonatype repo as explained here (https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide) it will allow us to start testing snapshot deployment and deploy to the staging repo before asking for sync to maven central (requesting access takes a couple of days for them to enable it).

thanks

This is me again :-)

This time I have a patch for the jutils dependency of jinput to try and get that to maven central as well

it has the targets mvninstall and mvndeploy

because this project doesn't have the ant tasks included you have to add that (and the extra config to the build.xml) or add the ant tasks to .ant/lib in your home dir(in linux)

Hi

I'll start looking at this next week hopefully. I'm helping fun a festival this weekend and it's all about to kick off :)

Cheers

Endolf

Mazon: would you mind applying the patch I attached in that post: http://lwjgl.org/forum/index.php/topic,3707.msg20794.html#msg20794
we talked about it some time ago, you weren't sure because it requires jdk 1.6 for the scripting, I asked endolf and he said that the build farm has jdk 1.6 and the requirement would only apply when building the maven version.

This patch would also prevent stuff like this http://lwjgl.org/forum/index.php/topic,3953.0.html from happening

and would allow me to send you the next changes needed to use the jinput version that endolf got into maven central.

thanks

ok, will do this tonight CEST

fixed in r3519

I made a new patch that should get us closer to having lwjgl in maven central ;D

This new patch:

• uses jinput from maven central, instead of deploying
• compares the jinput jar version with the declared maven version and aborts the build if they are different (the check can be overridden by a command line parameter)
• changes the naming convention of how native jars declare the OS to match the ones used in jinput on maven central
• supports install to local repo, deploy to repo, and deploy with gpg signing (needed to deploy to maven central)

One issue remaining is that the MappedObject feature in lwjgl_utils is using a version of asm that isn't released yet, and isn't in maven central, so for now it doesn't add asm as a declared dependency. This means that if someone wants to use the ljwgl maven release, and use MappedObjects, they have to add the asm library to their project.

With this patch applied, we could start talking about deploying snapshot artifacts so that people can check the nightlies using maven in an easy way, and also start talking about releasing lwjgl into maven central  :)

thanks

Created new patch, that adds to the build a flag -Dsnapshot=true that transform the lwjgl.version into a maven snapshot version, this would enable the hudson build server to build snapshot versions without having to specify the maven version in the hudson job config, it would just say -Dsnapshot=true and the ant build would make the transformation itself.

Also added some better logging for when there is a difference in the jinput version between the one used for the regular ant build, and the declared maven dependency.

thanks

Mazon: I made a tiny huge bug in the last patch :-) the result was that it was forcing a snapshot version when the user didn't want to, this patch fixes that, sorry for the error.

I am happy to say, that I started the process to get lwjgl 2.8.0 into maven central :-)
Sadly I forgot that the first time you have to add a comment to a jira issue to ask them to enable the maven central sync so even though everything is done from our side, I don't think the release will reach maven central until monday or tuesday.

The process to make the release has some server side checks and the maven build that we had was failing some checks so I made a patch that should fix that in the future.

Mazon after applying the patch and if you make the 2.8.1 release, it would be useful if you could run:
     
         ant maven-full -Doverridejinput=true

and save the file dist/lwjgl-maven-2.8.1.zip so that I can later perform the 2.8.1 release into maven central

the "-Doverridejinput=true" is used to prevent the build from aborting with this error caused by the difference in the jinput version between the regular lwjgl build and the declared maven dependency
Jinput version in project (2.0.0-b01) is different from the declared jinput version for maven (2.0.5) add -Doverridejinput=true as a command line option to avoid this check

thanks

Achievement Unlocked: LWJGL 2.8.0 and 2.8.1 are available from Maven Central

I will make some detailed tutorial for how to use the dependency, and some choices I made about how to bundle the artifacts

the main dependency you need to add is

<dependency>
    <groupId>org.lwjgl.lwjgl</groupId>
    <artifactId>lwjgl</artifactId>
    <version>2.8.1</version>
</dependency>

this will bring 3 jars with the natives, the transitive dependencies will have a classifier like [natives-windows, natives-osx, natives-linux] you can use the dependency plugin to unpack those jars into a dir so that you can point to it your java.library.path  or use my maven natives plugin (http://code.google.com/p/mavennatives/)  that will unpack all the dependencies with a classifier like "natives-*" into target/natives

There is a sample project (http://mavennatives.googlecode.com/svn/trunk/maven-nativedependencies-example/) using lwjgl 2.8.1 from central, with this project, if you run "mvn package" you will get in your target dir, a zip file that contains a the main jar, the dependencies, the natives exploded and a couple of batch scripts (bat and sh) to run the test.

In the maven central search page (http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.lwjgl.lwjgl%22%20) you can see all the artifacts that make the lwjgl project, with the coordinates for lwjgl_utils and lwjgl_util_applet

If there is anything I can help you with don´t hesitate to ask, I am on the irc channel as "ruben01"

Hope people find this useful, and thanks for all the help from the lwjgl team.

Rubén

Thanks a ton for spearheading this ruben01!

I will make a front page entry on the site tonight (CEST) - please remind me

Quote from: ruben01 on October 20, 2011, 01:34:43
<dependency>
   <groupId>org.lwjgl.lwjgl</groupId>
   <artifactId>lwjgl</artifactId>
   <version>2.8.1</version>
</dependency>

Don't know much about Maven, but just curious, is it normal for package names to be like the groupId "org.lwjgl.lwjgl" (i.e. lwjgl being mentioned twice instead of just "org.lwjgl") ?

The choice I made when building this, was to take the groupId as ${organization}-${project} so in here it would be org.lwjgl as the organization and lwjgl as the project.

This is not too pretty on the eyes when the project shares the name with the organization, but I think it is still better and makes it nicer if you ever add other projects under the umbrella of lwjgl.org, examples of that could be, if you split the appletloader into a separate project from lwjgl so org.lwjgl.appletloader would be the groupId, or anything else you may want to do.

In maven projects are usually made of different artifacts grouped under the same groupId.

In our games we usually have for example

groupId: com.gemserk.games.thecodenameofthegame
artifactsIds: thecodenameofthegame-core, thecodenameofthegame-android, thecodenameofthegame-desktop

Personally I have seen both decisions even from the maven guys, having multiple projects all share the same groupId using it only as the organization or as a category of stuff like org.maven.plugins and all the artifacts from the different projects live under the same groupID. Or using the groupId as the organization + projectName.