Loader issues using maven / gradle

[abcdef]

Hi

I have a project that uses LWJGL and I upload this to my local maven repo, the pom file contains the following, note it has the natives listed.

<dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.lwjgl</groupId>
        <artifactId>lwjgl-bom</artifactId>
        <version>3.2.3</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl</artifactId>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-glfw</artifactId>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-openal</artifactId>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-opengl</artifactId>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-vulkan</artifactId>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.darkyellow</groupId>
      <artifactId>core</artifactId>
      <version>1.0</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.darkyellow</groupId>
      <artifactId>resource</artifactId>
      <version>1.0</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl</artifactId>
      <classifier>natives-windows</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl</artifactId>
      <classifier>natives-linux</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl</artifactId>
      <classifier>natives-macos</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-glfw</artifactId>
      <classifier>natives-windows</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-glfw</artifactId>
      <classifier>natives-linux</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-glfw</artifactId>
      <classifier>natives-macos</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-openal</artifactId>
      <classifier>natives-windows</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-openal</artifactId>
      <classifier>natives-linux</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-openal</artifactId>
      <classifier>natives-macos</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-opengl</artifactId>
      <classifier>natives-windows</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-opengl</artifactId>
      <classifier>natives-linux</classifier>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.lwjgl</groupId>
      <artifactId>lwjgl-opengl</artifactId>
      <classifier>natives-macos</classifier>
      <scope>runtime</scope>
    </dependency>
  </dependencies>

If I then reference this maven artifact in another project the loader  does not work correctly

[LWJGL] Version: 3.2.3 build 13
[LWJGL]          OS: Linux v5.4.0-42-generic
[LWJGL]         JRE: 11.0.8 amd64
[LWJGL]         JVM: OpenJDK 64-Bit Server VM v11.0.8+10-LTS by Azul Systems, Inc.
[LWJGL] Loading JNI library: lwjgl
[LWJGL]         Module: org.lwjgl
[LWJGL]         linux/x64/org/lwjgl/liblwjgl.so not found in java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
[LWJGL]         liblwjgl.so not found in java.library.path
[LWJGL] Failed to load a library. Possible solutions:
        a) Add the directory that contains the shared library to -Djava.library.path or -Dorg.lwjgl.librarypath.
        b) Add the JAR that contains the shared library to the classpath.
10/08/2020 06:52:42.381 [ERROR] [org.darkyellow.application.lwjgl.LwjglExceptionHandler.uncaughtException()] An exception has occured in the DarkYellow library
java.lang.UnsatisfiedLinkError: Failed to locate library: liblwjgl.so
        at org.lwjgl.system.Library.loadSystem(Library.java:162)
        at org.lwjgl.system.Library.loadSystem(Library.java:62)
        at org.lwjgl.system.Library.<clinit>(Library.java:50)
        at org.lwjgl.glfw.GLFW.<clinit>(GLFW.java:674)

If I re add the native dependencies only to the top level project it works. Is there a way to get the dependencies added to the original project and not have to repeat the native dependencies declaration?

On a side note there aren't any vulkan native dependencies, why is this?

[spasi]

Afaik runtime scoped dependencies are transitive and should be resolved in the top level project. Does it work if you change the scope to compile? Also, what's the output of mvn dependency:tree?

The Vulkan bindings need natives on macOS only (the MoltenVK binary which implements Vulkan over Metal).

[abcdef]

This is the output on the top level project from gradle without the natives in the dependencies

The gradle version is 6.2.2

runtimeClasspath - Runtime classpath of source set 'main'.
+--- my:core:1.0
+--- my:resource:1.0
|    \--- my.core:1.0
+--- my:viewer.assets:1.0
+--- my:application.lwjgl:1.0
|    +--- org.lwjgl:lwjgl-bom:3.2.3
|    |    +--- org.lwjgl:lwjgl:3.2.3 (c)
|    |    +--- org.lwjgl:lwjgl-glfw:3.2.3 (c)
|    |    +--- org.lwjgl:lwjgl-openal:3.2.3 (c)
|    |    +--- org.lwjgl:lwjgl-opengl:3.2.3 (c)
|    |    \--- org.lwjgl:lwjgl-vulkan:3.2.3 (c)
|    +--- org.lwjgl:lwjgl -> 3.2.3
|    +--- org.lwjgl:lwjgl-glfw -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- org.lwjgl:lwjgl-openal -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- org.lwjgl:lwjgl-opengl -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- org.lwjgl:lwjgl-vulkan -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- my:core:1.0
|    \--- my:resource:1.0 (*)

Here is the output when it is added

runtimeClasspath - Runtime classpath of source set 'main'.
+--- my:core:1.0
+--- my:resource:1.0
|    \--- my:core:1.0
+--- my:viewer.assets:1.0
+--- my:application.lwjgl:1.0
|    +--- org.lwjgl:lwjgl-bom:3.2.3
|    |    +--- org.lwjgl:lwjgl:3.2.3 (c)
|    |    +--- org.lwjgl:lwjgl-glfw:3.2.3 (c)
|    |    +--- org.lwjgl:lwjgl-openal:3.2.3 (c)
|    |    +--- org.lwjgl:lwjgl-opengl:3.2.3 (c)
|    |    \--- org.lwjgl:lwjgl-vulkan:3.2.3 (c)
|    +--- org.lwjgl:lwjgl -> 3.2.3
|    +--- org.lwjgl:lwjgl-glfw -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- org.lwjgl:lwjgl-openal -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- org.lwjgl:lwjgl-opengl -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- org.lwjgl:lwjgl-vulkan -> 3.2.3
|    |    \--- org.lwjgl:lwjgl:3.2.3
|    +--- my:core:1.0
|    \--- my:resource:1.0 (*)
+--- org.lwjgl:lwjgl-bom:3.2.3 (*)
+--- org.lwjgl:lwjgl -> 3.2.3
+--- org.lwjgl:lwjgl-glfw -> 3.2.3 (*)
+--- org.lwjgl:lwjgl-openal -> 3.2.3 (*)
\--- org.lwjgl:lwjgl-opengl -> 3.2.3 (*)

and this  is the dependency from my.application.lwjgl and the gradle code

runtimeClasspath - Runtime classpath of source set 'main'.
+--- org.lwjgl:lwjgl-bom:3.2.3
|    +--- org.lwjgl:lwjgl:3.2.3 (c)
|    +--- org.lwjgl:lwjgl-glfw:3.2.3 (c)
|    +--- org.lwjgl:lwjgl-openal:3.2.3 (c)
|    +--- org.lwjgl:lwjgl-opengl:3.2.3 (c)
|    \--- org.lwjgl:lwjgl-vulkan:3.2.3 (c)
+--- org.lwjgl:lwjgl -> 3.2.3
+--- org.lwjgl:lwjgl-glfw -> 3.2.3
|    \--- org.lwjgl:lwjgl:3.2.3
+--- org.lwjgl:lwjgl-openal -> 3.2.3
|    \--- org.lwjgl:lwjgl:3.2.3
+--- org.lwjgl:lwjgl-opengl -> 3.2.3
|    \--- org.lwjgl:lwjgl:3.2.3
+--- org.lwjgl:lwjgl-vulkan -> 3.2.3
|    \--- org.lwjgl:lwjgl:3.2.3
+--- my:core:1.0
\--- my:resource:1.0
     \--- my:core:1.0

dependencies {
    implementation platform("org.lwjgl:lwjgl-bom:$lwjglVersion")
    implementation "org.lwjgl:lwjgl"
    runtimeOnly "org.lwjgl:lwjgl::natives-windows"
    runtimeOnly "org.lwjgl:lwjgl::natives-linux"
    runtimeOnly "org.lwjgl:lwjgl::natives-macos"
    implementation "org.lwjgl:lwjgl-glfw"
    runtimeOnly "org.lwjgl:lwjgl-glfw::natives-windows"
    runtimeOnly "org.lwjgl:lwjgl-glfw::natives-linux"
    runtimeOnly "org.lwjgl:lwjgl-glfw::natives-macos"
    implementation "org.lwjgl:lwjgl-openal"
    runtimeOnly "org.lwjgl:lwjgl-openal::natives-windows"
    runtimeOnly "org.lwjgl:lwjgl-openal::natives-linux"
    runtimeOnly "org.lwjgl:lwjgl-openal::natives-macos"
    implementation "org.lwjgl:lwjgl-opengl"
    runtimeOnly "org.lwjgl:lwjgl-opengl::natives-windows"
    runtimeOnly "org.lwjgl:lwjgl-opengl::natives-linux"
    runtimeOnly "org.lwjgl:lwjgl-opengl::natives-macos"
    implementation "org.lwjgl:lwjgl-vulkan"
    implementation 'my:core:1.0'
    implementation 'my:resource:1.0'
}
[/code[

[abcdef]

Is there a reason you went the classifier approach with maven for the native jars? I'm not sure it works that well with heirachical dependencies.

[spasi]

Because it's cleaner this way. This is the repository of LWJGL artifacts: https://repo1.maven.org/maven2/org/lwjgl/. One folder for each binding + the LWJGL core. If you enter GLFW for example, you'll see this: https://repo1.maven.org/maven2/org/lwjgl/lwjgl-glfw/3.2.3/. Everything related to GLFW is in there.

Also, I'm not aware of any technical reason that artifacts with classifiers can't get resolved transitively.

However, I'd like to point out that the LWJGL native artifacts are not meant to be used like that. You end up downloading all natives, but only using a subset of them (the matching platform/arch ones). The platform cannot be resolved at the library level of course, so you need to do it at the top/application level. Also, when running on Java 9+ and using modules, having the same native module for multiple platforms in the module path will create conflicts. The module-info class for a particular module is exactly the same in all native jars.

[abcdef]

Ignore the fact that I am selecting multiple classifiers, it doesn't work with one either.

Turns out this could be a gradle.issue

https://github.com/gradle/gradle/issues/1487

[abcdef]

This issue doesn't happen with gradle 6.3 (just so happened I was on the last version that didn't work)

It was this issue that was resolved and it mentions lwjgl3 in it as the use case (so looks like the issue has been seen by someone else)

https://github.com/gradle/gradle/issues/11825

[spasi]

Great, thanks!