Maven jlink plug-in: native libraries not found when running jlinked image

[mmgavioli]

I understand this has probably more to do with the Maven jlink plug-in than with LWJGL, but after searching for days elsewhere I thought that, among LWJGL users, someone else is likely to have faced (and possibly solved) this same issue.

Context: Linux Mint 21.1, Java Adoptium 17.0.7 JDK, Maven 3.6.3. Application using LWJGL with the Maven jlink plug-in to create a self-standing runnable image.
Error: when running the generated image:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Failed to locate library: liblwjgl.so

as soon the code accesses LWJGL (in this case the GLFW.glfwInit() method): some loader seems unable to locate the native libraries INSIDE the collected modules.

I am reasonably sure the image DOES contain the native libraries, specifically within the lib/modules file: I tried removing all native libraries dependencies and the generated lib/modules file is shorter approx. by the length of the relevant jar's. Classes from dependencies are located without problems, but libraries within dependencies are not. I suspect it is a matter of additional configuration of the Maven jlink plug-in, but after searching as much as I can I found nothing related (or relatable) with point.

Any hint, pointer or suggestion is WELCOME! Thanks!

For completeness, this is the output of the jlink plug-in, showing that the plug-in knows about all needed native libraries:

[INFO] --- maven-jlink-plugin:3.1.0:jlink (default-cli) @ treni ---
[INFO]  -> module: org.lwjgl.stb ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-stb/3.3.2/lwjgl-stb-3.3.2.jar )
[INFO]  -> module: org.slf4j ( /home/mmg/.m2/repository/org/slf4j/slf4j-api/2.0.7/slf4j-api-2.0.7.jar )
[INFO]  -> module: imgui.natives.linux ( /home/mmg/.m2/repository/io/github/spair/imgui-java-natives-linux/1.86.10/imgui-java-natives-linux-1.86.10.jar )
[INFO]  -> module: org.lwjgl ( /home/mmg/.m2/repository/org/lwjgl/lwjgl/3.3.2/lwjgl-3.3.2.jar )
[INFO]  -> module: imgui.binding ( /home/mmg/.m2/repository/io/github/spair/imgui-java-binding/1.86.10/imgui-java-binding-1.86.10.jar )
[INFO]  -> module: org.lwjgl.stb.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-stb/3.3.2/lwjgl-stb-3.3.2-natives-linux.jar )
[INFO]  -> module: org.lwjgl.glfw ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-glfw/3.3.2/lwjgl-glfw-3.3.2.jar )
[INFO]  -> module: org.joml ( /home/mmg/.m2/repository/org/joml/joml/1.10.5/joml-1.10.5.jar )
[INFO]  -> module: org.lwjgl.opengl ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-opengl/3.3.2/lwjgl-opengl-3.3.2.jar )
[INFO]  -> module: org.slf4j.jul ( /home/mmg/.m2/repository/org/slf4j/slf4j-jdk14/2.0.7/slf4j-jdk14-2.0.7.jar )
[INFO]  -> module: org.lwjgl.opengl.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-opengl/3.3.2/lwjgl-opengl-3.3.2-natives-linux.jar )
[INFO]  -> module: org.lwjgl.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl/3.3.2/lwjgl-3.3.2-natives-linux.jar )
[INFO]  -> module: com.vistamaresoft.treni ( /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/target/classes )
[INFO]  -> module: org.lwjgl.glfw.natives ( /home/mmg/.m2/repository/org/lwjgl/lwjgl-glfw/3.3.2/lwjgl-glfw-3.3.2-natives-linux.jar )
[INFO] Building zip: /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/target/treni-0.0.1.zip

This is (a shortened version for brevity of) the module pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

<!-- Several descriptive tags removed for brevity -->
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <target.name>treni</target.name>
        <!-- Versions -->
        <project.version>0.0.1</project.version>
        <java.version>17</java.version>
        <joml.version>1.10.5</joml.version>
        <joml-primitives.version>1.10.4</joml-primitives.version>
        <lwjgl.version>3.3.2</lwjgl.version>
        <imgui-java.version>1.86.10</imgui-java.version>
        <exec-maven-plugin.version>3.0.0</exec-maven-plugin.version>
        <maven-compiler-plugin.version>3.11.0</maven-compiler-plugin.version>
        <maven-dependency-plugin.version>3.6.0</maven-dependency-plugin.version>
        <maven-jar-plugin.version>3.3.0</maven-jar-plugin.version>
        <maven-jlink-plugin.version>3.1.0</maven-jlink-plugin.version>
        <maven-resources-plugin.version>3.3.1</maven-resources-plugin.version>
    </properties>

    <profiles>
        <profile>
            <id>lwjgl-natives-linux-amd64</id>
            <activation>
                <os>
                    <family>unix</family>
                    <arch>amd64</arch>
                </os>
            </activation>
            <properties>
                <lwjgl.natives>natives-linux</lwjgl.natives>
                <imgui.native>natives-linux</imgui.native>
                <imgui.native.module>linux</imgui.native.module>
            </properties>
        </profile>
        <!-- Windows and MacOs profiles removed for brevity -->
    </profiles>

    <dependencies>
        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl</artifactId>
            <version>${lwjgl.version}</version>
        </dependency>
        <-- more dependencies removed for brevity -->

<!-- Natives -->

        <dependency>
            <groupId>org.lwjgl</groupId>
            <artifactId>lwjgl</artifactId>
            <version>${lwjgl.version}</version>
            <classifier>${lwjgl.natives}</classifier>
            <scope>runtime</scope>
        </dependency>
        <-- mode native dependencies removed for brevity -->

    </dependencies>

    <build>
        <pluginManagement>
            <plugins>

            <!-- more pulg-ins removed for brevity -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-jlink-plugin</artifactId>
                    <version>${maven-jlink-plugin.version}</version>
                    <!-- run with `mvn jlink:jlink` separately from `mvn package` as combining both into `mvn package jlink:jlink` raises an error -->
                    <extensions>true</extensions>
                    <configuration>
                        <!-- Module paths overriding the Mavem local repo path for non-modular JAR, modularised with moditect-maven-plugin -->
                        <modulePaths>
                            <modulePath>${project.build.directory}/modules/imgui-java-binding-${imgui-java.version}.jar</modulePath>
                            <modulePath>${project.build.directory}/modules/imgui-java-natives-${imgui.native.module}-${imgui-java.version}.jar</modulePath>
                        </modulePaths>
                        <compress>2</compress>
                        <noHeaderFiles>true</noHeaderFiles>
                        <noManPages>true</noManPages>
                        <stripDebug>true</stripDebug>
                        <launcher>treni=com.vistamaresoft.treni/com.vistamaresoft.treni.Main</launcher>
                    </configuration>
                </plugin>

            </plugins>
        </pluginManagement>

        <!-- PLUG-IN EXECUTIONS -->

        <plugins>
            <plugin>
                <groupId>org.moditect</groupId>
                <artifactId>moditect-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>add-module-infos</id>
                        <phase>generate-resources</phase>
                        <goals>
                        <goal>add-module-info</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

[spasi]

Hey mmgavioli,

It might be an issue with your project's module-info. Could you post it please? It should require the native modules, e.g.

requires transitive org.lwjgl.natives;

I'm using a similar setup in a project of mine, with LWJGL and the moditect plugin to modularize non-modular dependencies. However, I'm not using the Maven jlink plugin. Maven builds everything and all dependencies are copied to target/modules (both already modular and those modularized by moditect). Then I run this script:

MODULE_PATH=target/<myapp>-SNAPSHOT.jar:target/modules

$JAVA_HOME/bin/java -p $MODULE_PATH --validate-modules

$JAVA_HOME/bin/jlink \
--module-path $MODULE_PATH:"$JAVA_HOME/jmods" \
--add-modules <myapp_module>,java.management,jdk.naming.dns,java.security.jgss,<add more modules here if necessary> \
--compress=1 \
--launcher <app>=<myapp_module>/<myapp_MainClass> \
--output build \
--ignore-signing-information \
--exclude-jmod-section=headers \
--exclude-jmod-section=man \
--dedup-legal-notices=error-if-not-same-content

This way I have full control and the call to java --validate-modules ensures that nothing is missing.

[mmgavioli]

Hello SPASI, thanks for your reply.

1) jlink: thanks for the example about using the jlink command directly: I'll try this way too.

2) native libs in module-info.java: My module-info.java does not include the native jar's in the requires clauses, as their modules (for instance, org.lwjgl.natives for the lwjgl-3-3-2-natives-linux.jar) are not found both by Eclipse AND by Maven itself; this is part of the output of mvn package if they are included:

[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/src/main/java/module-info.java:[15,42] module not found: org.lwjgl.natives
[ERROR] /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/src/main/java/module-info.java:[17,47] module not found: org.lwjgl.glfw.natives
[ERROR] /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/src/main/java/module-info.java:[19,49] module not found: org.lwjgl.opengl.natives
[ERROR] /home/mmg/Documents/projects/Eclipse_workspaces/Treni/treni/src/main/java/module-info.java:[21,42] module not found: org.lwjgl.stb.natives

Now, the native libraries are correctly listed in the pom.xml (at least nobody complains about them), are correctly downloaded to the local .m2 repo, recognised by Eclipse in the "Maven dependencies" list, listed by the jlink plug-in, etc... So, everything seems to work, except that it does not! I suspect some basic 'building block' is missing, but I cannot guess which one. This is not the first time I use jlink and the process usually works, but the need for the native libraries seems to block everything.

3) target/modules: Maven by itself does not put anything in it; in my case the moditect plug-in puts there the modularised JAR's, but this is all. Do you use the maven-dependency-plugin to have the dependency JAR's copied there?

Thanks again!

[spasi]

2) Hmm, I don't remember why exactly, but it looks like I'm overriding Maven's module path. In the maven-compiler-plugin configuration, I have:

<compilerArgs>
	<arg>--module-path</arg>
	<arg>${project.build.directory}/modules</arg>
</compilerArgs>

3) Yes, I use maven-dependency-plugin's copy-dependencies goal. In its configuration I have:

<outputDirectory>${project.build.directory}/modules</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
<includeGroupIds>org.lwjgl,<more modular group ids here, comma-separated></includeGroupIds>

[mmgavioli]

@SPASI: thank you for your continued support. Things definitely improved. It seems that:

1) adding the requires transitive <dependency_module>.natives clauses makes the native libraries accessible (even via the Maven jlink plug-in);
2) for this to work, though:
2a - the dependencies need to be moved into some target subfolder (say target/modules) prior to compilation or the compiler would not find them (I use the generate-resources phase);
2b - the Maven compiler plug-in directed to it (via the <compilerArgs> <arg>--module-path</arg> <arg>${project.build.directory}/modules</arg> </compilerArgs> directive)
2c - and the jlink plug-in as well via the <modulePaths> <modulePath>${project.build.directory}/modules</modulePath> </modulePaths> directive (parallel to what is set in your script)

There is still some missing link with the imgui-java library, but the main problem seems solved!

THANKS A LOT!