How to Bundle a Native Library and a Jni Library Inside a Jar

How to bundle a native library and a JNI library inside a JAR?

It is possible to create a single JAR file with all dependencies including the native JNI libraries for one or more platforms. The basic mechanism is to use System.load(File) to load the library instead of the typical System.loadLibrary(String) which searches the java.library.path system property. This method makes installation much simpler as the user does not have to install the JNI library on his system, at the expense, however, that all platforms might not be supported as the specific library for a platform might not be included in the single JAR file.

The process is as follows:

• include the native JNI libraries in the JAR file at a location specific to the platform, for example at NATIVE/${os.arch}/${os.name}/libname.lib
• create code in a static initializier of the main class to
  
  
  • calc the current os.arch and os.name
  • look for the library in the JAR file at the predefined location using Class.getResource(String)
  • if it exists, extract it to a temp file and load it with System.load(File).

I added functionality to do this for jzmq, the Java bindings of ZeroMQ (shameless plug). The code can be found here. The jzmq code uses a hybrid solution so that if an embedded library cannot be loaded, the code will revert to searching for the JNI library along the java.library.path.

Encapsulating a JNI library inside a jar file

You can't load library from inside JAR without extracting it in a first place.

Take a look here at full sample where native code is embedded inside JAR and extracted when needed.

https://github.com/mkowsiak/jnicookbook/tree/master/recipes/recipeNo031

Update

Well, in that case, when you need to pack more libs and you want to properly resolve locations, you need to play with runtime env a little bit.

Take a look here:

https://github.com/mkowsiak/jnicookbook/tree/master/recipes/recipeNo035

git clone https://github.com/mkowsiak/jnicookbook
cd jnicookbook/recipes/recipeNo035
make
make test

Have fun with JNI!

Packaging JNI Libraries

You can package the .dll into the .jar, but the physical .dll file must be unpackaged from there before it can be loaded because of how Windows handles .dlls. There's no way to load a .dll directly from inside the .jar.

This question seems to contain an appropriate implementation. Note, though, that it's somewhat error-prone: if the program hasn't write permissions to wherever it's going to extract the .dll,you're in a bizarre situation; you can't load the .dll because you don't have permissions to write it first.

A more robust solution is to use some kind of installer to install the whole application into a directory hierarchy such as

/whatEver/myApp/                 (The working directory of the app.)
/whatEver/myApp/myApp.jar        (The main app.)
/whatEver/myApp/lib/library.dll  (The JNI library.)

Then you can load the dll simply using a path relative to the working directory,

System.loadLibrary("lib/library.dll");

To be even more robust, you'll want to ensure that you're running a JVM with appropriate amount of bits before attempting to load the library. A little-advertised fact is that a 64-bit VM can't load 32-bit libraries, nor vice versa. See this question.

Java problem loading native lib inside jar

First off some good advices:

• remove all unnecessary jar files from your project (they might cause problems similar to yours)
• do not mix different versions of LWJGL jar files (might cause problems similar to yours)
• do not mix different versions of LWJGL native .dll files (might cause problems similar to yours)
• use only really needed native .dll files (sometime unnecessary ones may cause errors like yours)

Now here is your code updated by me for automatic detection of all native files in internal natives folder without need of manually defining them + rest is basically yours - tested myself, all works as it should.

//INITIALIZATION OF NATIVE LIBS PACKED INSIDE THE COMPILLED JAR
static {
    try {
        String internalNativesFolder = "natives";

        //EXTRACT NATIVE LIB(S) FROM JAR
        File tmpDir = new File(App.FILE_APPPATH.getAbsolutePath() + App.LOMKA_R + internalNativesFolder);
        if (tmpDir.exists()) {
            FileUtils.deleteDirectory(tmpDir);
        }
        tmpDir.mkdir();

        //GETS ALL FILES IN INTERNAL NATIVES FOLDER
        URI uri = App.class.getResource(App.LOMKA_R + internalNativesFolder).toURI();
        Path myPath;
        if (uri.getScheme().equals("jar")) {
            FileSystem fileSystem = FileSystems.newFileSystem(uri, Collections.<String, Object>emptyMap());
            myPath = fileSystem.getPath(App.LOMKA_R + internalNativesFolder);
        } else {
            myPath = Paths.get(uri);
        }
        Stream<Path> walk = Files.walk(myPath, 1);
        List<String> libNames = new ArrayList<>();
        for (Iterator<Path> it = walk.iterator(); it.hasNext();) {
            String fN = it.next().getFileName().toString();
            if (!fN.equals(internalNativesFolder)) {
                libNames.add(fN);
            }
        }

        //COPIES THOSE FILES TO TEMPORARY LOCAL DIRECTORY
        for (String libName : libNames) {
            URL url = Class.class.getResource(App.LOMKA_R + internalNativesFolder + App.LOMKA_R + libName);
            File nativeLibTmpFile = new File(tmpDir, libName);
            try (InputStream in = url.openStream()) {
                Files.copy(in, nativeLibTmpFile.toPath());
            }
        }

        //ADD PATH TO EXTRACTED LIBS FOLDER TO JAVA
        System.setProperty("java.library.path", tmpDir.getAbsolutePath());
        Field fieldSysPath = ClassLoader.class.getDeclaredField("sys_paths");
        fieldSysPath.setAccessible(true);
        fieldSysPath.set(null, null);

        //LOAD LIB DLL FILE(S)
        libNames.forEach(libName -> {
            System.load(new File(tmpDir, libName).getAbsolutePath());
        });

    } catch (IOException | SecurityException | IllegalArgumentException | NoSuchFieldException | IllegalAccessException | URISyntaxException ex) {
        Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
    }
}

Add SmartID Reader JNI library into existing gradle project

Found the solution:

1. To load external JNI library it must be added in project folder (/lib/ for example)

2. JAR wrapper file (swig generated in my case) must be added as a dependency (right click on jar > add as a library or manually in project structure modules Dependencies). In my case I'm using gradle

       compile fileTree(include: ['*.jar'], dir: 'lib')
   

   in dependencies {} block

3. After that classes from jar will be available in project and all that we need - add native library. There are few ways: set -Djava.library.path=... as JVM run argument,or configure it on the fly like described below:
   https://habr.com/en/post/118027/
   Another way - just place library in system folder or add new directory in system variable (Path on Windows or LD_LIBRARY_PATH on linux). Explanation here: How should I load native libraries for JNI to avoid an UnsatisfiedLinkError?

Regarding to shell script: In p.1 we add jar-wrapper in our classpath
LD_LIBRARY_PATH for linux must be specified or another described solution where jvm will be able to find native library.

And the last - SmartId Reader hasn't ubuntu sdk, so centOS package isn't working on ubuntu (jvm SIGSEGV error, Problematic frame: ld-linux-x86-64.so). I just using windows sdk, it solved issue.

Using FatJar for native libraries

To oversimplify, you can think about the dependencies of a Java application as two separate things:

1. The classpath. This includes your classes, as well as the jars of any libraries you're using.
2. The native library path. This includes resources used for native code.

It looks like you're setting the first one correctly, but not the second one.

Without One-JAR, you'd specify your native library path using the java.library.path command-line argument. Try specifying that when you run your jar.

If that works, then you can look into how One-JAR handles native libraries. This looks like a good starting point.

Google is your friend, and for more information, try searching "java native libraries", "java.library.path", or "onejar native libraries".

More info can also be found at these related questions:

How to bundle a native library and a JNI library inside a JAR?

What is LD_LIBRARY_PATH and how to use it?

Shameless self-promotion: You can also use a tool I created called JarMatey, which is pretty similar to One-JAR.

---

Related Topics