Add native libraries loading support

[ileasile]

During the investigation of #214, it turned out that the only way to use a JNI library in notebook is to call System.load() or System.loadLibrary() inside the library itself. In some cases (see RDKit) library authors don't do it and don't provide any methods for native library loading.

In "general" project setup all dependencies (user code and libraries) are loaded with one URL classloader, so the user may call System.load() in their code, and it works. In kernel, library dependencies are loaded with URL classloader, and snippets loaded with another org.jetbrains.kotlin.scripting.compiler.plugin.impl.CompiledScriptClassLoader. So, loaded JNI wrapper just can't see the native definitions which were loaded by System.load() call in the snippet code.

I propose the following solution for this problem:

1. Define a new file annotation @file:NativeLibrary(path: String)
2. On this annotation, compile a single class with the following code:

object Loader_$id {
   fun load() = System.load("$path")
}

3. Load this class with the same URL classloader with which all other dependencies are loaded. To do it, just add the path to the compiled class to addedClasspath list in org.jetbrains.kotlinx.jupyter.dependencies.JupyterScriptDependenciesResolverImpl#resolveFromAnnotations.
4. Before cell execution but after the dependencies were loaded execute snippet Loader_$id.load()

After all, it allows to write the code like this:

@file:DependsOn("<jni-wrapper.jar>")
@file:NativeLibrary("<native-library.dll>")
// some code that uses methods from <jni-wrapper.jar>

Current workaround for this problem is to make up a wrapper for the native library with the wrapper inside it, see the warpper for RDKit here

[altavir]

The native library does not make a lot of sense without its wrapper. The annotation that only hot-fixes the problem, but does not add anything new seems to be a rather weak solution.
Since Jupyter is not constrained by the JDK version I wonder if we could consider using JEP 389.

[ileasile]

I came up to a hacky solution that solves this problem a bit easier. From the user perspective it looks like this:

@file:DependsOn(<jni-wrapper.jar>")
import some.wrapper.Clazz
loadLibrary(Clazz::class.java, "<path-to-native-library>")

// some code that uses methods from <jni-wrapper.jar>

All we need to do is to implement loadLibrary function. Here it is:

fun loadLibrary(javaClass: Class<*>, name: String, isAbsolute: Boolean = true) {
    val clazz = ClassLoader::class.java
    val method = clazz.getDeclaredMethod("loadLibrary", Class::class.java, java.lang.String::class.java, Boolean::class.java)
    method.isAccessible = true
    method.invoke(null, javaClass, name, isAbsolute)
}

ClassLoader.loadLibrary is a package-private static method. We need to invoke it to load a library not from "current" classloader. So, we simply do it using reflection.

[ileasile]

Let's discuss the aforementioned JEP in a separate issue

[SemanticBeeng]

wonder if we could consider using JEP 389.

polynote uses jep.
https://github.com/polynote/polynote/blob/965c4709a6be161c08a98eaaf840365559103ff3/polynote-kernel/src/main/scala/polynote/kernel/interpreter/python/PythonInterpreter.scala#L871-L877
https://twitter.com/semanticbeeng/status/1317397102104858625

cc @ileasile