JNA fails to load with GraalVM throwing JNA: Problems loading core IDs: java.lang.Object

[amahfouz1]

• Version of JNA : 5.3.1
• Version and vendor of the java virtual machine:
  openjdk version "1.8.0_242"
  OpenJDK Runtime Environment (build 1.8.0_242-b06)
  OpenJDK 64-Bit Server VM GraalVM CE 20.0.0 (build 25.242-b06-jvmci-20.0-b02, mixed mode)
• Operating system: Oracle Linux 7 and Centos 7
• System architecture (CPU type, bitness of the JVM) : x86_64
• Complete description of the problem: JNA native lib fails to load with GraalVM:
  After generating a Linux native executable using GraalVM when JNA is booting up and loading the libjnidispatch.so library it fails with: JNA: Problems loading core IDs: java.lang.Object
  This is the line throwing this error:

  jna/native/dispatch.c

  Line 3267 in e6ab3ec

  fprintf(stderr, "JNA: Problems loading core IDs: %s\n", err);

Steps to reproduce:
Repo: https://github.com/amahfouz1/clj-pcap-native

• Build docker image:
  I have a Dockerfile handy in the repo to help to build the image.

cd clj-pcap-native
docker build -t clj-pcap .

• Run docker image mounting the project:

docker run -it -v `pwd`:/clj-pcap clj-pcap /bin/sh

• Generate native executable:

cd /clj-pcap
lein native-image

• Run the executable:

/clj-pcap/target/default+uberjar/cljpcap -Djna.debug_load=true

sh-4.2# /clj-pcap/target/default+uberjar/cljpcap -Djna.debug_load=true
Mar 15, 2020 8:13:53 PM com.sun.jna.Native extractFromResourcePath
INFO: Looking in classpath from sun.misc.Launcher$AppClassLoader@16c0128 for /com/sun/jna/linux-x86-64/libjnidispatch.so
Mar 15, 2020 8:13:53 PM com.sun.jna.Native extractFromResourcePath
INFO: Found library resource at resourcew:com/sun/jna/linux-x86-64/libjnidispatch.so
Mar 15, 2020 8:13:53 PM com.sun.jna.Native extractFromResourcePath
INFO: Extracting library to /root/.cache/JNA/temp/jna2847104491610838225.tmp
JNA: Problems loading core IDs: java.lang.Object

The libjndispatch.so is found and the C code is now started executing and it fails.

[matthiasblaesing]

native-image needs to be aware what class meta data has to be included:

https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md#jni-java-native-interface
https://github.com/oracle/graal/blob/master/substratevm/CONFIGURE.md

Not much JNA can do there for you.

[amahfouz1]

Thanks for your attention and your reply. Actually I tried the metadata part before with no luck. I just pushed it to the repo https://github.com/amahfouz1/clj-pcap-native/blob/master/project.clj#L27.
For me, I am trying to know why JNA is failing within the SubstrateVM is there any JVM arguments that need to be passed to get it to load or are there any requirements for JNA that I am missing.
I don't know why it's failing at this line

jna/native/dispatch.c

Line 1380 in e6ab3ec

if (!LOAD_CREF(env, Object, "java/lang/Object")) return "java.lang.Object";

and what LOAD_CREF is exactly trying to do. I appreciate your help in connecting the dots.

[amahfouz1]

@matthiasblaesing I think it's trying to load java/lang/Object but it's throwing ClassNotFoundException but why it can't find the class that's still mysterious for me.

[kangzq0530]

@matthiasblaesing I think it's trying to load java/lang/Object but it's throwing ClassNotFoundException but why it can't find the class that's still mysterious for me.

All classes you want to access by jni need to be registered during the native image building. Unregistered classes are not accessible at run time.

[amahfouz1]

That's the dilemma you get the JNA classes available during build time they load fine with no issues and this is because the loading code lives in a static block, but during the runtime, they never reload again and I start getting UnsatisfiedLinkError for the com.sun.jna.Native native methods that's why I decided to differ the initialization to runtime instead of build time then I started getting this weird error.
Actually I have two questions:

• Is there a way to point JNA during runtime to which JVM to use?
• Is there a way to get JNA reloaded?

[matthiasblaesing]

Sorry, but I don't have time to dig into GraalVM native image at this time. However, I would try a smaller approach first - try to compile jna.jar to a native image. The root issue already happens in the JNA initialization, so a clojure build is just noise.

My first impression is though (without having worked with native image), that the config file
https://github.com/amahfouz1/clj-pcap-native/blob/master/conf/jni-config.json

does not hold enough classes. JNA relies heavily on reflection and thus you'll need at least all classes the native code references: Everything that is referenced by:

• LOAD_PCREF - primitives
• LOAD_CREF - classes
• LOAD_MID - methods
• LOAD_FID - fields

In addition all types derived from Structure, NativeMapped and most probably several more need to be available at runtime.

[amahfouz1]

@matthiasblaesing thanks again for your reply. Now I feel I am going into the right direction. After I have been digging into it all day today I was able to get the same point but I am stuck with reflections on this line:

jna/native/dispatch.c

Line 1493 in e6ab3ec

return "Byte.value";

not sure what's Byte.value any idea?

Here is the updated jni-config.json: https://github.com/amahfouz1/clj-pcap-native/blob/master/conf/jni-config.json

[matthiasblaesing]

This would be better discussed on the mailing list (this is not a JNA issue). value is a private field of the java.lang.Byte class. It hold the primitive value of the class. This might help:

https://stackoverflow.com/questions/9871009/jniusing-wrapper-types-for-primitive-data-types-to-simulate-passing-by-pointers

[amahfouz1]

I think I figured it out it's java.lang.reflect.Field the error message was very vague :)

[amahfouz1]

:( nope I was wrong

[amahfouz1]

now for real, I figured it out :)

{
    "name" : "java.lang.Byte",
    "allDeclaredConstructors" : true,
    "allPublicConstructors" : true,
    "allDeclaredMethods" : true,
    "allPublicMethods" : true,
    "fields" : [
      { "name" : "value", "allowWrite" : true }
    ]
  }

This was the configuration needed.

[matthiasblaesing]

Closing this as it is not an issue in JNA.