[native-image] Error loading a referenced type: java.lang.IllegalAccessError #1711
Comments
|
I did add the |
neomatrix369
changed the title
|
I did some debugging and I think I need to do some of Although now I'm wrestling with the /**
* Configuration to adapt the
*/
@TargetClass(className = "org.bytedeco.javacpp.indexer.UnsafeRaw",
onlyWith = org.bytedeco.javacpp.indexer.BooleanIndexer.class)
final class Target_org_bytedeco_javacpp_indexer_UnsafeRaw {
@Alias @RecomputeFieldValue(kind = Kind.ArrayBaseOffset, declClass = byte[].class, isFinal = true)
private static long ARRAY_BYTE_BASE_OFFSET;
}
I think a no params constructor for I tried to open up the class via <--snipped-->
{
"name":"org.bytedeco.javacpp.indexer.BooleanIndexer",
"methods":[{"name":"<init>","parameterTypes":["long", "long"]}],
"allDeclaredMethods":true,
"allPublicMethods":true,
"allDeclaredConstructors":true,
"allPublicConstructors":true
},
<--snipped--> |
|
@neomatrix369, the class AlwaysIncluded implements BooleanSupplier {
@Override
public boolean getAsBoolean() {
return true;
}
}It's possible that you don't need |
Yes, I initially implemented it without the |
|
@peter-hofer so your suggestion worked, but now I'm getting this: I got this implementation from the graal source files, Ill dig into the stack traces to find out what the source of the issue is. |
|
Looking at UnsafeRaw.java, it seems like the name of the field is |
I just did that so we are on the same page, cause the Target is looking for this specific source file and not the Unsafe.java. |
|
So we are back to the same error as mentioned in #1711 (comment), except the warning no longer happens. I think our substitution worked and fixed the warning. Below is the error message in short: I'm going to try opening up |
|
So doing the above, did not have an effect, here are my new changes to the {
"name":"org.bytedeco.javacpp.indexer.Raw",
"allDeclaredMethods":true,
"allPublicMethods":true,
"allDeclaredConstructors":true,
"allPublicConstructors":true,
"allDeclaredClasses":true,
"allPublicClasses":true,
"methods":[{"name":"getInstance","parameterTypes":[] }]
}Even though the error message appears to be telling Raw is not accessible, making it accessible does not still resolve the error. Its possible the issue is somewhere else. |
|
Debugging the Here is the piece of code I was debugging when the silent failures were occurring: package com.oracle.graal.pointsto.infrastructure
public class WrappedConstantPool implements ConstantPool {
<--snipped--->
public static void loadReferencedType(ConstantPool cp, int cpi, int opcode, boolean initialize) {
ConstantPool root = cp;
while (root instanceof WrappedConstantPool) {
root = ((WrappedConstantPool) root).wrapped;
}
try {
hsLoadReferencedType.invoke(root, cpi, opcode, initialize);
} catch (Throwable ex) {
Throwable cause = ex;
if (ex instanceof InvocationTargetException && ex.getCause() != null) {
cause = ex.getCause();
if (cause instanceof BootstrapMethodError && cause.getCause() != null) {
cause = cause.getCause();
}
} else if (ex instanceof ExceptionInInitializerError && ex.getCause() != null) {
cause = ex.getCause();
}
throw new UnresolvedElementException("Error loading a referenced type: " + cause.toString(), cause); <=== this error is swallowed somewhere up in the stack and not reported
}
}
<--snipped--->
}The error we get on the console at the end of the failed build is a side effect of another error(s). Is this a known issue / behaviour? Any reasons for it to behave in this fashion? I have added a substitute for the missing logger but I'm waiting to see if this impacts the execution of my program. |
|
Through debugging I also found out that the "Error loading a referenced type: java.lang.IllegalAccessError: tried to access class org.bytedeco.javacpp.indexer.Raw from class org.bytedeco.javacpp.indexer.BooleanIndexer" error is caused when the I figured |
|
@peter-hofer any thoughts or hints on the above issues? |
|
|
|
No. I'd need to see the full stack trace for the swallowed error. |
|
@dougxc @peter-hofer I'll provide them asap! For both the silent exception and the main issue with |
|
We should also find out where the exception is being swallowed and fix that if possible. |
Initially, I thought (after reading @cstancu's post) that the native-image builder has become smarter and is handling this commonly known Logger / LoggerFactory issue and not reporting the issue. But in that case if it did that, it should print out a warning saying I have handled this for you, but let me provide the data and you can make the best conclusion out of that I think. |
Silent exception: Slf4j |
|
Thanks @neomatrix369 but neither of those (jstack?) traces provide the stack trace of the swallowed exception. The best way to see this would be something like: |
|
Original exception: org.bytedeco.javacpp.indexer.BooleanIndexer |
I'll try to get that, the ones I provided are from IntelliJ's stack trace options capturing all the threads. |
|
This is the interesting trace: Since the accessing class is in the same package as the accessed class, I can only suspect some class loader problem or the fact this is called via reflection. @tkrodriguez do you have any ideas? |
Glad this can be used for further investigations! I suggest you add some sort of save stack trace to log files each time these kings of try...except get executed, I can do a PR with it if you like? Or some other solution so silent exceptions are not going ignored. |
@dougxc Please take look at the two gists on Silent exceptions again (#1711 (comment)), I have updated them with stacktraces like you wanted - from |
Yes, we are allowing reflective access to |
|
But since they are in the same package based on the name this access should always be ok. So unless there are multiple copies of these classes loaded I don't see how it could fail. Running with -verbose:class would show whether there are multiple copies. Otherwise I'd have to assume there's some exception below all this that hiding what's really going on, though I can't really see a sequence of exceptions which could cause this. It might be interesting to try using a fastdebug JVM and run with -XX:+TraceExceptions. This would produce quite a lot of output but you'll be able to see every exception that's occurring. Are the command to reproduce this problem relatively straightforward? |
Two classes are only in the same package if both their package name and class loader are identical. The problem you're seeing could be explained by a JVMCI JAR trying to access a class in the built-in JVMCI module(or vice-versa) for example. Both would have the same package name, but would not be the same package. It follows that any code which compares only package names to predict access is incorrect (though I'm not sure that's the case here, it's just a good rule to keep in mind). |
@tkrodriguez Is there a fastdebug JVM of GraalVM? CE or EE available? Can you provide via some download? Or is it easy to build via mx? Again the option These are my doubts before I can produce you the stacktrace you need: end-to-end! |
|
Tom means to pass There is no fastdebug version of GraalVM. It's something you'd have to build yourself using the fastdebug binaries at https://github.com/graalvm/openjdk8-jvmci-builder/releases/tag/jvmci-19.3-b03. If you manage to do that, then you can pass |
|
Yes sorry I was unclear. You can use |
|
As requested here is the jar file (zipped) with the scripts I use (in the folder called Commands to reproduce the issue: Below should give you an explanation on what the script can do, options etc...: |
I'll look into the fastdebug version sometime later, sounds useful for situations like the one we ar having currently. |
@dougxc @tkrodriguez did what I attached above work for you? I'm checking to see if its good for you to do your debugging and investigations. Pls let me know otherwise. |
|
Yes I'm able to reproduce the failure from the provided jar. I'm looking into it. |
|
The problem is that org.bytedeco.javacpp.indexer.Raw is loaded in 'sun/misc/Launcher$AppClassLoader' and org.bytedeco.javacpp.indexer.BooleanIndexer is loaded in com/oracle/svm/hosted/NativeImageClassLoader. Looking at the -verbose:class output I think the problem is that there are 2 copies of those packages. One is in jre/lib/svm/builder/javacpp.jar and the other is in your jar MLPMnist-1.0.0-bin.jar which is why they end up in different loaders. Your copy has extra classes which is what allows it to see the other package. If they had exactly the same classes you'd just end up using the copy in Substrate. So basically this is a collision between classes used by Substrate and the classes being packaged. @christianwimmer It might be good if Substrate has some sanity checking when loading classes to make sure they are getting loaded from the expected source to make failures of this kind more obvious. Maybe these kinds of support classes used by Substrate need to be repackaged with different names to avoid collisions with user code? |
|
Yes, we need to make sure that we do not use any external libraries without adding a package prefix. |
|
Thanks @tkrodriguez for your anaylsis and @christianwimmer for chipping in! Looks like its all under control now, does it look like there is a quick fix for this might unblock a lot of libraries/frameworks/apps. |
|
It's possible if you replaced the javacpp.jar in the GraalVM image with a version that matches what you need everything would work fine. I think javacpp.jar is only used by the LLVM backend which isn't in use here. Actually just deleting it would probably let you move forward. |
|
@tkrodriguez thanks for the tips, will do that and see where it goes from here! |
The build worked perfect! Thanks! |
|
I have managed to build and run the native-image, although I get the below exception: Build model....
2019-10-09 04:44:51,893: o.n.l.f.Nd4jBackend - Loaded [CpuBackend] backend
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:290)
at java.lang.Class.ensureInitialized(DynamicHub.java:467)
at org.deeplearning4j.nn.conf.NeuralNetConfiguration$Builder.seed(NeuralNetConfiguration.java:671)
at org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerTrain.execute(MLPMnistSingleLayerTrain.java:64)
at org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerRunner.execute(MLPMnistSingleLayerRunner.java:101)
at org.deeplearning4j.feedforward.mnist.MLPMnistSingleLayerRunner.main(MLPMnistSingleLayerRunner.java:72)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:5900)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5766)
at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:202)
at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)
... 5 more
Caused by: java.lang.NullPointerException
at org.nd4j.linalg.factory.Nd4j.isSupportedPlatform(Nd4j.java:5908)
at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:5785)
... 9 moreEven though I have loaded the |
|
I have resolved the above error. Although native image still breaks out just as its loading - other JNI related issues are coming to the surface. |
|
The last I tested, this issue seems to be fixed starting 19.2.1, let me know if you don't agree. I'll close it, for now, we can reopen if it reoccurs. |
|
I don't think this is fixed as long as |
I was hoping it was fixed, one less thing for you guys to worry about, thanks for keeping me posted! |
|
Fixed in 9140f26 |
Uh oh!
There was an error while loading. Please reload this page.
I get this error when building a native-image from an uber jar, the detailed version of the error is as follows:
Is it expecting the class to be opened via the
reflect-config.jsonconfiguration, which isn't clear from the error message?The line of code that's causing the error is over here: https://github.com/bytedeco/javacpp/blob/master/src/main/java/org/bytedeco/javacpp/indexer/BooleanIndexer.java#L87.
Here is the source to org.bytedeco.javacpp.indexer.Raw.java
The text was updated successfully, but these errors were encountered: