Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support GraalVM / native image builds using the GraalVM Java 11 build #5682

Closed
Sanne opened this issue Nov 22, 2019 · 21 comments · Fixed by #5854
Closed

Support GraalVM / native image builds using the GraalVM Java 11 build #5682

Sanne opened this issue Nov 22, 2019 · 21 comments · Fixed by #5854
Assignees
Labels
kind/enhancement New feature or request
Milestone

Comments

@Sanne
Copy link
Member

Sanne commented Nov 22, 2019

Description
A follow up to #4218 which did the upgrade to GraalVM version 19.3 but focused only on the "JDK8 flavoured edition"

@gwenneg
Copy link
Member

gwenneg commented Nov 22, 2019

@Sanne Do you plan on working on this one or can I give it a go at the end of my day?

@Sanne
Copy link
Member Author

Sanne commented Nov 22, 2019

It will be at least ~5 days until I have time to open my IDE again :-/

Feel free to try! And don't hesitate to ask for help.

@gwenneg gwenneg self-assigned this Nov 22, 2019
@dxps
Copy link
Contributor

dxps commented Nov 22, 2019

A quick run using:

  • OpenJDK 11
    openjdk version "11.0.4" 2019-07-16
    OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
    OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3, mixed mode, sharing)
    
  • GraalVM 19.3.0
  • Quarkus 1.0.0.CR2
➜  library-svc git:(master) ✗ ./mvnw verify -Pnative
...
[INFO] --- quarkus-maven-plugin:1.0.0.CR2:native-image (default) @ library-svc ---
[INFO] [org.hibernate.jpa.boot.internal.PersistenceXmlParser] HHH000318: Could not find any META-INF/persistence.xml file in the classpath
[INFO] [org.hibernate.Version] HHH000412: Hibernate Core {5.4.9.Final}
[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building native image source jar: /home/mariusi/dev/devisions/eda_playground/library-svc/target/library-svc-0.1-native-image-source-jar/library-svc-0.1-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/mariusi/dev/devisions/eda_playground/library-svc/target/library-svc-0.1-native-image-source-jar/library-svc-0.1-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on OpenJDK 64-Bit Server VM
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] /home/mariusi/apps/graalvm/graalvm-ce-19.3.0/bin/native-image -J-DCoordinatorEnvironmentBean.transactionStatusManagerEnable=false -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar library-svc-0.1-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http,https --enable-all-security-services -H:NativeLinkerOption=-no-pie -H:+JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace library-svc-0.1-runner
[library-svc-0.1-runner:13745]    classlist:   9,009.22 ms
[library-svc-0.1-runner:13745]        setup:     353.40 ms
Error: substitution target for io.quarkus.vertx.core.runtime.graal.Target_sun_misc_URLClassPath is not loaded. Use field `onlyWith` in the `TargetClass` annotation to make substitution only active when needed.
com.oracle.svm.core.util.UserError$UserException: substitution target for io.quarkus.vertx.core.runtime.graal.Target_sun_misc_URLClassPath is not loaded. Use field `onlyWith` in the `TargetClass` annotation to make substitution only active when needed.
	at com.oracle.svm.core.util.UserError.abort(UserError.java:65)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.findTargetClass(AnnotationSubstitutionProcessor.java:823)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:252)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:230)
	at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:876)
	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:825)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:528)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:445)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Error: Image build request failed with exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  29.849 s
[INFO] Finished at: 2019-11-22T15:40:13+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:1.0.0.CR2:native-image (default) on project library-svc: Failed to generate a native image: Failed to augment application classes: Build failure: Build failed due to errors
[ERROR] 	[error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: java.lang.RuntimeException: Failed to build native image
[ERROR] 	at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:289)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] 	at io.quarkus.deployment.ExtensionLoader$1.execute(ExtensionLoader.java:941)
[ERROR] 	at io.quarkus.builder.BuildContext.run(BuildContext.java:415)
[ERROR] 	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2011)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1535)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1426)
[ERROR] 	at java.base/java.lang.Thread.run(Thread.java:834)
[ERROR] 	at org.jboss.threads.JBossThread.run(JBossThread.java:479)
[ERROR] Caused by: java.lang.RuntimeException: Image generation failed
[ERROR] 	at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:278)
[ERROR] 	... 12 more
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
➜  library-svc git:(master) ✗ 

@zeldanerd24
Copy link

@DEVisions that specific issue looks like it has been fixed by #5336, but looks to be currently targeted for the 1.1.0 milestone

@dmlloyd
Copy link
Member

dmlloyd commented Nov 22, 2019

As soon as we have a clean test run, we should start the process of requiring JDK 11 by printing a deprecation warning when JDK 8 is used.

@dmlloyd dmlloyd mentioned this issue Nov 22, 2019
18 tasks
@gwenneg
Copy link
Member

gwenneg commented Nov 22, 2019

@DEVisions @zeldanerd24 Quarkus 1.0.0.CR2 currently supports GraalVM 19.2.1 and it will stay the same in Quarkus 1.0.0.Final. The GraalVM 19.3.0 support (which includes native image execution using JDK 11) should be introduced in Quarkus 1.1.0.

@dxps
Copy link
Contributor

dxps commented Nov 22, 2019

@gwenneg Thanks for the feedback also. I am interested of getting the native image of a Java 11 based project. That's why I wanted to test with GraalVM 19.3.0.
Do we have an ETA for Quarkus 1.1.0, please?

@gwenneg
Copy link
Member

gwenneg commented Nov 22, 2019

If you want to run an early test with Quarkus master and GraalVM 19.3.0 (you would have to build Quarkus locally), I can ping you here when it's ready. This is a work in progress for now.

I'll let @gsmet answer the ETA question :)

@dxps
Copy link
Contributor

dxps commented Nov 22, 2019

@gwenneg Yes, please. Thank you.

@dxps
Copy link
Contributor

dxps commented Nov 25, 2019

Quarkus is now 1.0!!! Hooray!!! 😄
@gsmet Don't want to put pressure, but as I want to promote this internally (vs the SpringBoot alternative), I would love to also show the native image benefits. So, as we are using Java 11, by when would you know this issue would be solved?
Thanks

@gsmet
Copy link
Member

gsmet commented Nov 25, 2019

@DEVisions well, it will be solved when we will have fixed all the remaining issues :). Hopefully for 1.1.0, which is planned for mid-december but we will see.

Keep in mind that GraalVM + JDK 11 is in tech preview, as per the GraalVM team.

@gwenneg
Copy link
Member

gwenneg commented Dec 2, 2019

@DEVisions: This issue has been closed automatically because of the merge of #5854, but the Quarkus compatibility with GraalVM 19.3.0 is still a work in progress. There's actually a GraalVM blocking issue (oracle/graal#1893) that affects both JDK 8 and 11 editions. It could force us to wait for GraalVM 19.3.1 before 19.3 is integrated into Quarkus, but I'm looking for a workaround to use 19.3.0. I'll keep you posted when I have new information about this subject.

@dxps
Copy link
Contributor

dxps commented Dec 2, 2019

@gwenneg Great, appreciate it! Thanks a lot!

@gsmet gsmet reopened this Dec 2, 2019
@gsmet
Copy link
Member

gsmet commented Dec 2, 2019

I'm reopening this one. In any case, it will need documentation, tests and so on.

@gwenneg
Copy link
Member

gwenneg commented Dec 6, 2019

I created a GraalVM issue tonight about the remaining problem in Quarkus with JDK 11: oracle/graal#1941. I'm not sure whether the cause is in Quarkus or in GraalVM.

Debugging the native image generation didn't give much information so far because the issue is happening with a proxied annotation (and I couldn't tell what was the proxy target interface from the debugging data). I'll take another look at it tomorrow.

@gwenneg
Copy link
Member

gwenneg commented Dec 7, 2019

It seems I found the cause and it looks very much like a GraalVM bug. The good news is that there's an easy and harmless workaround in Quarkus so it shouldn't be a problem for our GraalVM 19.3.0 JDK 11 support :) This needs to be confirmed with some extensive tests which I will run tomorrow.

@gwenneg
Copy link
Member

gwenneg commented Dec 7, 2019

@gsmet: How would you like to communicate in the doc about the GraalVM + JDK 11 support in Quarkus? Since CI doesn't run JDK 11 native tests for now, I suppose we can't just write that we provide a production-grade support. Should it be qualified as preview in the doc instead?

@gsmet
Copy link
Member

gsmet commented Dec 7, 2019

@gwenneg well, GraalVM says the JDK 11 support is preview for them so it will be for us too.

That being said, we need to tackle the CI issue.

@gwenneg
Copy link
Member

gwenneg commented Dec 15, 2019

All issues mentioned here have been fixed except for #6018 which can randomly happen with any native integration tests build. It usually affects 5 to 10 integration tests modules when I run a full Quarkus native build locally. I'll run another full build tonight and check the results tomorrow to make sure nothing got left behind or hidden by #6018.

@gwenneg
Copy link
Member

gwenneg commented Jan 27, 2020

We had many successful JDK 11 native tests runs in CI (using GitHub actions like this one). Native image generation and tests with JDK 11 are therefore fully supported now.

@gwenneg gwenneg closed this as completed Jan 27, 2020
@gsmet gsmet modified the milestones: 1.1.0.Final, 1.2.0.Final Feb 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants