Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for AOT processing with GraalVM tracing agent is not consistent #30511

Closed
sbrannen opened this issue May 16, 2023 · 11 comments
Closed

Support for AOT processing with GraalVM tracing agent is not consistent #30511

sbrannen opened this issue May 16, 2023 · 11 comments
Assignees
Labels
theme: aot An issue related to Ahead-of-time processing type: bug A general bug
Milestone

Comments

@sbrannen
Copy link
Member

sbrannen commented May 16, 2023

Overview

In Spring Framework 6.0.9, we made changes to the Spring TestContext Framework (TCF) to allow AOT processing with the GraalVM tracing agent and Native Build Tools.

Specifically, we introduced a TestAotDetector utility that is specific to the TCF. This detector considers the current runtime to be in "AOT runtime mode" if the spring.aot.enabled Spring property is set to true or the GraalVM org.graalvm.nativeimage.imagecode JVM system property is set to any non-empty value other than agent.

Since Spring Boot's testing support uses AotDetector.useGeneratedArtifacts() in various places, the Boot team should investigate whether Spring Boot Test should migrate from AotDetector to TestAotDetector.

Related Issues

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label May 16, 2023
@mhalbritter mhalbritter added type: bug A general bug theme: aot An issue related to Ahead-of-time processing and removed status: waiting-for-triage An issue we've not yet triaged or decided on labels May 16, 2023
@mhalbritter
Copy link
Contributor

mhalbritter commented May 16, 2023

Isn't there a more general problem? When using the agent with bootRun, this fails too. Using -Pagent, which enables the GraalVM agent, sets the environment variable org.graalvm.nativeimage.imagecode to agent. The AotDetector.useGeneratedArtifacts() then returns true because NativeDetector.inNativeImage returns true, because it only checks org.graalvm.nativeimage.imagecode for null (and does not ignore agent, like the TestAotDetector does).

Say i don't use AOT processing, then try to use -Pagent bootRun. We will now think we run in a native image (NativeDetector.inNativeImage returned true) and try to load classes which aren't there (the AOT processed ApplicationContextInitializer).

Shouldn't the NativeDetector be changed to ignore agent?

@mhalbritter
Copy link
Contributor

mhalbritter commented May 16, 2023

If I run -Pagent test, I get this stacktrace while running processTestAot:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::       (v3.0.7-SNAPSHOT)

2023-05-16T14:38:10.409+02:00 ERROR 17669 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalArgumentException: Could not find class [com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:334) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.instantiateInitializer(AotApplicationContextInitializer.java:80) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.initialize(AotApplicationContextInitializer.java:71) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.lambda$forInitializerClasses$0(AotApplicationContextInitializer.java:61) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:605) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:385) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:309) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1388) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContextForAotProcessing(SpringBootContextLoader.java:113) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:263) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:232) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.lambda$processAheadOfTime$4(TestContextAotGenerator.java:204) ~[spring-test-6.0.9.jar:6.0.9]
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[na:na]
        at org.springframework.util.MultiValueMapAdapter.forEach(MultiValueMapAdapter.java:179) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:196) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:158) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.performAotProcessing(TestAotProcessor.java:91) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:72) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:39) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AbstractAotProcessor.process(AbstractAotProcessor.java:82) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.test.context.SpringBootTestAotProcessor.main(SpringBootTestAotProcessor.java:63) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
Caused by: java.lang.ClassNotFoundException: com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
        at java.base/java.lang.Class.forName0(Native Method) ~[na:na]
        at java.base/java.lang.Class.forName(Class.java:467) ~[na:na]
        at org.springframework.util.ClassUtils.forName(ClassUtils.java:284) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:324) ~[spring-core-6.0.9.jar:6.0.9]
        ... 25 common frames omitted

2023-05-16T14:38:10.419+02:00  WARN 17669 --- [           main] o.s.t.c.aot.TestContextAotGenerator      : Failed to generate AOT artifacts for test classes [com.lingh.AddRemoveDatasourceTest]

org.springframework.test.context.aot.TestContextAotException: Failed to load ApplicationContext for AOT processing for test class [com.lingh.AddRemoveDatasourceTest]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:272) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:232) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.lambda$processAheadOfTime$4(TestContextAotGenerator.java:204) ~[spring-test-6.0.9.jar:6.0.9]
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[na:na]
        at org.springframework.util.MultiValueMapAdapter.forEach(MultiValueMapAdapter.java:179) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:196) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:158) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.performAotProcessing(TestAotProcessor.java:91) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:72) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:39) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AbstractAotProcessor.process(AbstractAotProcessor.java:82) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.test.context.SpringBootTestAotProcessor.main(SpringBootTestAotProcessor.java:63) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
Caused by: java.lang.IllegalArgumentException: Could not find class [com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:334) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.instantiateInitializer(AotApplicationContextInitializer.java:80) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.initialize(AotApplicationContextInitializer.java:71) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.lambda$forInitializerClasses$0(AotApplicationContextInitializer.java:61) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:605) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:385) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:309) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1388) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContextForAotProcessing(SpringBootContextLoader.java:113) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:263) ~[spring-test-6.0.9.jar:6.0.9]
        ... 11 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
        at java.base/java.lang.Class.forName0(Native Method) ~[na:na]
        at java.base/java.lang.Class.forName(Class.java:467) ~[na:na]
        at org.springframework.util.ClassUtils.forName(ClassUtils.java:284) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:324) ~[spring-core-6.0.9.jar:6.0.9]
        ... 25 common frames omitted

That's because org.springframework.boot.SpringApplication#addAotGeneratedInitializerIfNecessary (which is not in test support code!) has been called and it thinks it runs in a native image.

@sbrannen
Copy link
Member Author

Shouldn't the NativeDetector be changed to ignore agent?

That's a very good question, @mhalbritter, and it might be the best solution for the time being.

I've been advocating since early 2020 that applications and build tools need a first-class mechanism for determining when an application is running in the JVM with the GraalVM tracing agent active, but it hasn't gained much traction from the GraalVM team.

In my last comment in that issue, I stated the following.

please have the agent set a different system property that can be queried in order to determine if code is running on the JVM with the agent enabled.

Lately, I believe that would be the better choice for the community. The org.graalvm.nativeimage.imagecode system property should probably remain reserved for use with the buildtime and runtime values.

@sdeleuze, @bclozel, @wilkinsona, @philwebb, @jhoeller, thoughts?

@wilkinsona
Copy link
Member

wilkinsona commented May 16, 2023

Given the lack of traction on oracle/graal#2395, I think it makes sense to broaden our reliance on the NBT plugins setting org.graalvm.nativeimage.imagecode to agent. There doesn't appear to be any other path forward here that isn't blocked.

@sdeleuze
Copy link
Contributor

sdeleuze commented May 16, 2023

Shouldn't the NativeDetector be changed to ignore agent?

The agent should detect the codepath used on native to generate the most accurate hints, so I don't think we should change NativeDetector behavior.

Maybe we should raise that point in the next meeting we have with the GraalVM team?

@wilkinsona
Copy link
Member

I don't think we should change NativeDetector behavior.

But we could change AotDetector though? IMO, useGeneratedArtifacts() ought to return false when running with the agent.

@sbrannen
Copy link
Member Author

I don't think we should change NativeDetector behavior.

But we could change AotDetector though? IMO, useGeneratedArtifacts() ought to return false when running with the agent.

Yes, exactly -- like what TestAotDetector does.

That's actually what I meant to say previously, but I accidentally replied to the suggestion for changing the NativeDetector. Sorry for the mix-up.

And that would make the TestAotDetector obsolete.

@snicoll
Copy link
Member

snicoll commented May 21, 2023

I think that's what we should do, so I am moving this back to framework.

@snicoll snicoll transferred this issue from spring-projects/spring-boot May 21, 2023
@snicoll snicoll changed the title Investigate test AOT processing support with GraalVM tracing agent and Native Build Tools Support AOT processing with GraalVM tracing agent is not consistent May 21, 2023
@snicoll snicoll changed the title Support AOT processing with GraalVM tracing agent is not consistent Support for AOT processing with GraalVM tracing agent is not consistent May 21, 2023
@snicoll snicoll added this to the 6.0.x milestone May 21, 2023
@mhalbritter
Copy link
Contributor

mhalbritter commented May 22, 2023

The agent should detect the codepath used on native to generate the most accurate hints, so I don't think we should change NativeDetector behavior.

I'm not sure I agree to that line of reasoning. IMHO the NativeDetector should return true if (and only if) running a native image and AotDetector should return true if (and only if) running in AOT mode (which is always the case in native image).

When running my integration tests while not using AOT mode with the agent attached, I would assume that it will generate hints for all dynamic behavior (reflection dependency injection etc.) and that both NativeDetector and AotDetector return false.

If I want to generate hints with the agent for the AOT code path, then I would need to run the tests in AOT mode, too.

What's the idea behind NativeDetector or AotDetector return true when running with an attached agent? Better developer experience because users can run integration tests and only get hints for the AOT codepath out of the box?

@sdeleuze
Copy link
Contributor

NativeDetector is on purpose pretty unopinionated and flexible, as soon as org.graalvm.nativeimage.imagecode is set it returns true, which means the app runs using one of the flavors of native image. In practice, it is set to buildtime or runtime by native-image compiler, or to agent by various tools that need to emulate native code path on the JVM.

This behavior looks correct to me as for some use cases, it is important to exercise the native code path on the JVM. The goal is not just to have more optimal hints, the goal is also to not break when running the app on native with tracing agent generated hints due to missing hints.

@snicoll
Copy link
Member

snicoll commented May 22, 2023

This behavior looks correct to me as for some use cases, it is important to exercise the native code path on the JVM

Yes, but to be consistent with that statement, said code should run with AOT optimizations and it obviously can't. It looks wrong to me to require the native specific code path to run, while the most important bit, the one that avoids a bunch of code paths thanks to AOT, doesn't run. NativeDetector is obviously very generic on purpose but having it enabled while AOT is not (in the form of optimization, or execution) feels wrong to me.

@sdeleuze sdeleuze self-assigned this May 25, 2023
@sdeleuze sdeleuze modified the milestones: 6.0.x, 6.0.10 May 25, 2023
sdeleuze added a commit to sdeleuze/spring-framework that referenced this issue May 26, 2023
This commit refines how native agent detection works
for both test and application executions.

It rolls back the introduction of TestAotDetector done in 1113096
and instead updates AotDetector.useGeneratedArtifacts()
to only detect "buildtime" and "runtime" imagecode system
property values by leveraging a new method
NativeDetector.inNativeImage(NativeDetector.Context...).

Closes spring-projectsgh-30511
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme: aot An issue related to Ahead-of-time processing type: bug A general bug
Projects
None yet
Development

No branches or pull requests

6 participants