Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime init the cleaner instance in ResourceCleaner #31452

Merged
merged 1 commit into from
Mar 1, 2023

Conversation

jerboaa
Copy link
Contributor

@jerboaa jerboaa commented Feb 27, 2023

Inject an accessor for the ResourceCleaner.CLEANER static variable so that the associated cleaner won't get initialized at build time and ends up being in the native image heap.

Closes: #31440

@quarkus-bot
Copy link

quarkus-bot bot commented Feb 27, 2023

Thanks for your pull request!

The title of your pull request does not follow our editorial rules. Could you have a look?

  • title should preferably start with an uppercase character (if it makes sense!)

This message is automatically generated by a bot.

@jerboaa jerboaa changed the title [resteasy] Runtime init the cleaner instance in ResourceCleaner Runtime init the cleaner instance in ResourceCleaner Feb 27, 2023
@jerboaa
Copy link
Contributor Author

jerboaa commented Feb 27, 2023

Mandrel workflow with this is running here: https://github.com/graalvm/mandrel/actions/runs/4285051293

@jerboaa
Copy link
Contributor Author

jerboaa commented Feb 27, 2023

It should probably be in the resteasy-common extension rather than resteasy-client as I had it before.

@jerboaa
Copy link
Contributor Author

jerboaa commented Feb 27, 2023

@zakkak Please take a look when you get a chance! Thanks.

Copy link
Contributor

@zakkak zakkak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but did you try just reinitializing the class instead?

@jerboaa
Copy link
Contributor Author

jerboaa commented Feb 28, 2023

... but did you try just reinitializing the class instead?

I did not. Then again, wouldn't for runtime-reinited classes the same restrictions apply (instances of those classes are disallowed in the image heap)? I'll have a look and will report back.

@zakkak
Copy link
Contributor

zakkak commented Feb 28, 2023

... but did you try just reinitializing the class instead?

I did not. Then again, wouldn't for runtime-reinited classes the same restrictions apply (instances of those classes are disallowed in the image heap)?

No, fields of reinited classes are treated differently. The issue with instances of certain classes being present in the native heap is that those instances were generated at build time and they are thus not valid at runtime. When reiniting a class the said fields are "uninitialized" in the initial native heap and get initialized at run time which is valid.

@jerboaa
Copy link
Contributor Author

jerboaa commented Feb 28, 2023

... but did you try just reinitializing the class instead?

I did not. Then again, wouldn't for runtime-reinited classes the same restrictions apply (instances of those classes are disallowed in the image heap)?

No, fields of reinited classes are treated differently. The issue with instances of certain classes being present in the native heap is that those instances were generated at build time and they are thus not valid at runtime. When reiniting a class the said fields are "uninitialized" in the initial native heap and get initialized at run time which is valid.

I thought this is for runtime initialized classes. Runtime re-inited classes are initialized both at build time and again at runtime:
https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/classinitialization/InitKind.java#L44..L45

Anyway, in this case both seem to work.

Runtime-reinit:
https://github.com/quarkusio/quarkus/compare/main...jerboaa:quarkus:runtime-reinit-resource?expand=1
Runtime-init:
https://github.com/quarkusio/quarkus/compare/main...jerboaa:quarkus:runtime-init-resource?expand=1

Shall I update this patch with the last (runtime-init) option?

@zakkak
Copy link
Contributor

zakkak commented Feb 28, 2023

I thought this is for runtime initialized classes. Runtime re-inited classes are initialized both at build time and again at runtime: https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/classinitialization/InitKind.java#L44..L45

Correct, they are initialized at build time for the build to proceed and to increase the level of optimization, but the values from the static fields are not used at runtime (if I am not mistaken they are not even placed in the generated native heap), instead they get re-initialized at run time. Because of this they are allowed to be present in the heap.

Anyway, in this case both seem to work.

I didn't suggest pure runtime initialization because I thought the class needed to be build-time initialized. Great that you tried both.

Runtime-reinit: https://github.com/quarkusio/quarkus/compare/main...jerboaa:quarkus:runtime-reinit-resource?expand=1 Runtime-init: https://github.com/quarkusio/quarkus/compare/main...jerboaa:quarkus:runtime-init-resource?expand=1

Shall I update this patch with the last (runtime-init) option?

Yes, please.

@jerboaa
Copy link
Contributor Author

jerboaa commented Feb 28, 2023

@zakkak Done now. Thanks for the reviews. I've also rescheduled a run of the mandrel tests with this:
https://github.com/graalvm/mandrel/actions/runs/4285051293

Copy link
Contributor

@zakkak zakkak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

ResourceCleaner contains a static reference to a java.lang.ref.Cleaner
instance which are disallowed in the native image heap. Runtime
initializing that class instead fixes this issue.

Closes: quarkusio#31440
@jerboaa
Copy link
Contributor Author

jerboaa commented Feb 28, 2023

@gsmet Hi! Could you please help approving the workflow and getting this integrated? It fixes many broken integration tests on the mandrel CI side. Thanks!

@zakkak zakkak added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Mar 1, 2023
@gsmet
Copy link
Member

gsmet commented Mar 1, 2023

@zakkak just to be sure, you have the permission to approve a workflow run, right? Wondering if it's only admins.

@zakkak
Copy link
Contributor

zakkak commented Mar 1, 2023

@zakkak just to be sure, you have the permission to approve a workflow run, right? Wondering if it's only admins.

Yes, I was the one who approved it this morning. I am also going to merge this once the CI completes. @jerboaa was probably not aware that I have the power to do both :)

Probably it didn't help that I didn't notice that the workflow was blocked for some unknown (since this is not the first contribution of @jerboaa) reason.

@quarkus-bot
Copy link

quarkus-bot bot commented Mar 1, 2023

Failing Jobs - Building 37b5375

Status Name Step Failures Logs Raw logs
✔️ Devtools Tests - JDK 11
✔️ Devtools Tests - JDK 11 Windows
Devtools Tests - JDK 17 Build Failures Logs Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Devtools Tests - JDK 17 #

- Failing: integration-tests/devtools 

📦 integration-tests/devtools

io.quarkus.devtools.commands.CreateProjectTest.createMultipleTimes line 254 - More details - Source on GitHub

java.util.concurrent.TimeoutException: createMultipleTimes() timed out after 3 seconds
	at org.junit.jupiter.engine.extension.TimeoutExceptionFactory.create(TimeoutExceptionFactory.java:29)
	at org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:58)

@zakkak
Copy link
Contributor

zakkak commented Mar 1, 2023

The failure is unrelated to the PR. Merging it.

@zakkak zakkak merged commit fe7b6a2 into quarkusio:main Mar 1, 2023
@zakkak zakkak removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label Mar 13, 2023
@zakkak zakkak added this to the 3.0 - main milestone Mar 13, 2023
@zakkak
Copy link
Contributor

zakkak commented Mar 13, 2023

Marking this for backport since although the Cleaner instances were disallowed only recently the issue is actually present in older versions as well.

Not sure if it's worth backporting to 2.7 as well.

@gsmet
Copy link
Member

gsmet commented Mar 16, 2023

Actually, AFAICS, ResourceCleaner doesn't exist in 2.16.

@zakkak
Copy link
Contributor

zakkak commented Mar 17, 2023

Indeed, the ResourceCleaner was added in 6.1.0.Final and 2.16 is using 4.7.7.

zakkak added a commit to zakkak/quarkus that referenced this pull request Aug 14, 2023
org.jboss.resteasy.spi.ResourceCleaner needs to be runtime initialized
both in resteasy-classic and resteasy-reactive.

Follow-up to quarkusio#31452
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Several quarkus integration tests fail to compile to native with latest GraalVM master
3 participants