Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce allocation in matching of unknown properties #44448

Merged
merged 1 commit into from
Nov 15, 2024

Conversation

radcortez
Copy link
Member

@radcortez radcortez commented Nov 12, 2024

Use io.smallrye.config.PropertyName instead of io.smallrye.config.KeyMap. io.smallrye.config.PropertyName requires less allocations and it is faster when matching property names, because it does not need to split the String.

This comment has been minimized.

This comment has been minimized.

@gsmet
Copy link
Member

gsmet commented Nov 13, 2024

I'm a bit worried about the test failures. But unfortunately there's nothing in the log about the 500 errors that the client encounters.

@gsmet
Copy link
Member

gsmet commented Nov 13, 2024

I asked for a rebase.

@radcortez
Copy link
Member Author

I'm a bit worried about the test failures. But unfortunately there's nothing in the log about the 500 errors that the client encounters.

Yes, I've executed this locally and got the same error. Apparently, they are related. I'll have a look.

@radcortez
Copy link
Member Author

The issue was caused by the runtime property quarkus.rest-client.logging.scope could also match the build time property quarkus.rest-client.[name].scope.

I did change the logic in the recording to skip check in other type phases once a match is found, but we need to check all phases.

This comment has been minimized.

@gsmet
Copy link
Member

gsmet commented Nov 14, 2024

I think the native test failing is probably still a problem.

@radcortez
Copy link
Member Author

Didn't notice. Let me have a look.

@radcortez
Copy link
Member Author

The issue was caused by * properties not having max priority when inserted into the Set. We do this in SmallRyeConfig, but it is an internal API, and I forgot to add it. For now, it stays here, but I'll provide a proper API for it in SmallRyeConfig.

Copy link

quarkus-bot bot commented Nov 14, 2024

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit a04c136.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.


Flaky tests - Develocity

⚙️ JVM Tests - JDK 17 Windows

📦 integration-tests/grpc-hibernate

com.example.grpc.hibernate.BlockingRawTest.shouldAdd - History

  • Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:975)
	at com.example.grpc.hibernate.BlockingRawTestBase.shouldAdd(BlockingRawTestBase.java:59)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)

⚙️ Native Tests - HTTP

📦 integration-tests/rest-client-reactive

io.quarkus.it.rest.client.BasicTestIT.shouldCreateClientSpans - History

  • expected: <1> but was: <2> - org.opentest4j.AssertionFailedError
org.opentest4j.AssertionFailedError: expected: <1> but was: <2>
	at io.quarkus.it.rest.client.BasicTest.shouldCreateClientSpans(BasicTest.java:216)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at io.quarkus.test.junit.QuarkusTestExtension.interceptTestMethod(QuarkusTestExtension.java:805)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

@radcortez radcortez requested a review from gsmet November 15, 2024 09:00
@radcortez radcortez added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Nov 15, 2024
@radcortez
Copy link
Member Author

radcortez commented Nov 15, 2024

Failing tests are just flaky now.

public static Set<PropertyName> toPropertyNames(final Set<String> names) {
Map<PropertyName, String> propertyNames = new HashMap<>();
for (String name : names) {
PropertyName propertyName = new PropertyName(name);
Copy link
Contributor

@franz1981 franz1981 Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if...PropertyName become a mutable class (which can wrap different names) and we allocate a single instance of it - if it is often NOT added?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need this at start to validate names. Adding is only done once. My plan was to write a proper API for this in SR Config.

Map<PropertyName, String> propertyNames = new HashMap<>();
for (String name : names) {
PropertyName propertyName = new PropertyName(name);
if (propertyNames.containsKey(propertyName)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending what's the actual common case you want to optimize for and the number of names, you can just use remove checking for null, to save containsKey + remove to happen.
If this one is not the common scenario, instead, you can just perform a propertyNames.putIfAbsent and after checking that's not null, handling the subsequent remove/change/addition logic

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not exactly to optimize, but because different String may be equals (for instance, when they match *), then we need to give priority to the * name when inserting it into the collection.

}
}
unknownProperties.removeAll(toRemove);
return propertyNames.keySet();
Copy link
Contributor

@franz1981 franz1981 Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that sounds paranoid (it is a bit I admit), but beware keySet if the lifecycle of the String used as values is to go away, the key set within HashMap retain a strong reference to the original map (including the entries and values) preventing the Strings to be GCd - see https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/HashMap.java#L989 - the hidden this due to be an inner class

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can rewrite it, but I'm not very concerned. This is used only during build time, and the method calls are all local when validating / generating the configuration, so everything should be GC by the end of the call.

Copy link
Member

@gsmet gsmet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this in, we can still improve later.

@gsmet gsmet merged commit 0d1aa9a into quarkusio:main Nov 15, 2024
52 checks passed
@quarkus-bot quarkus-bot bot added this to the 3.18 - main milestone Nov 15, 2024
@quarkus-bot quarkus-bot bot removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label Nov 15, 2024
@quarkus-bot quarkus-bot bot added the kind/enhancement New feature or request label Nov 15, 2024
@gsmet
Copy link
Member

gsmet commented Nov 15, 2024

I think we will have to backport it to include it in 3.17 as it's fixing a regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants