Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved TestStep creation performance #2667

Merged
merged 8 commits into from
Dec 22, 2022
Merged

Conversation

jkronegg
Copy link
Contributor

@jkronegg jkronegg commented Dec 20, 2022

🤔 What's changed?

Refactored TestAbortedExceptions Predicate creation.

⚡️ What's your motivation?

Performance gain (Predicate creation is about 50'000 times faster).
Fixes #2666

🏷️ What kind of change is this?

  • 🏦 Refactoring/debt/DX (improvement to code design, tooling, documentation etc. without changing behaviour)

♻️ Anything particular you want feedback on?

Nothing.

📋 Checklist:

  • I agree to respect and uphold the Cucumber Community Code of Conduct
  • I've changed the behaviour of the code
    • I have added/updated tests to cover my changes.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly.
  • Users should know about my change
    • I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

Changed implementation to use variant 6 proposed in cucumber#2666 .
Added test cases for `TestAbortedExceptions` `Predicate` with one negative example and two positive examples.
Corrected reference to non-existing code.
Added info in Unreleased section and corrected typo in Cucumber 7.10.0 section.
@@ -26,20 +26,18 @@ final class TestAbortedExceptions {

static Predicate<Throwable> createIsTestAbortedExceptionPredicate() {
ClassLoader defaultClassLoader = ClassLoaders.getDefaultClassLoader();
return Arrays.stream(TEST_ABORTED_EXCEPTIONS)
.flatMap(s -> {
return throwable -> Arrays.stream(TEST_ABORTED_EXCEPTIONS)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would a for loop be faster?

Copy link
Contributor Author

@jkronegg jkronegg Dec 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I benchmarked the following code:

static Predicate<Throwable> createIsTestAbortedExceptionPredicate7() {
    ClassLoader defaultClassLoader = ClassLoaders.getDefaultClassLoader();
    return throwable -> {
        for (String s : TEST_ABORTED_EXCEPTIONS) {
            try {
                Class<?> aClass = defaultClassLoader.loadClass(s);
                if (aClass.isInstance(throwable)) {
                    return true;
                }
            } catch (Throwable t) {
                rethrowIfUnrecoverable(t);
                log.debug(t,
                        () -> String.format(
                                "Failed to load class %s: will not be supported for aborted executions.", s));
            }
        }
        return false;
    };
}

The JMH benchmark says the variant 7 (the one with a loop) is slightly slower than the proposed version (variant 6), but not significantly :

Benchmark                                                               Mode  Cnt          Score          Error  Units
TestAbortedExceptionsBenchmark.createIsTestAbortedExceptionPredicate5  thrpt   25  264824252,996 ± 4775342,657  ops/s
TestAbortedExceptionsBenchmark.createIsTestAbortedExceptionPredicate6  thrpt   25  259886661,320 ±  1766902,768  ops/s
TestAbortedExceptionsBenchmark.createIsTestAbortedExceptionPredicate7  thrpt   25  254627500,665 ± 12269987,245  ops/s

I think the code readability is better with the Stream+anyMatch than with the loop, so I would prefer to keep the proposed version..

@mpkorstanje
Copy link
Contributor

mpkorstanje commented Dec 22, 2022

If you run mvn clean install it will run all tests and fix the formatting.

@mpkorstanje mpkorstanje self-assigned this Dec 22, 2022
@codecov
Copy link

codecov bot commented Dec 22, 2022

Codecov Report

Merging #2667 (9900d2d) into main (e32379e) will decrease coverage by 0.00%.
The diff coverage is 80.00%.

❗ Current head 9900d2d differs from pull request most recent head 42cd2a6. Consider uploading reports for the commit 42cd2a6 to get more accurate results

@@             Coverage Diff              @@
##               main    #2667      +/-   ##
============================================
- Coverage     84.82%   84.82%   -0.01%     
+ Complexity     2695     2694       -1     
============================================
  Files           322      322              
  Lines          9576     9574       -2     
  Branches        907      907              
============================================
- Hits           8123     8121       -2     
  Misses         1121     1121              
  Partials        332      332              
Impacted Files Coverage Δ
...io/cucumber/core/runner/TestAbortedExceptions.java 91.66% <80.00%> (-1.20%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@mpkorstanje mpkorstanje merged commit f53abb6 into cucumber:main Dec 22, 2022
@aslakhellesoy
Copy link
Contributor

Hi @jkronegg,

Thanks for your making your first contribution to Cucumber, and welcome to the Cucumber committers team! You can now push directly to this repo and all other repos under the cucumber organization! 🍾

In return for this generous offer we hope you will:

  • ✅ Continue to use branches and pull requests. When someone on the core team approves a pull request (yours or someone else's), you're welcome to merge it yourself.
  • 💚 Commit to setting a good example by following and upholding our code of conduct in your interactions with other collaborators and users.
  • 💬 Join the community Slack channel to meet the rest of the team and make yourself at home.
  • ℹ️ Don't feel obliged to help, just do what you can if you have the time and the energy.
  • 🙋 Ask if you need anything. We're looking for feedback about how to make the project more welcoming, so please tell us!

On behalf of the Cucumber core team,
Aslak Hellesøy
Creator of Cucumber

@mpkorstanje
Copy link
Contributor

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bad performance during TestStep creation
3 participants