-
-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can no longer limit the number of parallel sessions from JUnit 5 #9359
Comments
Thank you for providing the details and the link. I was able to use https://github.com/rbok78/framework-template to recreate the issue. |
@pujagani thanks for looking into this. I created a reproducible project for you to look at with the latest beta 3 code: should be able to just |
Thank you for sharing the sample repo and the update. I was able to reproduce the error. After running the tests a considerable number of times, I see the error intermittently where the number of tests exceeds the parallelism value. Upon digging deeper, I suspect the change in behaviour is due to the HTTP client used. Selenium Alpha-4 made a switch from OkHttp to AsyncHttpClient. This switch might have lead to a change in behaviour since both have very different threading models in the background. JUnit leverages the fork-join pool to run the tests in parallel. Thread pools in Java have a core thread pool size and max thread pool size. The parallelism value is used for configuring the core thread pool size. Whereas the max thread pool size to kept to the recommended 256 + the parallelism value. https://github.com/junit-team/junit5/blob/43638eb6a870e0d6c49224053dfeb39dcf0ef33f/junit-platform-engine/src/main/java/org/junit/platform/engine/support/hierarchical/DefaultParallelExecutionConfigurationStrategy.java#L44 A workaround for this is to have a custom strategy to configure the thread pool size. Providing an example below:
Update the pom.xml as follows:
I suggest please try the above changes and let me know if this is a workable solution. Appreciate the feedback. Thank you! |
Ah, I didn't realize alpha 4 is when we switched to the new AsyncHttpClient that completely makes sense, then. So, yes, this now works for "3"
Which indicates that the parallel is kind of all over the place. I've updated the repo with the code you suggested. I'm using values based on system properties I'm setting in the pom. I'm not familiar with the differences between the values getting set, so perhaps those shouldn't all be the same? |
Thank you for the quick feedback and for trying out the sample configuration.
Regarding providing the configuration, ideally, the values should be using the available processors (using |
Linked PR with a fix was merged and it will be available in @pujagani, could you please share the steps how to test this before the |
Thanks, Diego! Step 1 Step 2 The jar will be created in the following path :
Step 3 Step 4 Now everything is in place with the new changes. Step 5 |
Yup, this was fixed in beta4. I can now tell people to use JUnit 5 again! |
I have a similar problem. Please, tell me, what am I doing wrong? |
Thank you for providing the test project. You might want to add a Custom Strategy and add the details to pom as directed in #9359 (comment). Please try it out and provide feedback if that helps. |
@dsmoons you can see my repo reproduction here: https://github.com/titusfortner/bug9359/blob/main/pom.xml#L70 When the se version is changed to 4 beta this code works as expected |
@pujagani Maybe this can be fixed without using a Custom Strategy? It seems to be Selenium's bug, not JUnit. |
The JUnit fix for running in parallel when a library is using I linked to my repo so you can see the Custom Strategy example with working code. |
@titusfortner I understand, thank you. I will use the Custom Strategy with Beta4. |
No, it's a limitation of how JUnit 5 has implemented parallelization. The library that Java Selenium is using to manage asynchronous http calls (to support DevTools functionality), uses commands that JUnit assumes that no one will use. Custom Strategy is their recommended solution to avoid this. |
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
As pointed out in #10113 the solution only works for Java 11. Of interest, Selenium 4.0.0-alpha-3 does work with JUnit 5 in all Java versions, so it is related to what Perhaps this fixes things? junit-team/junit5#1858 |
Continuing discussion in #10113 |
💥 Regression Report
Can no longer limit the number of parallel sessions from JUnit 5 (v3.0.0-M5)
Likely related to this ticket: junit-team/junit5#2273
Last working Selenium version
Worked up to version: 4.0.0-alpha-3
Stopped working in version: 4.0.0-alpha-4
To Reproduce
This is with Maven Surefire plugin, but should be the same behavior for Gradle I think.
Run
mvn test
with a test suite that has this plugin config in thepom.xml
file:Expected behavior
Only 3 driver sessions should be running at a time, instead, all tests in the suite are executed.
The text was updated successfully, but these errors were encountered: