Feature Request: Run tests in parallel

[iwubcode]

I believe the test executables currently run one at a time. However, longer running tests can queue up the short-quick tests and make the overall test run take longer than it could if the tests were ran in parallel. It'd be nice if the test-explorer supported multiple tests to be ran at a single time. Maybe a MaxProcesses setting or something to control how many run at once?

Thanks again!

[JohnnyHendriks]

There is some support for running tests in parallel via the Test Explorer. I think you have to set the <MaxCpuCount> setting in <RunConfiguration> for this. The limitation here is that it only runs tests in different executables in parallel. Tests that are part of the same executable are not run in parallel. Note you have to also activate the parallel toggle button on the Test Explorer to activate this functionality.

I agree having parallel execution also when tests are in the same executable would be a nice to have. I will explore this possibility and if possible add it to a future version. As for some tests it would probably be better that they are not run in parallel with other tests I will probably include an opt-out via a tag (e.g., [tec2-notparallel]) that you can set for a test case that needs to opt-out when parallel execution with the same executable is configured to be on.

[iwubcode]

Ah, multiple processes (with the <MaxCpuCount>) is likely good enough for me, as I have quite a few processes that are created. Thank you!

[Trass3r]

Any news on this?

(Just for the record, catch itself doesn't support parallelism: https://github.com/catchorg/Catch2/blob/master/docs/limitations.md#running-multiple-tests-in-parallel)

[JohnnyHendriks]

I'm currently working on finalizing a new version, where it is possible to choose to run multiple tests in a single process from within the test explorer. Currently for each individual test case a new process is started. If you have a lot of test cases in a single Catch2 executable, this has the potential to make the tests run significantly faster via the Test Explorer. Next step would be to add the possibility to also run the tests for a single Catch2 executable in parallel. My goal is to have the new version out before end of the year.

[horenmar]

For running tests in single Catch2 binary in parallel, look at sharding (assuming a new-enough version of Catch2).