Introduce mechanism to stop test run with first failing test

[jlink]

Subject says it all. Would be especially useful for TCR (test && commit || reset) workflow.
Feature could be globally configured or switched of from IDE / build tool.

#431 is related and pitest/pitest-junit5-plugin#15 might also profit.

[marcphilipp]

I wonder if/how this could be implemented on the platform level. I guess we could provide a platform feature to enable it and add a isStopped() method to ExecutionRequest which engines should check before starting a execution of a new test. Thoughts?

[jlink]

You're right, a new channel from platform to engine is needed to signal the stop. I wonder if such a channel could be used for other useful stuff as well...

The simplest solution I can think of, however, is System.exit. I'm not sure, though, what kind of adverse effects that might bring.

[marcphilipp]

Since build tools/IDEs run in the same JVM I don't think that would be a viable solution. Moreover, even if a test run's being cancelled, we should stick to the contracts in TestExecutionListener.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution.

[stale]

This issue has been automatically closed due to inactivity. If you have a good use case for this feature, please feel free to reopen the issue.