Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce conditionally disabling tests #86

Open
cspray opened this issue May 10, 2021 · 1 comment
Open

Introduce conditionally disabling tests #86

cspray opened this issue May 10, 2021 · 1 comment
Labels
enhancement New feature or request
Milestone

Comments

@cspray
Copy link
Member

cspray commented May 10, 2021

Right now you can only disable a test statically, sometimes you might want to disable a test only if certain conditions are met. For example, if the code under test is optional for your application and requires a certain extension the test suite should disable any tests that require that extension if it is not available. This is currently not possible. We should avoid introducing a TestCase::markDisabled... it requires all of the hooks and test processing to still be invoked when we could, and should, be skipping past these potentially costly, error-triggering procedures. Instead we should introduce a #[DisabledIf($method, $reason)] Attribute. The $method should be a public method on the class it was annotated on. If it returns true the test will be disabled otherwise it will run as normal.

@cspray cspray added enhancement New feature or request good first issue Good for newcomers labels May 10, 2021
@cspray cspray added this to the 0.6.0 Release milestone May 10, 2021
@cspray cspray removed the good first issue Good for newcomers label Jul 31, 2021
@cspray
Copy link
Member Author

cspray commented Jul 31, 2021

I decided to move this out of the 0.6 release because getting this implemented is going to require changing around a fair amount of how we think of disabled tests. Statistics for pre running will need to be adjusted because we no longer can reliably tell whether or not a given test/case/suite is disable until runtime. I believe that this feature is important enough and has enough impact on the existing codebase that the implementation needs more thought.

@cspray cspray modified the milestones: 0.6.0 Release, 0.7.0 Release Jul 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant