-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add error check for skipped tests in merged groups #84284
Conversation
Tagging subscribers to this area: @hoyosjs Issue DetailsIf I test is written in the old style (with a Main and OutputType==Exe without an attribute such as RequiresProcessIsolation) in a merged test group directory, it will be skipped. This change adds a check to detect those cases. I have struggled with ways to automatically set OutputType. Directory.Build.props is too early (the test project file will override it). Directory.Build.targets is too late as the C# targets files will already have been processed and set other variables based on the value of OutputType. This Target doesn't execute until later, but since it is an error it doesn't matter how those additional properties were set. Since this adds more boilerplate to each merged test directory, I created a src/tests/Directory.Merged.props to share all of that. This catches 3 tests that aren't currently executing. Unrelated:
Completes #84182
|
PTAL @trylek @jkoritzinsky |
/azp run runtime-coreclr outerloop |
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great to me, thanks Mark for cleaning this up and for fixing the previously overlooked cases!
<InMergedTestDirectory>true</InMergedTestDirectory> | ||
<BuildAsStandalone Condition="'$(BuildAsStandalone)' == ''">false</BuildAsStandalone> | ||
|
||
<AssemblyName Condition="'$(BuildAsStandalone)' != 'true'">$(MSBuildProjectName.Replace('_il_d', '').Replace('_il_r', ''))</AssemblyName> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, it would be helpful to have some comments. Especially, to describe why we need this weird _il_d
/_il_r
handling.
Both errors you are seeing in the outerloop run should be fixed now by my and Michal's PR I merged it in earlier today but after you triggered the outerloop runs. |
Thank you all. @BruceForstall I will fold that into my next changes (to merge immediately since a new check and new tests are a race condition in the repo) |
If I test is written in the old style (with a Main and OutputType==Exe without an attribute such as RequiresProcessIsolation) in a merged test group directory, it will be skipped. This change adds a check to detect those cases.
I have struggled with ways to automatically set OutputType. Directory.Build.props is too early (the test project file will override it). Directory.Build.targets is too late as the C# targets files will already have been processed and set other variables based on the value of OutputType. This Target doesn't execute until later, but since it is an error it doesn't matter how those additional properties were set.
Since this adds more boilerplate to each merged test directory, I created a src/tests/Directory.Merged.props to share all of that.
This catches 3 tests that aren't currently executing.
Unrelated:
Resolves #84182