Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split out non concurrent test collections. #6937

Merged
merged 5 commits into from
Jan 8, 2024

Conversation

michaelgsharp
Copy link
Member

Split out non-concurrent test collections to improve performance.

@ghost ghost assigned michaelgsharp Jan 3, 2024
Copy link

codecov bot commented Jan 3, 2024

Codecov Report

Attention: 65 lines in your changes are missing coverage. Please review.

Comparison is base (373a864) 68.80% compared to head (a8dfb40) 68.81%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main    #6937    +/-   ##
========================================
  Coverage   68.80%   68.81%            
========================================
  Files        1249     1249            
  Lines      249436   249644   +208     
  Branches    25468    25481    +13     
========================================
+ Hits       171635   171783   +148     
- Misses      71211    71271    +60     
  Partials     6590     6590            
Flag Coverage Δ
Debug 68.81% <75.00%> (+<0.01%) ⬆️
production 63.28% <95.45%> (+0.01%) ⬆️
test 88.41% <68.04%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
.../Microsoft.ML.TorchSharp/NasBert/NasBertTrainer.cs 80.36% <100.00%> (+0.24%) ⬆️
...c/Microsoft.ML.TorchSharp/TorchSharpBaseTrainer.cs 84.98% <100.00%> (ø)
...ft.ML.Core.Tests/UnitTests/TestResourceDownload.cs 0.00% <ø> (ø)
...ft.ML.TensorFlow.Tests/TensorFlowEstimatorTests.cs 98.50% <ø> (ø)
...t/Microsoft.ML.TensorFlow.Tests/TensorflowTests.cs 91.71% <ø> (ø)
test/Microsoft.ML.Tests/DatabaseLoaderTests.cs 94.41% <100.00%> (+0.17%) ⬆️
...rosoft.ML.TorchSharp.Tests/ObjectDetectionTests.cs 100.00% <ø> (ø)
test/Microsoft.ML.TorchSharp.Tests/QATests.cs 40.50% <ø> (ø)
...oft.ML.TorchSharp.Tests/TextClassificationTests.cs 85.17% <ø> (ø)
src/Microsoft.ML.TorchSharp/NasBert/NerTrainer.cs 91.10% <94.73%> (+2.86%) ⬆️
... and 1 more

... and 5 files with indirect coverage changes

@michaelgsharp michaelgsharp marked this pull request as ready for review January 4, 2024 20:58
test/Microsoft.ML.Tests/DatabaseLoaderTests.cs Outdated Show resolved Hide resolved
<ProjectReference Include="..\..\src\Microsoft.Data.Analysis\Microsoft.Data.Analysis.csproj" />
</ItemGroup>

<ItemGroup Condition="'$(TargetArchitecture)' != 'x64'">
Copy link
Member

@ericstj ericstj Jan 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider trying out @ViktorHofer's solution for excluding the entire project from the SLN traversal. dotnet/xdt@main...DoCoolFiltering

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought I would do that in another PR so that we can start getting the benefits now of the faster tests/no timeouts. And I have excluded this assembly from helix testing as well for non x64, so it won't affect those tests at all either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not fully understand @ericstj comment. This looks excluding some source file when running on x64. How excluding the entire project work here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this was just a consider. I'm completely fine excluding things as you did, since it looks like we already do that for a lot of other projects. If you feel strongly about not building this then file an issue on that.

@michaelgsharp michaelgsharp requested a review from tarekgh January 5, 2024 22:42
@michaelgsharp
Copy link
Member Author

@tarekgh could you take a quick look at this when you have a minute? It just splits out some test assemblies so we can run more in parallel.

Copy link
Member

@tarekgh tarekgh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a couple of questions. LGTM!

@michaelgsharp michaelgsharp merged commit 9f4a389 into dotnet:main Jan 8, 2024
25 checks passed
@michaelgsharp
Copy link
Member Author

/backport to release/3.0

Copy link

github-actions bot commented Jan 9, 2024

Started backporting to release/3.0: https://github.com/dotnet/machinelearning/actions/runs/7455070236

@michaelgsharp
Copy link
Member Author

/backport to release/3.0

Copy link

github-actions bot commented Jan 9, 2024

Started backporting to release/3.0: https://github.com/dotnet/machinelearning/actions/runs/7464450572

@github-actions github-actions bot locked and limited conversation to collaborators Feb 9, 2024
@michaelgsharp michaelgsharp deleted the test-splitting branch November 18, 2024 20:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants