-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Testing light gbm bad allocation #6968
Testing light gbm bad allocation #6968
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #6968 +/- ##
=======================================
Coverage 68.80% 68.81%
=======================================
Files 1249 1249
Lines 249686 249686
Branches 25485 25485
=======================================
+ Hits 171806 171814 +8
+ Misses 71286 71282 -4
+ Partials 6594 6590 -4
Flags with carried forward coverage won't be shown. Click here to find out more.
|
/azp run |
Azure Pipelines successfully started running 2 pipeline(s). |
/azp run |
Azure Pipelines successfully started running 2 pipeline(s). |
/azp run |
Azure Pipelines successfully started running 2 pipeline(s). |
/azp run |
Azure Pipelines successfully started running 2 pipeline(s). |
[CollectionDefinition(nameof(NoParallelizationDefinition), DisableParallelization = true)] | ||
public class NoParallelizationDefinition { } | ||
|
||
[Collection(nameof(NoParallelizationDefinition))] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there something specific about these LightGBM tests that differs from others?
https://github.com/search?q=repo%3Adotnet%2Fmachinelearning%20LightGBMFact&type=code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Honestly I'm not sure. I just put this tag around the test I saw failing and it seems to have worked. So its possible there is? Or its possible it just has to do with test ordering? I'm not sure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be good to at comment that we've observed concurrency issues in LightGBM when running these in parallel. Consider keeping an issue open to track that and link to the issue since we still don't understand why it's failing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems reasonable since it decreases CI flakiness - but make sure we have this commented with an issue linking.
Just saw this in 3.0 branch Code Coverage build. Maybe we should backport this test reliability fix? |
/backport to release/3.0 |
Started backporting to release/3.0: https://github.com/dotnet/machinelearning/actions/runs/8272862595 |
Changes some of the LightGBM tests to run sequentially to try and fix the bad allocation issue.
This is in regards to issue #6961. I'll leave that issue open so we can continue to monitor after this goes in.