-
Notifications
You must be signed in to change notification settings - Fork 597
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changed exact-match tests for ModelSegments to allow for numerical differences due to Java version. #8111
base: master
Are you sure you want to change the base?
Changed exact-match tests for ModelSegments to allow for numerical differences due to Java version. #8111
Conversation
480c92c
to
4a36405
Compare
Maybe it would be better to pair this with the upcoming Java 17 upgrade, so that we don't have to explicitly check the Java version? What do you think @cmnbroad ? |
Github actions tests reported job failures from actions build 3577583009
|
…fferences due to Java version.
4a36405
to
bc9b5a7
Compare
I suppose we can wait, but I still think there is some inherent instability that is transient that we're going to have to deal with at some point. It looks like the last time I ran the CI tests on the Java 17 branch, these tests were failing on the docker job but passing on the the non-docker job, both with (probably different versions) of Java 17. I was hoping that narrowing it down to a small range of Java versions might help. |
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #8111 +/- ##
===============================================
+ Coverage 84.031% 86.643% +2.612%
- Complexity 37445 38950 +1505
===============================================
Files 2335 2335
Lines 182679 182684 +5
Branches 20053 20053
===============================================
+ Hits 153507 158283 +4776
+ Misses 22124 17367 -4757
+ Partials 7048 7034 -14
|
See some of my findings about numerical differences across 8/11/17 and possible causes in this old Slack thread: https://broadinstitute.slack.com/archives/C1HH1V5EC/p1657634295565369 We’re starting to get into some relatively hairy issues there, IMO! But just in case it wasn’t clear: 1) None of these numerical differences should be scientifically concerning in the end, and 2) I think we still have numerical reproducibility within each fixed Java version (although if we happen to see any evidence to the contrary, please point to them here). So I don’t think we have too much to worry about once the test infrastructure settles. |
Also, looks like tests are passing, so the runtime version check seems to have done the trick. I’ll leave it up to you both what the next steps should be, no strong opinions on my end! |
Closes #8107.