-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Address JSON test failures #42960
Address JSON test failures #42960
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good from an innerloop perspective but I'm concerned about the outerloop quality with this change. Isn't there a way to make this quicker (parallelism) or can't we reduce the number of tests without loosing much or any coverage? cc @ericstj in case he has opinions
src/libraries/System.Text.Json/tests/Serialization/ContinuationTests.cs
Outdated
Show resolved
Hide resolved
@ViktorHofer does outerloop have the same sort of timeouts as innerloop? I thought it didn't. Adding parallelism could decrease execution time of this one test in isolation, but may not shorten the overall test execution if other tests provide enough work to saturate the cores. I don't have enough of a handle over the fanout of our test runner to understand what's best. @ViktorHofer are you aware of guidelines here? |
I'm not aware of any guidelines here but if my memory serves me right, @stephentoub had to deal with such a large amount of data rows in the past. Stephen do you maybe know to best handle such a large amount of rows? |
AFAICT we don't have time limits on outer-loop tests, so I'll keep this PR as-is to unblock clean CI per #42677. There likely multiple System.Text.Json tests that could benefit from some optimization, so I'll defer a one-off change for |
@layomia do we have a tracking issue for that? |
@ViktorHofer I'll use #42677 to track this. |
Thanks |
Updating these tests as they are newly added and resource intensive and they were checked in around the time System.Text.Json tests started timing out on Mono + Windows - #42677.
I assume what is happening here is that the total amount of time it takes to run all the JSON tests has increased beyond the allotted time, which is why the new tests are being disabled, rather than all the unit tests qualified with
[Long Running Test]
in the various failure logs in the above issue (e.g. this one).Also making
InvalidJsonForTypeShouldFail
outerloop to address the failure onnet5.0-Windows_NT-Release-x86-CoreCLR_checked-Windows.10.Amd64.Open
. Fixes #42817.