-
-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Include parent testset descriptions in immediate test report #32938
base: master
Are you sure you want to change the base?
Conversation
I think this is a good change. Please speak out if you have any objections. |
WeakRef seems like the wrong choice, why not just a normal reference? Otherwise, seems like @andreasnoack already approved this and so it's good to merge. |
I removed WeakRef |
@vtjnash Do you want to review the post-merge diff? Or can I merge this? |
You'll need to fix the doctest failures. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't remember ever looking at this before
Unless I'm missing something, given that the parent testsets are stored in the task local storage, it doesn't seem necessary to add |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @rfourquet for review. I agree: since we got to this record
function by calling get_testset
, and that function had the full list of testset parents already, it looks like we don't need to add a field to do this printing.
Thanks, @rfourquet. Yes, that's a good point. I don't remember why I didn't take that strategy. It's probably easy to fix. |
With this PR, the failure messages printed immediately after the test failure (i.e., not
Test Summary
) include testset descriptions of all parent test sets. This clarifies the context in which the failure occurs. This is important especially in Julia CI (Base.runtests
) which does not printTest Summary
. Without a change like this, heavily parameterized test likestdlib/LinearAlgebra/test/addmul.jl
cannot produce any meaningful failure message (JuliaLang/LinearAlgebra.jl#655).Some examples. The contents between
Test Set:
andTest Failed at
are added in this PR: