Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ui tests should just show the stderr diff #46826

Closed
nikomatsakis opened this issue Dec 18, 2017 · 8 comments
Closed

ui tests should just show the stderr diff #46826

nikomatsakis opened this issue Dec 18, 2017 · 8 comments
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@nikomatsakis
Copy link
Contributor

Currently, when a UI test fails, it prints a sort of overwhelming amount of stuff:

println!("normalized {}:\n{}\n", kind, actual);
println!("expected {}:\n{}\n", kind, expected);
println!("diff of {}:\n", kind);
for diff in diff::lines(expected, actual) {
match diff {
diff::Result::Left(l) => println!("-{}", l),
diff::Result::Both(l, _) => println!(" {}", l),
diff::Result::Right(r) => println!("+{}", r),
}
}

I think what I would like is something like:

If the expected text is empty, just dump the actual (normalized) output, so people can copy-and-paste it if they want.

Else, just print the diff. And probably it'd be nice to strip off the "prefix" and "suffix" of unchanged lines, and just show the parts that vary (we could imagine breaking things into hunks).

@nikomatsakis nikomatsakis added A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 18, 2017
@nikomatsakis
Copy link
Contributor Author

cc @oli-obk @petrochenkov -- two people who care about test suite I know, sorry if I'm forgetting to cc others =)

@durka
Copy link
Contributor

durka commented Dec 19, 2017

And probably it'd be nice to strip off the "prefix" and "suffix" of unchanged lines, and just show the parts that vary (we could imagine breaking things into hunks).

Sounds like reinventing features from existing diff libraries where there might be a crate that can do it nicely!

@omakk
Copy link

omakk commented Dec 19, 2017

Hello!

I'm a Rust-learner and was wondering if I could pick up this issue?

@oli-obk
Copy link
Contributor

oli-obk commented Dec 20, 2017

@omakk great! Feel free to ping me on irc or ask any questions here.

I think an experimental implementation of using a diff crate would be the way to go as @durka suggested.

@ritiek
Copy link
Member

ritiek commented Jan 3, 2018

@omakk If you're busy at the moment, do you mind if I take on this issue?

@omakk
Copy link

omakk commented Jan 3, 2018

@ritiek Yes, go ahead! I unfortunately can not find the time to look at this. Apologies for the late update.

@petrochenkov
Copy link
Contributor

cc @petrochenkov

I never read these diffs, personally, just run update-all-references.sh immediately if there are errors.

kennytm added a commit to kennytm/rust that referenced this issue Jan 12, 2018
…omatsakis

Show only stderr diff when a ui test fails

Addresses rust-lang#46826.

This PR will print the normalized output if expected text is empty otherwise it will just print the diff.

Should we also show a few (actual == expected) lines above & below when displaying the diff? What about indicating line numbers as well so one can quickly check mismatch lines in .stderr file?
@ritiek
Copy link
Member

ritiek commented Jan 14, 2018

I suppose this can now be closed with #47185.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

6 participants