Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check stdout/stderr of run-pass run portion #63751

Closed
Mark-Simulacrum opened this issue Aug 20, 2019 · 7 comments · Fixed by #63825
Closed

Check stdout/stderr of run-pass run portion #63751

Mark-Simulacrum opened this issue Aug 20, 2019 · 7 comments · Fixed by #63825
Assignees
@tmandry
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.

Comments

@Mark-Simulacrum
Copy link
Member

We currently check the compiler's output, but it's sometimes useful to check that the run-pass test itself is outputting the same as before. In particular, @tmandry expressed a desire to test libtest in this fashion.

Taking a look at the code, we probably want to add another header flag (check-run-results) around here in this file; and then when running the generated executable here we should replicate this expected loading code down. If possible, it'd be great to extract some sort of function that would load results and compare them, but I'm not sure how feasible that is in a clean manner.
@Mark-Simulacrum Mark-Simulacrum added A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. labels Aug 20, 2019
@Mark-Simulacrum
Copy link
Member Author

I've assigned to @tmandry for now because I believe they might be interested in landing this more quickly than perhaps mentoring will be able to but let us know if you're interested!

@Centril
Copy link
Contributor

Centril commented Aug 20, 2019

This would be great; we could use this to simplify the hacks in testing dbg! for example.

@tmandry
Copy link
Member

tmandry commented Aug 21, 2019

I would like to get it soon, but I probably won't be getting to this until next week at least :)

@tmandry
Copy link
Member

tmandry commented Aug 21, 2019

Thanks for providing the mentoring instructions, by the way!

@nathanwhit
Copy link
Member

I'm interested in working on this (or rather, I've done some work on this based on the mentoring instructions) if that's okay. Based on what I've done so far, I think I'll be ready to submit a PR within the next day or so.

@nathanwhit
Copy link
Member

To test the functionality, I updated the test for the dbg! macro (noted by @Centril) to use the new header flag. Would it be more reasonable to make a separate PR for that once these changes are approved, or could I include it in the same PR? Apologies if this is off-topic.

@Mark-Simulacrum
Copy link
Member Author

Not at all! That seems like a reasonable change to make in the same PR, though it would be good to split it out into a separate commit for easier reviewing. Thanks!

nathanwhit added a commit to nathanwhit/rust that referenced this issue Aug 23, 2019
@nathanwhit
Compare run-pass outputs if check-run-results flag enabled
b9ba8f9 
Addresses rust-lang#63751
@nathanwhit nathanwhit mentioned this issue Aug 23, 2019
Merged
nathanwhit added a commit to nathanwhit/rust that referenced this issue Aug 28, 2019
@nathanwhit
Update dbg macro test to use check-run-results flag for run-pass test
12e0420 
Updates the dbg-macro-expected behavior test to remove the workaround and use the `check-run-result` flag/feature in compiletest. This serves to test the feature on a real use-case (as mentioned in rust-lang#63751)
Centril added a commit to Centril/rust that referenced this issue Aug 29, 2019
@Centril
Rollup merge of rust-lang#63825 - nathanwhit:check-run-results, r=Mar…
57cc7f6 
…k-Simulacrum

Allow checking of run-pass execution output in compiletest

Closes rust-lang#63751
Adds a `check-run-results` flag to compiletest headers, which if enabled checks the output of the execution of a run-pass test's binary against expected output.
Centril added a commit to Centril/rust that referenced this issue Aug 29, 2019
@Centril
Rollup merge of rust-lang#63825 - nathanwhit:check-run-results, r=Mar…
7e6cf90 
…k-Simulacrum

Allow checking of run-pass execution output in compiletest

Closes rust-lang#63751
Adds a `check-run-results` flag to compiletest headers, which if enabled checks the output of the execution of a run-pass test's binary against expected output.
bors added a commit that referenced this issue Sep 3, 2019
@bors
Auto merge of #63825 - nathanwhit:check-run-results, r=Mark-Simulacrum
c11b9bb 
Allow checking of run-pass execution output in compiletest

Closes #63751
Adds a `check-run-results` flag to compiletest headers, which if enabled checks the output of the execution of a run-pass test's binary against expected output.
@bors bors closed this as completed in 6c18a3d Sep 4, 2019
@tmandry tmandry mentioned this issue Sep 14, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tmandry tmandry

Labels
A-testsuite Area: The testsuite used to check the correctness of rustc E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow checking of run-pass execution output in compiletest nathanwhit/rust
4 participants
@Centril @tmandry @Mark-Simulacrum @nathanwhit