Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remap paths in UI tests by default #105924

Merged
merged 3 commits into from
Jan 21, 2023
Merged

Remap paths in UI tests by default #105924

merged 3 commits into from
Jan 21, 2023

Conversation

TimNN
Copy link
Contributor

@TimNN TimNN commented Dec 19, 2022

If you think this needs further discussions / something RFC-like, please let me know the best forum for that.

This PR runs UI tests with a remapped "src base" directory by default.

Why? Because some UI tests currently depend on the length of the absolute path to the src/test/ui directory. Remapping makes the tests independent of the absolute path.

The path to the source file (which is absolute on CI) is part of the type name of closures. rustc diagnostic output depends on the length of type names (long type names are truncated). So a long absolute path leads to long closure type names, which leads to truncation and changed diagnostics.

(I initially tried just disabling type name truncation, but that made some error messages stupid long (thousands of characters, IIRC)).

Additional changes:

  • All boolean compiletest directives now support explicit no- versions to disable them.
  • Adapt existing tests when necessary:
    • Disable remapping for individual tests that fail with it enabled (when there's no obvious alternative fix).
    • For tests that already check something remapping related switch to the new option unless we gain something significant by keeping the manual remap.

Passed Windows CI in https://github.com/rust-lang/rust/actions/runs/3933100590

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 19, 2022
@TimNN TimNN force-pushed the ui-remap branch 2 times, most recently from 7ec33cb to ba526ed Compare December 20, 2022 11:46
@ChrisDenton
Copy link
Member

(edit: Though I see now that Windows seems to only run for auto so I guess I'll have to figure something else out for that).

See the Using CI to test section of the dev guide for how to test Windows jobs in a PR.

@TimNN
Copy link
Contributor Author

TimNN commented Jan 6, 2023

@ChrisDenton: Thanks for pointing out the docs! I though something like that would be possible, but didn't realize it was officially documented. (I just got back to this PR and finished setting up a windows VM...)

@rustbot rustbot added the T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. label Jan 6, 2023
@TimNN TimNN force-pushed the ui-remap branch 3 times, most recently from c7ec808 to 1f9a3d3 Compare January 6, 2023 13:46
@TimNN TimNN changed the title WIP: Remap paths in UI tests by default Remap paths in UI tests by default (and reverse path mappings for diagnostics where possible) Jan 6, 2023
@TimNN TimNN changed the title Remap paths in UI tests by default (and reverse path mappings for diagnostics where possible) Remap paths in UI tests by default Jan 6, 2023
@TimNN
Copy link
Contributor Author

TimNN commented Jan 6, 2023

r? bootstrap

@jyn514
Copy link
Member

jyn514 commented Jan 10, 2023

r? @Mark-Simulacrum

(#106657)

@rustbot rustbot assigned Mark-Simulacrum and unassigned jyn514 Jan 10, 2023
@TimNN
Copy link
Contributor Author

TimNN commented Jan 13, 2023

I've been considering this a bit more and will be moving the first two commits to separate PRs (rental & path mapping undo).

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 13, 2023
Update `rental` hack to work with remapped paths.

This PR simply switches to an already-existing helper instead of hard-coding a specific enum variant. The new revision of the test fails without the other changes in this PR.

Context: I'm exploring running UI tests with remapped paths by default in rust-lang#105924 and the rental test was one of the ones that failed.

This may also be useful in the context of rust-lang/rfcs#3127 ("New rustc and Cargo options to allow path sanitisation by default").
@Mark-Simulacrum
Copy link
Member

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 14, 2023
JohnTitor pushed a commit to JohnTitor/rust that referenced this pull request Jan 14, 2023
Update `rental` hack to work with remapped paths.

This PR simply switches to an already-existing helper instead of hard-coding a specific enum variant. The new revision of the test fails without the other changes in this PR.

Context: I'm exploring running UI tests with remapped paths by default in rust-lang#105924 and the rental test was one of the ones that failed.

This may also be useful in the context of rust-lang/rfcs#3127 ("New rustc and Cargo options to allow path sanitisation by default").
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 16, 2023
Heuristically undo path prefix mappings.

Because the compiler produces better diagnostics if it can find the source of (potentially remapped) dependencies.

The new test fails without the other changes in this PR. Let me know if you have better suggestions for the test directory. I moved the existing remapping test to be in the same location as the new one.

Some more context: I'm exploring running UI tests with remapped paths by default in rust-lang#105924 and this was one of the issues discovered.

This may also be useful in the context of rust-lang/rfcs#3127 ("New rustc and Cargo options to allow path sanitisation by default").
@TimNN
Copy link
Contributor Author

TimNN commented Jan 16, 2023

  • Rebased on latest master, on top of the split-out rental and path mapping changes.
  • Updated more test cases to use the new option instead of manual remapping.
  • Updated the PR description in the first comment.
  • Temporarily added the Windows CI again.

Should generally be ready for review, I'll ping the bot to update the labels once Windows CI is green.

@TimNN
Copy link
Contributor Author

TimNN commented Jan 16, 2023

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 16, 2023
@Mark-Simulacrum
Copy link
Member

So a long absolute path leads to long closure type names, which leads to truncation and changed diagnostics.

Hm, I thought that we had a separate filtering pass which made this not true - #96551?

Or is the truncation happening within rustc?

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 21, 2023
@TimNN
Copy link
Contributor Author

TimNN commented Jan 21, 2023

The truncation is happening in rustc. It shows a very abbreviated type name and includes another note stating something along the lines of "the full type name has been written to [some tmp path]".

@TimNN
Copy link
Contributor Author

TimNN commented Jan 21, 2023

Here is an example diff we see if the filename is too long:

---- [ui] rust-llvm-integrate-prototype/tests/ui/never_type/fallback-closure-wrap.rs#fallback stdout ----

diff of stderr:

10	   |
11	   = note: expected unit type `()`
12	                   found type `!`
-	   = note: required for the cast from `[closure@$DIR/fallback-closure-wrap.rs:18:40: 18:47]` to the object type `dyn FnMut()`
+	   = note: required for the cast from `[closure@fallback-closure-wrap.rs:18:40]` to the object type `dyn FnMut()`
+	   = note: the full name for the casted type has been written to '$TEST_BUILD_DIR/never_type/fallback-closure-wrap.fallback/fallback-closure-wrap.long-type-6621918892693463368.txt'
14	
15	error: aborting due to previous error
16	

edit: not sure if this going to work, but let's try to avoid spamming people: @rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 21, 2023
@Mark-Simulacrum
Copy link
Member

Huh! This is news to me, maybe a recent change.

In any case, this seems like a reasonable approach to solving that problem -- @bors r+ rollup=iffy

@bors
Copy link
Contributor

bors commented Jan 21, 2023

📌 Commit cd1d0bc has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 21, 2023
@bors
Copy link
Contributor

bors commented Jan 21, 2023

⌛ Testing commit cd1d0bc with merge 52372f9...

@bors
Copy link
Contributor

bors commented Jan 21, 2023

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 52372f9 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 21, 2023
@bors bors merged commit 52372f9 into rust-lang:master Jan 21, 2023
@rustbot rustbot added this to the 1.68.0 milestone Jan 21, 2023
@TimNN TimNN deleted the ui-remap branch January 21, 2023 20:58
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (52372f9): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.3% [-0.3%, -0.3%] 2
Improvements ✅
(secondary)
-0.6% [-0.8%, -0.3%] 7
All ❌✅ (primary) -0.3% [-0.3%, -0.3%] 2

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-5.1% [-5.1%, -5.1%] 1
All ❌✅ (primary) - - 0

Cycles

This benchmark run did not return any relevant results for this metric.

bors added a commit to rust-lang/miri that referenced this pull request Jan 23, 2023
Heuristically undo path prefix mappings.

Because the compiler produces better diagnostics if it can find the source of (potentially remapped) dependencies.

The new test fails without the other changes in this PR. Let me know if you have better suggestions for the test directory. I moved the existing remapping test to be in the same location as the new one.

Some more context: I'm exploring running UI tests with remapped paths by default in rust-lang/rust#105924 and this was one of the issues discovered.

This may also be useful in the context of rust-lang/rfcs#3127 ("New rustc and Cargo options to allow path sanitisation by default").
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants