Fixup Windows verbatim paths when used with the `include!` macro #125205

ChrisDenton · 2024-05-17T08:30:16Z

On Windows, the following code can fail if the OUT_DIR environment variable is a verbatim path (i.e. begins with \\?\):

include!(concat!(env!("OUT_DIR"), "/src/repro.rs"));

This is because verbatim paths treat / literally, as if it were just another character in the file name.

The good news is that the standard library already has code to fix this. We can simply use components to normalize the path so it works as intended.

rustbot · 2024-05-17T08:30:24Z

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2024-05-17T08:30:26Z

Some changes occurred in run-make tests.

cc @jieyouxu

ChrisDenton · 2024-05-17T08:51:06Z

While I'd very much welcome a review from jieyouxu, this will need lang approval. I think it could just be considered a bug fix but it might also be considered a change in language.

cc @joshtriplett maybe?

Noratrieb · 2024-05-17T11:26:13Z

Given that I see most projects already using / as the separator, calling it a bug fix seems fair. Still, I agree that lang should see it.

jieyouxu · 2024-05-17T14:08:54Z

Changes look reasonable from compiler side. Rolling a lang reviewer to decide if this behavior change is acceptable.

r? lang

bors · 2024-07-29T05:26:54Z

☔ The latest upstream changes (presumably #125443) made this pull request unmergeable. Please resolve the merge conflicts.

When using `concat!` to join paths, the Unix path separator (`/`) is often used. This breaks on Windows if the base path is a verbatim path (i.e. starts with `\\?\`).

joshtriplett · 2024-09-25T15:47:21Z

It looks like this is limited to verbatim paths, which seems completely safe. This wouldn't be safe to do for arbitrary Windows driver-interpreted paths, but it seems entirely safe for Windows verbatim paths.

ChrisDenton · 2024-09-25T16:01:01Z

Thanks! So I'm going to go out on a limb and say jieyouxu's positive review still stands.

@bors r=jieyouxu

bors · 2024-09-25T16:01:05Z

📌 Commit edc97a0 has been approved by jieyouxu

It is now in the queue for this repository.

traviscross · 2024-09-25T16:25:47Z

@rustbot labels -I-lang-easy-decision

This one turned out to not be an easy call for us. One question that came up is:

@ChrisDenton: Is there any way that someone could have used the behavior that exists today correctly with forward slashes in the string? I.e., is there any plausibly valid use case that we might be breaking?

(h/t @tmandry)

ChrisDenton · 2024-09-25T16:29:26Z

Is there any way that someone could have used the behavior that exists today correctly with forward slashes in the string? I.e., is there any plausibly valid use case that we might be breaking?

I cannot think of one. I can think of an implausible scenario. Like someone could create a named pipe, containing / in the name (which is technically allowed but very hard to do normally) then they intentionally use a verbatim prefix so they can access it.

ChrisDenton · 2024-09-25T16:52:18Z

But I would note that if they're doing something that's not only platform specific but requires a fair amount of hacks (and would break when using normal \\.\PIPE paths).

The scenario for supporting this is a lot more plausible: someone sets an environment variable in a build script using Path::canonicalize (which makes verbatim paths on Windows). Then they do the common concat thing expecting it to work cross-platform.

traviscross · 2024-09-25T20:38:47Z

@rfcbot reviewed

Based on the responses here, what's proposed seems like at least the best worst option.

nikomatsakis · 2024-10-02T15:31:37Z

@rfcbot reviewed

tmandry · 2024-10-02T15:31:50Z

@rfcbot reviewed

scottmcm · 2024-10-02T15:31:54Z

I mean you can't use concat in path, right?

This isn't concat-specific though, because it'll change if you put a string literal in the include too.

I just think that we should do it for all compiler paths if we're doing it here.

rfcbot · 2024-10-02T15:32:07Z

🔔 This is now entering its final comment period, as per the review above. 🔔

traviscross · 2024-10-02T16:22:56Z

@rustbot labels -I-lang-nominated

We discussed this in the meeting today... and it's now in FCP.

rfcbot · 2024-10-12T15:39:25Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

ChrisDenton · 2024-10-22T10:39:52Z

I mean you can't use concat in path, right?

This isn't concat-specific though, because it'll change if you put a string literal in the include too.

I just think that we should do it for all compiler paths if we're doing it here.

Oh, it turns out we do something kind of similar for path already:

rust/compiler/rustc_expand/src/module.rs

Lines 195 to 200 in f225713

    
           // On windows, the base path might have the form 
        
           // `\\?\foo\bar` in which case it does not tolerate 
        
           // mixed `/` and `\` separators, so canonicalize 
        
           // `/` to `\`. 
        
           #[cfg(windows)] 
        
           let path_str = path_str.replace("/", "\\");

ChrisDenton · 2024-10-22T10:56:20Z

This has passed lang FCP and was earlier approved by @jieyouxu so I'll merge this now. I'll look more at #[path] but it already looks like it does the equivalent by virtue of using Path::join (which handles verbatim paths).

rust/compiler/rustc_expand/src/module.rs

Line 202 in f225713

Some(dir_path.join(path_str))

@bors r=jieyouxu

bors · 2024-10-22T10:56:23Z

📌 Commit edc97a0 has been approved by jieyouxu

It is now in the queue for this repository.

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#125205 (Fixup Windows verbatim paths when used with the `include!` macro) - rust-lang#131049 (Validate args are correct for `UnevaluatedConst`, `ExistentialTraitRef`/`ExistentialProjection`) - rust-lang#131549 (Add a note for `?` on a `impl Future<Output = Result<..>>` in sync function) - rust-lang#131731 (add `TestFloatParse` to `tools.rs` for bootstrap) - rust-lang#131732 (Add doc(plugins), doc(passes), etc. to INVALID_DOC_ATTRIBUTES) - rust-lang#132006 (don't stage-off to previous compiler when CI rustc is available) - rust-lang#132022 (Move `cmp_in_dominator_order` out of graph dominator computation) - rust-lang#132033 (compiletest: Make `line_directive` return a `DirectiveLine`) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#125205 - ChrisDenton:verbatim-include, r=jieyouxu Fixup Windows verbatim paths when used with the `include!` macro On Windows, the following code can fail if the `OUT_DIR` environment variable is a [verbatim path](https://doc.rust-lang.org/std/path/enum.Prefix.html) (i.e. begins with `\\?\`): ```rust include!(concat!(env!("OUT_DIR"), "/src/repro.rs")); ``` This is because verbatim paths treat `/` literally, as if it were just another character in the file name. The good news is that the standard library already has code to fix this. We can simply use `components` to normalize the path so it works as intended.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [rust](https://github.com/rust-lang/rust) | minor | `1.83.0` -> `1.84.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>rust-lang/rust (rust)</summary> ### [`v1.84.0`](https://github.com/rust-lang/rust/blob/HEAD/RELEASES.md#Version-1840-2025-01-09) [Compare Source](rust-lang/rust@1.83.0...1.84.0) \========================== <a id=" Language"></a> ## Language - [Allow `#[deny]` inside `#[forbid]` as a no-op](rust-lang/rust#121560) - [Show a warning when `-Ctarget-feature` is used to toggle features that can lead to unsoundness due to ABI mismatches](rust-lang/rust#129884) - [Use the next-generation trait solver in coherence](rust-lang/rust#130654) - [Allow coercions to drop the principal of trait objects](rust-lang/rust#131857) - [Support `/` as the path separator for `include!()` in all cases on Windows](rust-lang/rust#125205) - [Taking a raw ref (`raw (const|mut)`) of a deref of a pointer (`*ptr`) is now safe](rust-lang/rust#129248) - [Stabilize s390x inline assembly](rust-lang/rust#131258) - [Stabilize Arm64EC inline assembly](rust-lang/rust#131781) - [Lint against creating pointers to immediately dropped temporaries](rust-lang/rust#128985) - [Execute drop glue when unwinding in an `extern "C"` function](rust-lang/rust#129582) <a id="1.84.0-Compiler"></a> ## Compiler - [Add `--print host-tuple` flag to print the host target tuple and affirm the "target tuple" terminology over "target triple"](rust-lang/rust#125579) - [Declaring functions with a calling convention not supported on the current target now triggers a hard error](rust-lang/rust#129935) - [Set up indirect access to external data for `loongarch64-unknown-linux-{musl,ohos}`](rust-lang/rust#131583) - [Enable XRay instrumentation for LoongArch Linux targets](rust-lang/rust#131818) - [Extend the `unexpected_cfgs` lint to also warn in external macros](rust-lang/rust#132577) - [Stabilize WebAssembly `multivalue`, `reference-types`, and `tail-call` target features](rust-lang/rust#131080) - [Added Tier 2 support for the `wasm32v1-none` target](rust-lang/rust#131487) <a id="1.84.0-Libraries"></a> ## Libraries - [Implement `From<&mut {slice}>` for `Box/Rc/Arc<{slice}>`](rust-lang/rust#129329) - [Move `<float>::copysign`, `<float>::abs`, `<float>::signum` to `core`](rust-lang/rust#131304) - [Add `LowerExp` and `UpperExp` implementations to `NonZero`](rust-lang/rust#131377) - [Implement `FromStr` for `CString` and `TryFrom<CString>` for `String`](rust-lang/rust#130608) - [`std::os::darwin` has been made public](rust-lang/rust#123723) <a id="1.84.0-Stabilized-APIs"></a> ## Stabilized APIs - [`Ipv6Addr::is_unique_local`](https://doc.rust-lang.org/stable/core/net/struct.Ipv6Addr.html#method.is_unique_local) - [`Ipv6Addr::is_unicast_link_local`](https://doc.rust-lang.org/stable/core/net/struct.Ipv6Addr.html#method.is_unicast_link_local) - [`core::ptr::with_exposed_provenance`](https://doc.rust-lang.org/stable/core/ptr/fn.with_exposed_provenance.html) - [`core::ptr::with_exposed_provenance_mut`](https://doc.rust-lang.org/stable/core/ptr/fn.with_exposed_provenance_mut.html) - [`<ptr>::addr`](https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.addr) - [`<ptr>::expose_provenance`](https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.expose_provenance) - [`<ptr>::with_addr`](https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.with_addr) - [`<ptr>::map_addr`](https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.map_addr) - [`<int>::isqrt`](https://doc.rust-lang.org/stable/core/primitive.i32.html#method.isqrt) - [`<int>::checked_isqrt`](https://doc.rust-lang.org/stable/core/primitive.i32.html#method.checked_isqrt) - [`<uint>::isqrt`](https://doc.rust-lang.org/stable/core/primitive.u32.html#method.isqrt) - [`NonZero::isqrt`](https://doc.rust-lang.org/stable/core/num/struct.NonZero.html#impl-NonZero%3Cu128%3E/method.isqrt) - [`core::ptr::without_provenance`](https://doc.rust-lang.org/stable/core/ptr/fn.without_provenance.html) - [`core::ptr::without_provenance_mut`](https://doc.rust-lang.org/stable/core/ptr/fn.without_provenance_mut.html) - [`core::ptr::dangling`](https://doc.rust-lang.org/stable/core/ptr/fn.dangling.html) - [`core::ptr::dangling_mut`](https://doc.rust-lang.org/stable/core/ptr/fn.dangling_mut.html) - [`Pin::as_deref_mut`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.as_deref_mut) These APIs are now stable in const contexts - [`AtomicBool::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicBool.html#method.from_ptr) - [`AtomicPtr::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicPtr.html#method.from_ptr) - [`AtomicU8::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU8.html#method.from_ptr) - [`AtomicU16::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU16.html#method.from_ptr) - [`AtomicU32::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU32.html#method.from_ptr) - [`AtomicU64::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU64.html#method.from_ptr) - [`AtomicUsize::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicUsize.html#method.from_ptr) - [`AtomicI8::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI8.html#method.from_ptr) - [`AtomicI16::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI16.html#method.from_ptr) - [`AtomicI32::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI32.html#method.from_ptr) - [`AtomicI64::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI64.html#method.from_ptr) - [`AtomicIsize::from_ptr`](https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicIsize.html#method.from_ptr) - [`<ptr>::is_null`](https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.is_null-1) - [`<ptr>::as_ref`](https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.as_ref-1) - [`<ptr>::as_mut`](https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.as_mut) - [`Pin::new`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.new) - [`Pin::new_unchecked`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.new_unchecked) - [`Pin::get_ref`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.get_ref) - [`Pin::into_ref`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.into_ref) - [`Pin::get_mut`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.get_mut) - [`Pin::get_unchecked_mut`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.get_unchecked_mut) - [`Pin::static_ref`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.static_ref) - [`Pin::static_mut`](https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.static_mut) <a id="1.84.0-Cargo"></a> ## Cargo - [Stabilize MSRV-aware resolver config](rust-lang/cargo#14639) - [Stabilize resolver v3](rust-lang/cargo#14754) <a id="1.84-Rustdoc"></a> ## Rustdoc - [rustdoc-search: improve type-driven search](rust-lang/rust#127589) <a id="1.84.0-Compatibility-Notes"></a> ## Compatibility Notes - [Enable by default the `LSX` target feature for LoongArch Linux targets](rust-lang/rust#132140) - [The unstable `-Zprofile` flag (“gcov-style” coverage instrumentation) has been removed.](rust-lang/rust#131829) This does not affect the stable flags for coverage instrumentation (`-Cinstrument-coverage`) and profile-guided optimization (`-Cprofile-generate`, `-Cprofile-use`), which are unrelated and remain available. - Support for the target named `wasm32-wasi` has been removed as the target is now named `wasm32-wasip1`. This completes the [transition](rust-lang/compiler-team#607) [plan](rust-lang/compiler-team#695) for this target following [the introduction of `wasm32-wasip1`](rust-lang/rust#120468) in Rust 1.78. Compiler warnings on [use of `wasm32-wasi`](rust-lang/rust#126662) introduced in Rust 1.81 are now gone as well as the target is removed. - [The syntax `&pin (mut|const) T` is now parsed as a type which in theory could affect macro expansion results in some edge cases](rust-lang/rust#130635 (comment)) - [Legacy syntax for calling `std::arch` functions is no longer permitted to declare items or bodies (such as closures, inline consts, or async blocks).](rust-lang/rust#130443 (comment)) - [Declaring functions with a calling convention not supported on the current target now triggers a hard error](rust-lang/rust#129935) - [The next-generation trait solver is now enabled for coherence, fixing multiple soundness issues](rust-lang/rust#130654) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

rustbot assigned jieyouxu May 17, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 17, 2024

ChrisDenton added the T-lang Relevant to the language team, which will review and decide on the PR/issue. label May 17, 2024

This comment has been minimized.

Sign in to view

ChrisDenton force-pushed the verbatim-include branch from f9bf99b to 2b7a45c Compare May 17, 2024 08:44

rustbot assigned joshtriplett and unassigned jieyouxu May 17, 2024

ChrisDenton added S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). and removed T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 22, 2024

workingjubilee added the I-lang-nominated Nominated for discussion during a lang team meeting. label Jul 15, 2024

ChrisDenton force-pushed the verbatim-include branch from 2b7a45c to 9d52111 Compare July 31, 2024 12:22

rustbot added the A-run-make Area: port run-make Makefiles to rmake.rs label Jul 31, 2024

This comment has been minimized.

Sign in to view

Fix verbatim paths used with include!

edc97a0

When using `concat!` to join paths, the Unix path separator (`/`) is often used. This breaks on Windows if the base path is a verbatim path (i.e. starts with `\\?\`).

ChrisDenton force-pushed the verbatim-include branch from 9d52111 to edc97a0 Compare July 31, 2024 12:45

traviscross added the I-lang-easy-decision Issue: The decision needed by the team is conjectured to be easy; this does not imply nomination label Sep 25, 2024

joshtriplett removed the I-lang-easy-decision Issue: The decision needed by the team is conjectured to be easy; this does not imply nomination label Sep 25, 2024

rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels Oct 2, 2024

rustbot removed the I-lang-nominated Nominated for discussion during a lang team meeting. label Oct 2, 2024

rustbot mentioned this pull request Oct 12, 2024

Tracking issue for release notes of #125205: Fixup Windows verbatim paths when used with the include! macro #131609

Closed

3 tasks

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 22, 2024

matthiaskrgr mentioned this pull request Oct 22, 2024

Rollup of 8 pull requests #132035

Merged

bors merged commit 4d378f2 into rust-lang:master Oct 22, 2024
6 checks passed

rustbot added this to the 1.84.0 milestone Oct 22, 2024

ChrisDenton deleted the verbatim-include branch November 7, 2024 15:49

ChrisDenton mentioned this pull request Nov 7, 2024

Windows \\?\ verbatim paths break idiomatic use of OUT_DIR and include! rust-lang/cargo#13919

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixup Windows verbatim paths when used with the `include!` macro #125205

Fixup Windows verbatim paths when used with the `include!` macro #125205

ChrisDenton commented May 17, 2024

rustbot commented May 17, 2024

rustbot commented May 17, 2024

This comment has been minimized.

ChrisDenton commented May 17, 2024

Noratrieb commented May 17, 2024

jieyouxu commented May 17, 2024

bors commented Jul 29, 2024

This comment has been minimized.

joshtriplett commented Sep 25, 2024

ChrisDenton commented Sep 25, 2024 •

edited

Loading

bors commented Sep 25, 2024

traviscross commented Sep 25, 2024

ChrisDenton commented Sep 25, 2024

ChrisDenton commented Sep 25, 2024

traviscross commented Sep 25, 2024

nikomatsakis commented Oct 2, 2024

tmandry commented Oct 2, 2024

scottmcm commented Oct 2, 2024

rfcbot commented Oct 2, 2024

traviscross commented Oct 2, 2024

rfcbot commented Oct 12, 2024

ChrisDenton commented Oct 22, 2024

ChrisDenton commented Oct 22, 2024

bors commented Oct 22, 2024

Fixup Windows verbatim paths when used with the include! macro #125205

Fixup Windows verbatim paths when used with the include! macro #125205

Conversation

ChrisDenton commented May 17, 2024

rustbot commented May 17, 2024

rustbot commented May 17, 2024

This comment has been minimized.

ChrisDenton commented May 17, 2024

Noratrieb commented May 17, 2024

jieyouxu commented May 17, 2024

bors commented Jul 29, 2024

This comment has been minimized.

joshtriplett commented Sep 25, 2024

ChrisDenton commented Sep 25, 2024 • edited Loading

bors commented Sep 25, 2024

traviscross commented Sep 25, 2024

ChrisDenton commented Sep 25, 2024

ChrisDenton commented Sep 25, 2024

traviscross commented Sep 25, 2024

nikomatsakis commented Oct 2, 2024

tmandry commented Oct 2, 2024

scottmcm commented Oct 2, 2024

rfcbot commented Oct 2, 2024

traviscross commented Oct 2, 2024

rfcbot commented Oct 12, 2024

ChrisDenton commented Oct 22, 2024

ChrisDenton commented Oct 22, 2024

bors commented Oct 22, 2024

Fixup Windows verbatim paths when used with the `include!` macro #125205

Fixup Windows verbatim paths when used with the `include!` macro #125205

ChrisDenton commented Sep 25, 2024 •

edited

Loading