Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add longPathAware to the app manifest on Windows #7986

Closed
MOZGIII opened this issue Mar 11, 2020 · 14 comments
Closed

Add longPathAware to the app manifest on Windows #7986

MOZGIII opened this issue Mar 11, 2020 · 14 comments
Labels
A-filesystem Area: issues with filesystems C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` E-hard Experience: Hard O-windows OS: Windows S-needs-team-input Status: Needs input from team on whether/how to proceed.

Comments

@MOZGIII
Copy link

MOZGIII commented Mar 11, 2020

Describe the problem you are trying to solve

On Windows, when the directory where the project files are located is too long, cargo fails with various file-related errors.

Describe the solution you'd like

Rust should handle long path ok. We just have to add a longPathAware to the app manifest:
https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

Notes

This issue only occurs if the file paths cargo wants to touch are too long.
This happens with paths like these:

C:\Users\User\AppData\Local\Yarn\Cache\v6\.tmp\19aa03367eb9c4791798fcfbbdd8d7d4.e2c16fa02b2d9250fe13c2054c37a11aef5c0da3.prepare\native\target\x86_64-pc-windows-msvc\release\.fingerprint\semver-parser-dc26c18a352c24f5\lib-semver_parser-dc26c18a352c24f5.json
@MOZGIII MOZGIII added the C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` label Mar 11, 2020
@ehuss
Copy link
Contributor

ehuss commented Mar 18, 2020

I think this would be a very hard problem to solve, if not impossible. Some issues I can think of:

  • Rust doesn't have a way to embed the app manifest AFAIK.
  • The default registry key needed for that is OFF, so I don't think it would help most people.
  • All the windows-interfacing APIs (in C and Rust) would need to be audited/updated for the larger constant.
  • All of rustc and cargo would need to handle this.
  • IIRC, MSVC's linker can't handle long filenames, so it probably wouldn't work even if we tried.
  • Many other tools would not be able to handle these paths.
  • We could maybe try to use canonicalized device paths (\\?\) everywhere, but I think that would be very difficult and problematic. We already have problems with device paths on various filesystems. They are also not user-friendly, so would probably need some abstraction that could show different representations.

@MOZGIII
Copy link
Author

MOZGIII commented Mar 18, 2020

Thanks for the response! Maybe there's a workaround I could apply to handle this? Like setting a custom CARGO_TARGET_DIR - would that work? I think it should work...

@ehuss ehuss added the O-windows OS: Windows label Mar 24, 2020
@victoryang00
Copy link

It could be disable by the regedit.msc. so just stopping checking it in the cargo for windows'll be fine.

@RoguePointer80
Copy link

Rust doesn't have a way to embed the app manifest AFAIK.

If your CI has the MSVC toolsets installed, you have all that you need. Don't have them? Here is the direct download link.

The default registry key needed for that is OFF, so I don't think it would help most people

But for the people who DO have it on, it would help. In my opinion, that's a pretty good start! And when developers start seeing that their colleague got it working, they will ask "hey dude, what did you do?", and soon their IT department will turn the switch on for all new dev computers.

All the windows-interfacing APIs (in C and Rust) would need to be audited/updated for the larger constant.

If your windows-interfacing APIs are still using a static 260-char buffer, it's past time to fix that code. All Windows APIs have a way to query the required buffer length. The typical way is calling the function with a 0-length buffer, and the return code is the required length. Any skilled Windows developer should know that.

All of rustc and cargo would need to handle this.

Yep. Let's start with this tool (this issue is about Cargo, right?). When it's done, other executables will copy the solution. Lead by example!

IIRC, MSVC's linker can't handle long filenames, so it probably wouldn't work even if we tried.

Have you tried? Or are you assuming other tools might not work as an excuse not to try? How will the situation improve if every maintainer used the argument "tool X is not updated, so I'm not updating tool Y" ? I strongly believe that if everyone does their part, the ecosystem will quickly improve for everybody.

Many other tools would not be able to handle these paths

Maybe they already do, but unless cargo is updated, we'll never know. And what's the worst-case here? That cargo supports it, and we find out that a downstream tool gives an error. Then we can create an issue and a PR for that downstream tool, and keep making progress.

We could maybe try to use canonicalized UNC paths everywhere

Good point, excellent explanation. This is not a very good solution. The manifest solution seems a better approach.

@nitsky
Copy link

nitsky commented Jan 8, 2021

I just ran into this issue because I have a crate with a long name, over 100 characters, so the path to the crate's location in the target folder exceeds the 260 character limit. Because of this, even cargo build fails. Does anyone following this issue know of any workarounds?

@ghost
Copy link

ghost commented Sep 3, 2021

don't get your hopes up,
as far as I can tell, at least for explorer.exe,
on Windows 10 Enterprise Preview build 19044.1200.210812-1726.21H2 - August 2021
you, still, can not handle path with length longer than 260 (actually 258) characters.

that's because No version of windows's Explorer.exe embeds longPathAware in it's manifest file.
longPathAware has to embedded per executable.
If explorer.exe embeds longPathAware in it's manifest, legacy executables that don't embed in the manifest will simply fail to launch from long path. That's the reason Win32 Long Paths registry key is off by default.

Manifest files are an integral part of Modern Windows. A system programming lang like Rust shouldn't do dilly-dally in this space.

@ghost
Copy link

ghost commented Sep 3, 2021

@ehuss as @RoguePointer80 addresses your points, and regarding Registry Key off by default point, you should know the person installs a system programming lang like in the first place will most likely be A Developer, not a average-joe. so this point is moot.

First of all , Rust shouldn't call undocumented apis.
secondly, NT UNC(\\?\) paths are designed to be used by the Subsystems and KMDF Drivers, not user mode programs like Rust.
Third, that leaves the only way , that is to use embedded manifests per executables.
Fourth, all of these you listed above apparently are assumptions that seem to come from not using windows regularly.

@ghost
Copy link

ghost commented Sep 3, 2021

@ghost
Copy link

ghost commented Sep 3, 2021

I'll just export the embedded manifests from explorer exe/dll/mui, patch them

Wah ! Clever. forgot about SxS manifest support. I'm afraid about that "Patch part" i.e the post-patching consequences here, especially about touching core executables

@weihanglo weihanglo added A-filesystem Area: issues with filesystems E-hard Experience: Hard labels Jun 29, 2023
@weihanglo weihanglo added the S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. label Jul 16, 2023
@ChrisDenton
Copy link
Member

rustc has been using an application manifest file for a while now (since 1.63). There have yet to be any reported issues.

A manifest allows setting more than just longPathAware. rustc sets the supported OSes and also sets the code page to UTF-8. It's implemented as a build script which leverages MSVC's link.exe to do the actual embedding (alternatively, .cargo/config.toml might be used). So it's actually easy to do for that target.

I think at a minimum cargo should set the OSes it supports so as to avoid unnecessary compatibility shims. I'd also advocate for setting the other two options even if they're less likely to be needed in Rust itself (due to using the UTF-16 APIs).

@epage epage added S-needs-team-input Status: Needs input from team on whether/how to proceed. and removed S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. labels Oct 24, 2023
@ehuss
Copy link
Contributor

ehuss commented Dec 8, 2023

Closing as #13131 has added a manifest. That is just the beginning of long-path support. See the meta tracking issue at #9770, and #13141 for some more examples of what still doesn't work.

For people hitting the issue in git clones, be sure to check out #13020 which has more information (in short, add core.longpaths=true to your git config).

@ehuss ehuss closed this as completed Dec 8, 2023
@mati865
Copy link
Contributor

mati865 commented Dec 21, 2023

This issue has been solved only for MSVC targets, should the same be done for Windows GNU?

@ChrisDenton
Copy link
Member

Last I looked (which was awhile ago) that was more complicated because gnu didn't have linker arguments for adding a manifest so you either had to use a post-build step to edit the binary or else manually construct an object file with the right tables and link them (which is a much more involved process). Maybe that's changed though.

@mati865
Copy link
Contributor

mati865 commented Dec 21, 2023

Oh, that could be a problem, we would nee to use LLD to circumvent it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-filesystem Area: issues with filesystems C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` E-hard Experience: Hard O-windows OS: Windows S-needs-team-input Status: Needs input from team on whether/how to proceed.
Projects
None yet
Development

No branches or pull requests

9 participants