rustc (and cargo) handle non-utf-8 paths very poorly

[djzin]

Considering rust goes through all these pains to have separate types such as OsString and OsStr and Path and CString vs just plain old String, precisely to distinguish between those strings that contain utf-8 and those that don't, it is very unfortunate that neither rustc, nor cargo, nor the build system, nor anything really can handle non-utf-8 paths. To replicate:

$ mkdir $'\377'
$ cd $'\377'
$ cargo new project
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Utf8Error { valid_up_to: 48 }', ../src/libcore/result.rs:837
note: Run with `RUST_BACKTRACE=1` for a backtrace.
$ cat << EOF > main.rs
> fn main() {
>     println!("hello, world!");
> }
> EOF
$ rustc main.rs
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libcore/option.rs:323
note: Run with `RUST_BACKTRACE=1` for a backtrace.

[brson]

The thing to do here is probably go over these projects and at least make sure they are converting all user-input paths to UTF-8 in a way that is reported as an error. Actually making everything everything work with non-utf-8 paths is probably quite a task, and maybe not worth it. Furthermore, it may not even be desirable to support non-UTF-8 paths for the can of support worms that opens up.

[cuviper]

There's an old bug on this already which I can't find, but I know looked into it once. rustc_driver::main calls env::args(), and that Args iterator panics if it encounters invalid unicode.

[Mark-Simulacrum]

This is a duplicate of #15890.

[cuviper]

Aha, that's the bug I was trying to remember, and I'd even commented on it! :)