Windows: OsStr need not be valid UTF-8

[DemiMarie]

On Windows, filenames (and many other strings) are stored in UCS-2, not UTF-8. This corresponds to WTF-8, not UTF-8. Furthermore, the NT kernel uses length-specified strings via the UNICODE_STRING struct, and uses only \ as the path separator. Therefore, many APIs (such as NtCreateFile and NtCreateKey) actually allow NUL characters in names. While the NTFS driver does not allow NUL to be part of a filename, registry keys can have NUL characters in there names.

Therefore, I propose that OsStr on Windows be able to be created from any valid WTF-8, and that an API be provided that allows them to be created with NUL characters.

Also see #53261 (comment)

[ollie27]

Are OsStrExt and OsStringExt in std::os::windows::ffi not sufficient?

[retep998]

OsStr literally already is this. It stores WTF-8 instead of UTF-8 and is capable of holding NUL characters just fine.

[estebank]

Leaving open as under the purview of @rust-lang/docs, like #53261.

[Mark-Simulacrum]

Closing since this seems to be satisfied by the existing APIs (per above comments and reading the API docs today). #53261 touches on the more specific issue of std::ffi docs being misleading about the underlying representation of OsString on Windows, but that's an orthogonal and separate concern.