-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Put a bound on collection misbehavior #97316
Conversation
As currently written, when a logic error occurs in a collection's trait parameters, this allows *completely arbitrary* misbehavior, so long as it does not cause undefined behavior in std. However, because the extent of misbehavior is not specified, it is allowed for *any* code in std to start misbehaving in arbitrary ways which are not formally UB; consider the theoretical example of a global which gets set on an observed logic error. Because the misbehavior is only bound by not resulting in UB from safe APIs and the crate-level encapsulation boundary of all of std, this makes writing user unsafe code that utilizes std theoretically impossible, as it now relies on undocumented QOI that unrelated parts of std cannot be caused to misbehave by a misuse of std::collections APIs. In practice, this is a nonconcern, because std has reasonable QOI and an implementation that takes advantage of this freedom is essentially a malicious implementation and only compliant by the most langauage-lawyer reading of the documentation. To close this hole, we just add a small clause to the existing logic error paragraph that ensures that any misbehavior is limited to the collection which observed the logic error, making it more plausible to prove the soundness of user unsafe code. This is not meant to be formal; a formal refinement would likely need to mention that values derived from the collection can also misbehave after a logic error is observed, as well as define what it means to "observe" a logic error in the first place. This fix errs on the side of informality in order to close the hole without complicating a normal reading which can assume a reasonable nonmalicious QOI. See also [discussion on IRLO][1]. [1]: https://internals.rust-lang.org/t/using-std-collections-and-unsafe-anything-can-happen/16640
Hey! It looks like you've submitted a new PR for the library teams! If this PR contains changes to any Examples of
|
I said.. @rustbot label -T-libs |
Another detail is that if the |
I actually don't think that needs to be mentioned even in a formal refinement of misbehavior; obviously the code implementing the trait is being run, so whatever side effects it has happens. |
Co-authored-by: lcnr <rust@lcnr.de>
@rust-lang/libs-api: I am happy with the wording added here. The IRLO thread linked from the PR description has a concrete example of code that must assume the guarantee being added here. The first post is sufficient to read; the rest of the thread is… long and not especially enlightening. |
Team member @dtolnay has proposed to merge this. The next step is review by the rest of the tagged team members: No concerns currently listed. Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for info about what commands tagged team members can give me. |
🔔 This is now entering its final comment period, as per the review above. 🔔 |
@bors r+ |
📌 Commit e6b1003 has been approved by |
@bors r+ rollup |
💡 This pull request was already approved, no need to approve it again.
|
📌 Commit e6b1003 has been approved by |
Put a bound on collection misbehavior As currently written, when a logic error occurs in a collection's trait parameters, this allows *completely arbitrary* misbehavior, so long as it does not cause undefined behavior in std. However, because the extent of misbehavior is not specified, it is allowed for *any* code in std to start misbehaving in arbitrary ways which are not formally UB; consider the theoretical example of a global which gets set on an observed logic error. Because the misbehavior is only bound by not resulting in UB from safe APIs and the crate-level encapsulation boundary of all of std, this makes writing user unsafe code that utilizes std theoretically impossible, as it now relies on undocumented QOI (quality of implementation) that unrelated parts of std cannot be caused to misbehave by a misuse of std::collections APIs. In practice, this is a nonconcern, because std has reasonable QOI and an implementation that takes advantage of this freedom is essentially a malicious implementation and only compliant by the most langauage-lawyer reading of the documentation. To close this hole, we just add a small clause to the existing logic error paragraph that ensures that any misbehavior is limited to the collection which observed the logic error, making it more plausible to prove the soundness of user unsafe code. This is not meant to be formal; a formal refinement would likely need to mention that values derived from the collection can also misbehave after a logic error is observed, as well as define what it means to "observe" a logic error in the first place. This fix errs on the side of informality in order to close the hole without complicating a normal reading which can assume a reasonable nonmalicious QOI. See also [discussion on IRLO][1]. [1]: https://internals.rust-lang.org/t/using-std-collections-and-unsafe-anything-can-happen/16640 r? rust-lang/libs-api `@rustbot` label +T-libs-api -T-libs This technically adds a new guarantee to the documentation, though I argue as written it's one already implicitly provided.
Put a bound on collection misbehavior As currently written, when a logic error occurs in a collection's trait parameters, this allows *completely arbitrary* misbehavior, so long as it does not cause undefined behavior in std. However, because the extent of misbehavior is not specified, it is allowed for *any* code in std to start misbehaving in arbitrary ways which are not formally UB; consider the theoretical example of a global which gets set on an observed logic error. Because the misbehavior is only bound by not resulting in UB from safe APIs and the crate-level encapsulation boundary of all of std, this makes writing user unsafe code that utilizes std theoretically impossible, as it now relies on undocumented QOI (quality of implementation) that unrelated parts of std cannot be caused to misbehave by a misuse of std::collections APIs. In practice, this is a nonconcern, because std has reasonable QOI and an implementation that takes advantage of this freedom is essentially a malicious implementation and only compliant by the most langauage-lawyer reading of the documentation. To close this hole, we just add a small clause to the existing logic error paragraph that ensures that any misbehavior is limited to the collection which observed the logic error, making it more plausible to prove the soundness of user unsafe code. This is not meant to be formal; a formal refinement would likely need to mention that values derived from the collection can also misbehave after a logic error is observed, as well as define what it means to "observe" a logic error in the first place. This fix errs on the side of informality in order to close the hole without complicating a normal reading which can assume a reasonable nonmalicious QOI. See also [discussion on IRLO][1]. [1]: https://internals.rust-lang.org/t/using-std-collections-and-unsafe-anything-can-happen/16640 r? rust-lang/libs-api ``@rustbot`` label +T-libs-api -T-libs This technically adds a new guarantee to the documentation, though I argue as written it's one already implicitly provided.
…askrgr Rollup of 4 pull requests Successful merges: - rust-lang#97316 (Put a bound on collection misbehavior) - rust-lang#97578 (alloc: remove repeated word in comment) - rust-lang#97593 (:arrow_up: rust-analyzer) - rust-lang#97596 (Fixup feature name to be more consistent with others) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup
Pkgsrc changes: * Adjust patches as needed & checksum updates. Upstream changes: Version 1.63.0 (2022-08-11) ========================== Language -------- - [Remove migrate borrowck mode for pre-NLL errors.][95565] - [Modify MIR building to drop repeat expressions with length zero.][95953] - [Remove label/lifetime shadowing warnings.][96296] - [Allow explicit generic arguments in the presence of `impl Trait` args.] [96868] - [Make `cenum_impl_drop_cast` warnings deny-by-default.][97652] - [Prevent unwinding when `-C panic=abort` is used regardless of declared ABI.][96959] - [lub: don't bail out due to empty binders.][97867] Compiler -------- - [Stabilize the `bundle` native library modifier,][95818] also removing the deprecated `static-nobundle` linking kind. - [Add Apple WatchOS compile targets\*.][95243] - [Add a Windows application manifest to rustc-main.][96737] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Implement `Copy`, `Clone`, `PartialEq` and `Eq` for `core::fmt::Alignment`.][94530] - [Extend `ptr::null` and `null_mut` to all thin (including extern) types.][94954] - [`impl Read and Write for VecDeque<u8>`.][95632] - [STD support for the Nintendo 3DS.][95897] - [Make write/print macros eagerly drop temporaries.][96455] - [Implement internal traits that enable `[OsStr]::join`.][96881] - [Implement `Hash` for `core::alloc::Layout`.][97034] - [Add capacity documentation for `OsString`.][97202] - [Put a bound on collection misbehavior.][97316] - [Make `std::mem::needs_drop` accept `?Sized`.][97675] - [`impl Termination for Infallible` and then make the `Result` impls of `Termination` more generic.][97803] - [Document Rust's stance on `/proc/self/mem`.][97837] Stabilized APIs --------------- - [`array::from_fn`] - [`Box::into_pin`] - [`BinaryHeap::try_reserve`] - [`BinaryHeap::try_reserve_exact`] - [`OsString::try_reserve`] - [`OsString::try_reserve_exact`] - [`PathBuf::try_reserve`] - [`PathBuf::try_reserve_exact`] - [`Path::try_exists`] - [`Ref::filter_map`] - [`RefMut::filter_map`] - [`NonNull::<[T]>::len`][`NonNull::<slice>::len`] - [`ToOwned::clone_into`] - [`Ipv6Addr::to_ipv4_mapped`] - [`unix::io::AsFd`] - [`unix::io::BorrowedFd<'fd>`] - [`unix::io::OwnedFd`] - [`windows::io::AsHandle`] - [`windows::io::BorrowedHandle<'handle>`] - [`windows::io::OwnedHandle`] - [`windows::io::HandleOrInvalid`] - [`windows::io::HandleOrNull`] - [`windows::io::InvalidHandleError`] - [`windows::io::NullHandleError`] - [`windows::io::AsSocket`] - [`windows::io::BorrowedSocket<'handle>`] - [`windows::io::OwnedSocket`] - [`thread::scope`] - [`thread::Scope`] - [`thread::ScopedJoinHandle`] These APIs are now usable in const contexts: - [`array::from_ref`] - [`slice::from_ref`] - [`intrinsics::copy`] - [`intrinsics::copy_nonoverlapping`] - [`<*const T>::copy_to`] - [`<*const T>::copy_to_nonoverlapping`] - [`<*mut T>::copy_to`] - [`<*mut T>::copy_to_nonoverlapping`] - [`<*mut T>::copy_from`] - [`<*mut T>::copy_from_nonoverlapping`] - [`str::from_utf8`] - [`Utf8Error::error_len`] - [`Utf8Error::valid_up_to`] - [`Condvar::new`] - [`Mutex::new`] - [`RwLock::new`] Cargo ----- - [Stabilize the `--config path` command-line argument.][cargo/10755] - [Expose rust-version in the environment as `CARGO_PKG_RUST_VERSION`.][cargo/10713] Compatibility Notes ------------------- - [`#[link]` attributes are now checked more strictly,][96885] which may introduce errors for invalid attribute arguments that were previously ignored. Internal Changes ---------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Prepare Rust for LLVM opaque pointers.][94214] [94214]: rust-lang/rust#94214 [94530]: rust-lang/rust#94530 [94954]: rust-lang/rust#94954 [95243]: rust-lang/rust#95243 [95565]: rust-lang/rust#95565 [95632]: rust-lang/rust#95632 [95818]: rust-lang/rust#95818 [95897]: rust-lang/rust#95897 [95953]: rust-lang/rust#95953 [96296]: rust-lang/rust#96296 [96455]: rust-lang/rust#96455 [96737]: rust-lang/rust#96737 [96868]: rust-lang/rust#96868 [96881]: rust-lang/rust#96881 [96885]: rust-lang/rust#96885 [96959]: rust-lang/rust#96959 [97034]: rust-lang/rust#97034 [97202]: rust-lang/rust#97202 [97316]: rust-lang/rust#97316 [97652]: rust-lang/rust#97652 [97675]: rust-lang/rust#97675 [97803]: rust-lang/rust#97803 [97837]: rust-lang/rust#97837 [97867]: rust-lang/rust#97867 [cargo/10713]: rust-lang/cargo#10713 [cargo/10755]: rust-lang/cargo#10755 [`array::from_fn`]: https://doc.rust-lang.org/stable/std/array/fn.from_fn.html [`Box::into_pin`]: https://doc.rust-lang.org/stable/std/boxed/struct.Box.html#method.into_pin [`BinaryHeap::try_reserve_exact`]: https://doc.rust-lang.org/stable/alloc/collections/binary_heap/struct.BinaryHeap.html#method.try_reserve_exact [`BinaryHeap::try_reserve`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.try_reserve [`OsString::try_reserve`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.try_reserve [`OsString::try_reserve_exact`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.try_reserve_exact [`PathBuf::try_reserve`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.try_reserve [`PathBuf::try_reserve_exact`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.try_reserve_exact [`Path::try_exists`]: https://doc.rust-lang.org/stable/std/path/struct.Path.html#method.try_exists [`Ref::filter_map`]: https://doc.rust-lang.org/stable/std/cell/struct.Ref.html#method.filter_map [`RefMut::filter_map`]: https://doc.rust-lang.org/stable/std/cell/struct.RefMut.html#method.filter_map [`NonNull::<slice>::len`]: https://doc.rust-lang.org/stable/std/ptr/struct.NonNull.html#method.len [`ToOwned::clone_into`]: https://doc.rust-lang.org/stable/std/borrow/trait.ToOwned.html#method.clone_into [`Ipv6Addr::to_ipv4_mapped`]: https://doc.rust-lang.org/stable/std/net/struct.Ipv6Addr.html#method.to_ipv4_mapped [`unix::io::AsFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/trait.AsFd.html [`unix::io::BorrowedFd<'fd>`]: https://doc.rust-lang.org/stable/std/os/unix/io/struct.BorrowedFd.html [`unix::io::OwnedFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/struct.OwnedFd.html [`windows::io::AsHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.AsHandle.html [`windows::io::BorrowedHandle<'handle>`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.BorrowedHandle.html [`windows::io::OwnedHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.OwnedHandle.html [`windows::io::HandleOrInvalid`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.HandleOrInvalid.html [`windows::io::HandleOrNull`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.HandleOrNull.html [`windows::io::InvalidHandleError`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.InvalidHandleError.html [`windows::io::NullHandleError`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.NullHandleError.html [`windows::io::AsSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.AsSocket.html [`windows::io::BorrowedSocket<'handle>`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.BorrowedSocket.html [`windows::io::OwnedSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.OwnedSocket.html [`thread::scope`]: https://doc.rust-lang.org/stable/std/thread/fn.scope.html [`thread::Scope`]: https://doc.rust-lang.org/stable/std/thread/struct.Scope.html [`thread::ScopedJoinHandle`]: https://doc.rust-lang.org/stable/std/thread/struct.ScopedJoinHandle.html [`array::from_ref`]: https://doc.rust-lang.org/stable/std/array/fn.from_ref.html [`slice::from_ref`]: https://doc.rust-lang.org/stable/std/slice/fn.from_ref.html [`intrinsics::copy`]: https://doc.rust-lang.org/stable/std/intrinsics/fn.copy.html [`intrinsics::copy_nonoverlapping`]: https://doc.rust-lang.org/stable/std/intrinsics/fn.copy_nonoverlapping.html [`<*const T>::copy_to`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to [`<*const T>::copy_to_nonoverlapping`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to_nonoverlapping [`<*mut T>::copy_to`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to-1 [`<*mut T>::copy_to_nonoverlapping`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to_nonoverlapping-1 [`<*mut T>::copy_from`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_from [`<*mut T>::copy_from_nonoverlapping`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_from_nonoverlapping [`str::from_utf8`]: https://doc.rust-lang.org/stable/std/str/fn.from_utf8.html [`Utf8Error::error_len`]: https://doc.rust-lang.org/stable/std/str/struct.Utf8Error.html#method.error_len [`Utf8Error::valid_up_to`]: https://doc.rust-lang.org/stable/std/str/struct.Utf8Error.html#method.valid_up_to [`Condvar::new`]: https://doc.rust-lang.org/stable/std/sync/struct.Condvar.html#method.new [`Mutex::new`]: https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html#method.new [`RwLock::new`]: https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html#method.new
As currently written, when a logic error occurs in a collection's trait parameters, this allows completely arbitrary misbehavior, so long as it does not cause undefined behavior in std. However, because the extent of misbehavior is not specified, it is allowed for any code in std to start misbehaving in arbitrary ways which are not formally UB; consider the theoretical example of a global which gets set on an observed logic error. Because the misbehavior is only bound by not resulting in UB from safe APIs and the crate-level encapsulation boundary of all of std, this makes writing user unsafe code that utilizes std theoretically impossible, as it now relies on undocumented QOI (quality of implementation) that unrelated parts of std cannot be caused to misbehave by a misuse of std::collections APIs.
In practice, this is a nonconcern, because std has reasonable QOI and an implementation that takes advantage of this freedom is essentially a malicious implementation and only compliant by the most langauage-lawyer reading of the documentation.
To close this hole, we just add a small clause to the existing logic error paragraph that ensures that any misbehavior is limited to the collection which observed the logic error, making it more plausible to prove the soundness of user unsafe code.
This is not meant to be formal; a formal refinement would likely need to mention that values derived from the collection can also misbehave after a logic error is observed, as well as define what it means to "observe" a logic error in the first place. This fix errs on the side of informality in order to close the hole without complicating a normal reading which can assume a reasonable nonmalicious QOI.
See also discussion on IRLO.
r? rust-lang/libs-api @rustbot label +T-libs-api -T-libs
This technically adds a new guarantee to the documentation, though I argue as written it's one already implicitly provided.