From 1a8561e77b64768a0fe3719c2b30c9bc87055e3c Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 31 Aug 2020 19:40:31 -0400 Subject: [PATCH 01/18] Safer Transmute Continuation of https://github.com/rust-lang/project-safe-transmute/pull/5 --- text/0000-safer-transmute.md | 1768 ++++++++++++++++++++++++++++++++++ 1 file changed, 1768 insertions(+) create mode 100644 text/0000-safer-transmute.md diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md new file mode 100644 index 00000000000..f6d11b2d8a4 --- /dev/null +++ b/text/0000-safer-transmute.md @@ -0,0 +1,1768 @@ +# Safer Transmute RFC + +- Feature Name: `safer_transmute` +- Start Date: 2020-08-31 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + + +# Summary +[summary]: #summary + +We propose traits, namely `TransmuteInto` and `TransmuteFrom`, that are implemented *automatically* for combinations of types that may be safely transmuted. In other words, this RFC makes safe transmutation *as easy as 1..., 2..., `repr(C)`!* +```rust +use core::transmute::{ + TransmuteInto, + stability::{PromiseTransmutableInto, PromiseTransmutableFrom}, +}; + +#[derive(PromiseTransmutableInto, PromiseTransmutableFrom)] // declare `Foo` to be *stably* transmutable +#[repr(C)] +pub struct Foo(pub u8, pub u16); +// ^ there's a padding byte here, between these fields + +// Transmute fearlessly! +let _ : Foo = 64u32.transmute_into(); // Alchemy Achieved! +// ^^^^^^^^^^^^^^ provided by the `TransmuteInto` trait + +let _ : u32 = Foo(16, 12).transmute_into(); // Compile Error! + +// error[E0277]: the trait bound `u32: TransmuteFrom` is not satisfied +// --> src/demo.rs:15:27 +// | +// 15 | let _ : u32 = Foo(16, 12).transmute_into(); // Compile Error! +// | ^^^^^^^^^^^^^^ the trait `TransmuteFrom` is not implemented for `u32` +// | +// = note: required because of the requirements on the impl of `TransmuteInto` for `foo::Foo` +// = note: byte 8 of the source type may be uninitialized; byte 8 of the destination type cannot be uninitialized. +``` + + +# Motivation +[motivation]: #motivation + +Byte-reinterpretation conversions (such as those performed by `mem::transmute`, `mem::transmute_copy`, pointer casts, and `union`s) are invaluable in high performance contexts, are `unsafe`, and easy to get wrong. This RFC provides mechanisms that make many currently-unsafe transmutations entirely safe. For transmutations that are not entirely safe, this RFC's mechanisms make mistakes harder to make. + +This RFC's comprehensive approach provides additional benefits beyond the mere act of transmutation; namely: + - [authoritatively codifies language layout guarantees](#codifying-language-layout-guarantees) + - [allows crate authors to codify their types' layout stability guarantees](#expressing-library-layout-guarantees) + - [allows crate authors to codify their abstractions' layout requirements](#expressing-layout-requirements) + +Given the expressive foundation provided by this RFC, we also envision a range of future possibilities that will *not* require additional compiler support, including: + - [safe slice and `Vec` casting][0000-ext-container-casting.md] + - [a unified, generic `Atomic` type][0000-ext-generic-atomic.md] + - [a safe, generic alternative to `include_bytes!`][0000-ext-include-data.md] + - [traits for asserting the size and alignment relationships of types][0000-ext-layout-traits.md] + - [zerocopy-style traits for safe initialization][0000-ext-byte-transmutation.md] + - [bytemuck-style mechanisms for fallible reference casting][ext-ref-casting] + + +## Codifying Language Layout Guarantees +Documentation of Rust's layout guarantees for a type are often spread across countless issues, pull requests, RFCs and various official resources. It can be very difficult to get a straight answer. When transmutation is involved, users must reason about the *combined* layout properties of the source and destination types. + +This RFC proposes mechanisms that programmers will use to confidently answer such questions—by checking whether the `TransmuteFrom` and `TransmuteInto` traits are implemented, or (equivalently) by checking whether the `can_transmute` predicate (a `const fn`) is satisfied. + +## Expressing Library Layout Guarantees +There is no canonical way for crate authors to declare the SemVer layout guarantees of their types. Crate authors currently must state their layout guarantees using prose in their documentation. In contrast to structural stability (e.g., the declared visibility of fields), layout stability is expressed extra-linguistically. + +This isn't satisfactory: guarantees expressed in prose outside of the Rust programming language are guarantees that cannot be reasoned about *inside* the language. Whereas `rustc` can dutifully deny programmers access to private fields, it is unable to prevent programmers from making unfounded expectations of types' in-memory layouts. + +This RFC proposes simple-but-powerful [mechanisms][stability] for declaring layout stability guarantees. + +## Expressing Layout Requirements +Similarly, there is no canonical way for crate authors to declare the layout requirements of generic abstractions over types that have certain layout properties. + +For instance, a common bit-packing technique involves using the relationship between allocations and alignment. If a type is aligned to 2n, then the *n* least significant bits of pointers to that type will equal `0`. These known-zero bits can be packed with data. Since alignment cannot be currently reasoned about at the type-level, it's currently impossible to bound instantiations of a generic parameter based on minimum alignment. + +The mechanisms proposed by the RFC enable this, see [here][0000-ext-layout-traits.md]. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +## Terminology & Concepts + +### 📖 Transmutation +**Transmutation** is the act of reinterpreting the bytes corresponding to a value of one type as if they corresponded to a different type. Concretely, we mean the behavior of this function: +```rust +#[inline(always)] +unsafe fn transmute(src: Src) -> Dst +{ + #[repr(C)] + union Transmute { + src: ManuallyDrop, + dst: ManuallyDrop, + } + + ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst) +} +``` + +### 📖 Safer Transmutation +By **safer transmutation** we mean: *what `where` bound could we add to `transmute` restricts its type parameters `Src` and `Dst` in ways that statically limit the function's misuse?* Our answer to this question will ensure that transmutations are, by default, *sound*, *safe* and *stable*. + +### 📖 Soundness +A transmutation is ***sound*** if the mere act of transmuting a value from one type to another is not unspecified or undefined behavior. + +### 📖 Safety +A sound transmutation is ***safe*** if *using* the transmuted value cannot violate memory safety. + +### 📖 Stability +A safe transmutation is ***stable*** if the authors of the source type and destination types have indicated that the layouts of those types is part of their libraries' stability guarantees. + +## Concepts in Depth + +***Disclaimer:** While the high-level definitions of transmutation soundness, safety and stability are a core component of this RFC, the detailed rules and examples in this section are **not**. We expect that the initial implementation of `TransmuteFrom` may initially be considerably less sophisticated than the examples in this section (and thus forbid valid transmutations). Nonetheless, this section explores nuanced cases of transmutation soundness and safety to demonstrate that the APIs we propose can grow to handle that nuance.* + + +### 📖 When is a transmutation sound? +[sound transmutation]: #-when-is-a-transmutation-sound +A transmutation is ***sound*** if the mere act of transmuting a value from one type to another is not unspecified or undefined behavior. + +#### Well-Defined Representation +[`u8`]: core::u8 +[`f32`]: core::f32 + +Transmutation is *always unsound* if it occurs between types with unspecified representations. + +Most of Rust's primitive types have specified representations. That is, the precise layout characteristics of [`u8`], [`f32`] is a documented and guaranteed aspect of those types. + +In contrast, most `struct` and `enum` types defined without an explicit `#[repr(C)]` attribute do ***not*** have well-specified layout characteristics. + +To ensure that types you've define are soundly transmutable, you almost always (with very few exceptions) must mark them with the `#[repr(C)]` attribute. + +#### Requirements on Owned Values +[transmute-owned]: #requirements-on-owned-values + +Transmutations involving owned values must adhere to two rules to be sound. They must: + * [preserve or broaden the bit validity][owned-validity], and + * [preserve or shrink the size][owned-size]. + +##### Preserve or Broaden Bit Validity +[owned-validity]: #Preserve-or-Broaden-Bit-Validity +[`NonZeroU8`]: https://doc.rust-lang.org/beta/core/num/struct.NonZeroU8.html + +The bits of any valid instance of the source type must be a bit-valid instance of the destination type. + +For example, we are permitted to transmute a `Bool` into a [`u8`]: +```rust +#[derive(Default, PromiseTransmutableFrom, PromiseTransmutableInto)] +#[repr(u8)] +enum Bool { + True = 1, + False = 0, +} + +let _ : u8 = Bool::True.transmute_into(); +let _ : u8 = Bool::False.transmute_into(); +``` + +(Note: #[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] annotation connotes that all aspects of Bool's layout are part of its library stability guarantee.) + + +...because all possible instances of `Bool` are also valid instances of [`u8`]. However, transmuting a [`u8`] into a `Bool` is forbidden: +```rust +/* ⚠️ This example intentionally does not compile. */ +let _ : Bool = u8::default().transmute_into(); // Compile Error! +``` +...because not all instances of [`u8`] are valid instances of `Bool`. + +Another example: While laying out certain types, Rust may insert padding bytes between the layouts of fields. In the below example `Padded` has two padding bytes, while `Packed` has none: +```rust +#[repr(C)] +#[derive(Default, PromiseTransmutableFrom, PromiseTransmutableInto)] +struct Padded(pub u8, pub u16, pub u8); + +#[repr(C)] +#[derive(Default, PromiseTransmutableFrom, PromiseTransmutableInto)] +struct Packed(pub u16, pub u16, pub u16); + +assert_eq!(mem::size_of::(), mem::size_of::()); +``` + +We may safely transmute from `Packed` to `Padded`: +```rust +let _ : Padded = Packed::default().transmute_into(); +``` +...but not from `Padded` to `Packed`: +```rust +/* ⚠️ This example intentionally does not compile. */ +let _ : Packed = Padded::default().transmute_into(); // Compile Error! +``` +...because doing so would expose two uninitialized padding bytes in `Padded` as if they were initialized bytes in `Packed`. + +##### Preserve or Shrink Size +[owned-size]: #Preserve-or-Shrink-Size + +It's completely sound to transmute into a type with fewer bytes than the source type; e.g.: +```rust +let _ : [u8; 16] = [u8; 32]::default().transmute_into(); +``` +This transmute truncates away the final sixteen bytes of the `[u8; 32]` value. + +A value may ***not*** be transmuted into a type of greater size, if doing so would expose uninitialized bytes as initialized: +```rust +/* ⚠️ This example intentionally does not compile. */ +let _ : [u8; 32] = [u8; 16]::default().transmute_into(); // Compile Error! +``` + +#### Requirements on References +[transmute-references]: #requirements-on-references + +The [restrictions above that apply to transmuting owned values][transmute-owned] also apply to transmuting references. However, references carry a few additional restrictions. + +A [sound transmutation] must: + - [preserve or shrink size][reference-size], + - [preserve or relax alignment][reference-alignment], + - [preserve or shrink lifetimes][reference-lifetimes], + - [preserve or shrink uniqueness][reference-mutability], and + - and if the destination type is a mutate-able reference, [preserve validity][reference-validity]. + +##### Preserve or Shrink Size +[reference-size]: #Preserve-or-Shrink-Size + +You may preserve or decrease the size of the referent type via transmutation: +```rust +let _: &[u8; 3] = (&[0u8; 9]).transmute_into(); +``` + +However, you may **not**, under any circumstances, *increase* the size of the referent type: +```rust +/* ⚠️ This example intentionally does not compile. */ +let _: &[u8; 9] = (&[0u8; 3]).transmute_into(); // Compile Error! +``` +##### Preserve or Relax Alignment +[reference-alignment]: #Preserve-or-Relax-Alignment + +Unaligned loads are undefined behavior. You may transmute a reference into reference of more relaxed alignment: +```rust +let _: &[u8; 0] = (&[0u16; 0]).transmute_into(); +``` + +However, you may **not** transmute a reference into a reference of more-restrictive alignment: +```rust +/* ⚠️ This example intentionally does not compile. */ +let _: &[u16; 0] = (&[0u8; 0]).transmute_into(); // Compile Error! +``` + +##### Preserve or Shrink Lifetimes +[reference-lifetimes]: #Preserve-or-Shrink-Lifetimes + +You may transmute a reference into a reference of lesser lifetime: +```rust +fn shrink<'a>() -> &'a u8 { + static long : &'static u8 = &16; + long.transmute_into() +} +``` + +However, you may **not** transmute a reference into a reference of greater lifetime: +```rust +/* ⚠️ This example intentionally does not compile. */ +fn extend<'a>(short: &'a u8) -> &'static u8 { + short.transmute_into() // Compile Error! +} +``` + +##### Preserve or Shrink Uniqueness +[reference-mutability]: #Preserve-or-Shrink-Uniqueness + +You may preserve or decrease the uniqueness of a reference through transmutation: +```rust +let _: &u8 = (&42u8).transmute_into(); +let _: &u8 = (&mut 42u8).transmute_into(); +``` + +However, you may **not** transmute a shared reference into a unique reference: +```rust +/* ⚠️ This example intentionally does not compile. */ +let _: &mut u8 = (&42u8).transmute_into(); // Compile Error! +``` + +##### Mutate-able References Must Preserve Validity +[reference-validity]: #Mutate-able-References-Must-Preserve-Validity + +A mutate-able reference is: +- all unique (i.e., `&mut T`) references +- all shared (i.e., `&T`) references whose referent type contain any bytes produced by the contents of `UnsafeCell`. + +Unlike transmutations of owned values, the transmutation of a mutate-able reference may also not expand the bit-validity of the referenced type. For instance: +```rust +/* ⚠️ This example intentionally does not compile. */ +let mut x = NonZeroU8::new(42).unwrap(); +{ + let y : &mut u8 = (&mut x).transmute_into(); // Compile Error! + *y = 0; +} + +let z : NonZeroU8 = x; +``` +If this example did not produce a compile error, the value of `z` would not be a bit-valid instance of its type, [`NonZeroU8`]. + + + +### 📖 When is a transmutation safe? +A sound transmutation is ***safe*** if *using* the transmuted value safely cannot violate memory safety. Whereas soundness solely concerns the act of transmutation, *safety* is concerned with what might happen with a value *after* transmutation occurs. + +#### Implicit Constructability +A struct or enum variant is *fully implicitly constructable* at a given location only if, at that location, that type can be instantiated via its *implicit constructor*, and its fields are also *implicitly constructable*. + +The *implicit constructor* of a struct or enum variant is the constructor Rust creates implicitly from its definition; e.g.: +```rust +struct Point { + x: T, + y: T, +} + +let p = Point { x: 4, y: 2 }; + // ^^^^^^^^^^^^^^^^^^^^ An instance of `Point` is created here, via its implicit constructor. +``` + +Limiting implicit constructability is the fundamental mechanism with which type authors build safe abstractions for `unsafe` code, whose soundness is dependent on preserving invariants on fields. Usually, this takes the form of restricting the visibility of fields. For instance, consider the type `NonEmptySlice`, which enforces a validity constraint on its fields via its constructor: + +```rust +pub mod crate_a { + + #[repr(C)] + pub struct NonEmptySlice<'a, T> { + data: *const T, + len: usize, + lifetime: core::marker::PhantomData<&'a ()>, + } + + impl<'a, T> NonEmptySlice<'a, T> { + pub fn from_array(arr: &'a [T; N], len: usize) -> Self { + assert!(len <= N); + assert!(len > 0); + Self { + data: arr as *const T, + len, + lifetime: core::marker::PhantomData, + } + } + + pub fn first(&self) -> &'a T { + unsafe { &*self.data } + } + } + +} +``` +It is sound for `first` to be a *safe* method is because the `from_array` constructor ensures that `data` is safe to dereference, and because `from_array` is the *only* way to safely initialize `NonEmptySlice` outside of `crate_a` (note that `NonEmptySlice`'s fields are *not* `pub`). As a rule: any field that is not marked `pub` should be assumed to be private *because* it is subject to safety invariants. + +Unfortunately, field visibility modifiers are not a surefire indicator of whether a type is *fully* implicitly constructable. A type author may restrict the implicit constructability of a type even in situations where all fields of that type (*and all fields of those fields*) are `pub`; consider: +```rust +pub mod crate_a { + + #[repr(C)] + pub struct NonEmptySlice<'a, T>(pub private::NonEmptySliceInner<'a, T>); + + impl<'a, T> NonEmptySlice<'a, T> { + pub fn from_array(arr: &'a [T; N], len: usize) -> Self { + assert!(len <= N && len > 0); + Self( + private::NonEmptySliceInner { + data: arr as *const T, + len, + lifetime: core::marker::PhantomData, + } + ) + } + + pub fn first(&self) -> &'a T { + unsafe { &*self.0.data } + } + } + + // introduce a private module to avoid `private_in_public` error (E0446): + pub(crate) mod private { + #[repr(C)] + pub struct NonEmptySliceInner<'a, T> { + pub data: *const T, + pub len: usize, + pub lifetime: core::marker::PhantomData<&'a ()>, + } + } + +} +``` +In the above example, the definitions of both `NonEmptySlice` and its field `NonEmptySliceInner` are marked `pub`, and all fields of these types are marked `pub`. However, `NonEmptySlice` is *not* fully implicitly constructible outside of `crate_a`, because the module containing `NonEmptySliceInner` is not visibile outside of `crate_a`. + +#### Constructability and Transmutation +Transmutation supplies a mechanism for constructing instances of a type *without* invoking its implicit constructor, nor any constructors defined by the type's author. + +In the previous examples, it would be *unsafe* to transmute `0u128` into `NonEmptySlice` outside `crate_a`, because subsequent *safe* use of that value (namely, calling `first`) would violate memory safety. (However, it's completely safe to transmute `NonEmptySlice` into a `u128`.) + +For transmutations where the destination type involves mutate-able references, the constructability of the source type is also relevant. Consider: +```rust +/* ⚠️ This example intentionally does not compile. */ +let arr = [0u8, 1u8, 2u8]; +let mut x = NonEmptySlice::from_array(&arr, 2); +{ + let y : &mut u128 = (&mut x).transmute_into(); // Compile Error! + *y = 0u128; +} + +let z : NonEmptySlice = x; +``` +If this example did not produce a compile error, the value of `z` would not be a safe instance of its type, `NonEmptySlice`, because `z.first()` would dereference a null pointer. + +### 📖 When is a transmutation stable? +[stability]: #-when-is-a-transmutation-stable + +Since the soundness and safety of a transmutation is affected by the layouts of the source and destination types, changes to those types' layouts may cause code which previously compiled to produce errors. In other words, transmutation causes a type's layout to become part of that type's API for the purposes of SemVer stability. + +The question is, then: *how can the author of a type reason about transmutations they did not write, from-or-to types they did not write?* We address this problem by introducing two traits which both allow an author to opt-in to stability guarantees for their types, and allow third-parties to reason at compile-time about what guarantees are provided for such types. + +#### `PromiseTransmutableFrom` and `PromiseTransmutableInto` + +You may declare the stability guarantees of your type by implementing one or both of two traits: +```rust +pub trait PromiseTransmutableFrom +where + Self::Archetype: PromiseTransmutableFrom +{ + type Archetype: TransmuteInto +} + +pub trait PromiseTransmutableInto +where + Self::Archetype: PromiseTransmutableInto, +{ + type Archetype: TransmuteFrom +} +``` + +To implement each of these traits, you must specify an `Archetype`. An `Archetype` is a type whose layout exemplifies the extremities of your stability promise (i.e., the least/most constrained type for which it is valid to transmute your type into/from). + +By implementing `PromiseTransmutableFrom`, you promise that your type is guaranteed to be safely transmutable *from* `PromiseTransmutableFrom::Archetype`. Conversely, by implementing `PromiseTransmutableInto`, you promise that your type is guaranteed to be safely transmutable *into* `PromiseTransmutableInto::Archetype`. + +You are free to change the layout of your type however you like between minor crate versions so long as that change does not violates these promises. These two traits are capable of expressing simple and complex stability guarantees. + +#### Stability & Transmutation +Together with the `PromiseTransmutableFrom` and `PromiseTransmutableInto` traits, this impl of `TransmuteFrom` constitutes the formal definition of transmutation stability: +```rust +unsafe impl TransmuteFrom for Dst +where + Src: PromiseTransmutableInto, + Dst: PromiseTransmutableFrom, + + Dst::Archetype: TransmuteFrom +{} +``` +Why is this safe? Can we really safely judge whether `Dst` is transmutable from `Src` by assessing the transmutability of two different types? Yes! Transmutability is *transitive*. Concretely, if we can safely transmute: + - `Src` to `Src::Archetype` (enforced by `Src: PromiseTransmutableInto`), and + - `Dst::Archetype` to `Dst` (enforced by `Dst: PromiseTransmutableFrom`), and + - `Src::Archetype` to `Dst::Archetype` (enforced by `Dst::Archetype: TransmuteFrom`), + +...then it follows that we can safely transmute `Src` to `Dst` in three steps: + 1. we transmute `Src` to `Src::Archetype`, + 2. we transmute `Src::Archetype` to `Dst::Archetype`, + 3. we transmute `Dst::Archetype` to `Dst`. + +#### Common Use-Case: As-Stable-As-Possible +[stability-common]: #common-use-case-as-stable-as-possible +To promise that all transmutations which are currently safe for your type will remain so in the future, simply annotate your type with: +```rust +#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] +#[repr(C)] +pub struct Foo(pub Bar, pub Baz); +``` +This expands to: +```rust +#[repr(C)] +pub struct Foo(pub Bar, pub Baz); + +/// Generated `PromiseTransmutableFrom` for `Foo` +const _: () = { + use core::transmute::stability::PromiseTransmutableFrom; + + #[repr(C)] + pub struct TransmutableFromArchetype( + pub ::Archetype, + pub ::Archetype, + ); + + impl PromiseTransmutableFrom for TransmutableFromArchetype { + type Archetype = Self; + } + + impl PromiseTransmutableFrom for Foo { + type Archetype = TransmutableFromArchetype; + } +}; + +/// Generated `PromiseTransmutableInto` for `Foo` +const _: () = { + use core::transmute::stability::PromiseTransmutableInto; + + #[repr(C)] + pub struct TransmutableIntoArchetype( + pub ::Archetype, + pub ::Archetype, + ); + + impl PromiseTransmutableFrom for TransmutableIntoArchetype { + type Archetype = Self; + } + + impl PromiseTransmutableInto for Foo { + type Archetype = TransmutableIntoArchetype; + } +}; +``` +Since deriving *both* of these traits together is, by far, the most common use-case, we [propose][extension-promisetransmutable-shorthand] `#[derive(PromiseTransmutable)]` as an ergonomic shortcut. + + +#### Uncommon Use-Case: Weak Stability Guarantees +[stability-uncommon]: #uncommon-use-case-weak-stability-guarantees + +We also can specify *custom* `Archetype`s to finely constrain the set of transmutations we are willing to make stability promises for. Consider, for instance, if we want to leave ourselves the future leeway to change the alignment of a type `Foo` without making a SemVer major change: +```rust +#[repr(C)] +pub struct Foo(pub Bar, pub Baz); +``` +The alignment of `Foo` affects transmutability of `&Foo`. A `&Foo` cannot be safely transmuted from a `&Bar` if the alignment requirements of `Foo` exceed those of `Bar`. If we don't want to promise that `&Foo` is stably transmutable from virtually *any* `Bar`, we simply make `Foo`'s `PromiseTransmutableFrom::Archetype` a type with maximally strict alignment requirements: +```rust +const _: () = { + use core::transmute::stability::PromiseTransmutableFrom; + + #[repr(C, align(536870912))] + pub struct TransmutableFromArchetype( + pub ::Archetype, + pub ::Archetype, + ); + + impl PromiseTransmutableFrom for TransmutableFromArchetype { + type Archetype = Self; + } + + impl PromiseTransmutableFrom for Foo { + type Archetype = TransmutableFromArchetype; + } +}; +``` +Conversely, a `&Foo` cannot be safely transmuted *into* a `&Bar` if the alignment requirements of `Bar` exceed those of `Foo`. We reduce this set of stable transmutations by making `PromiseTransmutableFrom::Archetype` a type with minimal alignment requirements: +```rust +const _: () = { + use core::transmute::stability::PromiseTransmutableInto; + + #[repr(C, packed(1))] + pub struct TransmutableIntoArchetype( + pub ::Archetype, + pub ::Archetype, + ); + + impl PromiseTransmutableInto for TransmutableIntoArchetype { + type Archetype = Self; + } + + impl PromiseTransmutableInto for Foo { + type Archetype = TransmutableIntoArchetype; + } +}; +``` +Given these two stability promises, we are free to modify the alignment of `Foo` in SemVer-minor changes without running any risk of breaking dependent crates. + + +## Mechanisms of Transmutation + +Two traits provide mechanisms for transmutation between types: +```rust +// this trait is implemented automagically by the compiler +#[lang = "transmute_from"] +pub unsafe trait TransmuteFrom +where + Neglect: TransmuteOptions, +{ + #[inline(always)] + fn transmute_from(src: Src) -> Self + where + Src: Sized, + Self: Sized, + Neglect: SafeTransmuteOptions, + { + unsafe { Self::unsafe_transmute_from(src) } + } + + #[inline(always)] + unsafe fn unsafe_transmute_from(src: Src) -> Self + where + Src: Sized, + Self: Sized, + Neglect: TransmuteOptions, + { + use core::mem::ManuallyDrop; + + #[repr(C)] + union Transmute { + src: ManuallyDrop, + dst: ManuallyDrop, + } + + unsafe { + ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst) + } + } +} + +// implemented in terms of `TransmuteFrom` +pub unsafe trait TransmuteInto +where + Neglect: TransmuteOptions, +{ + fn transmute_into(self) -> Dst + where + Self: Sized, + Dst: Sized, + Neglect: SafeTransmuteOptions; + + unsafe fn unsafe_transmute_into(self) -> Dst + where + Self: Sized, + Dst: Sized, + Neglect: TransmuteOptions; +} + +unsafe impl TransmuteInto for Src +where + Src: ?Sized, + Dst: ?Sized + TransmuteFrom, + Neglect: TransmuteOptions, +{ + ... +} +``` + +In the above definitions, `Src` represents the source type of the transmutation, `Dst` represents the destination type of the transmutation, and `Neglect` is a parameter that [encodes][options] which static checks the compiler ought to neglect when considering if a transmutation is valid. The default value of `Neglect` is `()`, which reflects that, by default, the compiler does not neglect *any* static checks. + +### Neglecting Static Checks +[options]: #Neglecting-Static-Checks + +The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are unsafe, unsound, or unstable. However, you may explicitly opt-out of some static checks: + +| Transmute Option | Compromises | Usable With | +|---------------------|-------------|---------------------------------------------------------| +| `NeglectStabilty` | Stability | `transmute_{from,into}`, `unsafe_transmute_{from,into}` | +| `NeglectAlignment` | Safety | `unsafe_transmute_{from,into}` | +| `NeglectValidity` | Soundness | `unsafe_transmute_{from,into}` | + +`NeglectStabilty` implements the `SafeTransmuteOptions` and `TransmuteOptions` marker traits, as it can be used in both safe and unsafe code. The selection of multiple options is encoded by grouping them as a tuple; e.g., `(NeglectAlignment, NeglectValidity)` is a selection of both the `NeglectAlignment` and `NeglectValidity` options. + +We introduce two marker traits which serve to group together the options that may be used with safe transmutes, and those which may be used with `unsafe` transmutes: +```rust +pub trait SafeTransmuteOptions: private::Sealed +{} + +pub trait TransmuteOptions: SafeTransmuteOptions +{} + +impl SafeTransmuteOptions for () {} +impl TransmuteOptions for () {} +``` + +#### `NeglectStability` +[`NeglectStability`]: #neglectstability + +By default, `TransmuteFrom` and `TransmuteInto`'s methods require that the [layouts of the source and destination types are SemVer-stable][stability]. The `NeglectStability` option disables this requirement. +```rust +pub struct NeglectStability; + +impl SafeTransmuteOptions for NeglectStability {} +impl TransmuteOptions for NeglectStability {} +``` + +Prior to the adoption of the [stability declaration traits][stability], crate authors documented the layout guarantees of their types with doc comments. The `TransmuteFrom` and `TransmuteInto` traits and methods may be used with these types by requesting that the stability check is neglected; for instance: + +```rust +fn serialize(val : LibraryType, dst: W) -> std::io::Result<()> +where + LibraryType: TransmuteInto<[u8; size_of::()], NeglectStability> +{ + ... +} +``` + +Neglecting stability over-eagerly cannot cause unsoundness or unsafety. For this reason, it is the only transmutation option available on the safe methods `transmute_from` and `transmute_into`. However, neglecting stability over-eagerly may cause your code to cease compiling if the authors of the source and destination types make changes that affect their layout. + +By using the `NeglectStability` option to transmute types you do not own, you are committing to ensure that your reliance on these types' layouts is consistent with their documented stability guarantees. + +#### `NeglectAlignment` +[ext-ref-casting]: #NeglectAlignment + +By default, `TransmuteFrom` and `TransmuteInto`'s methods require that, when transmuting references, the minimum alignment of the destination's referent type is no greater than the minimum alignment of the source's referent type. The `NeglectAlignment` option disables this requirement. +```rust +pub struct NeglectAlignment; + +impl TransmuteOptions for NeglectAlignment {} +``` + +By using the `NeglectAlignment` option, you are committing to ensure that the transmuted reference satisfies the alignment requirements of the destination's referent type. For instance: +```rust +/// Try to convert a `&T` into `&U`. +/// +/// This produces `None` if the referent isn't appropriately +/// aligned, as required by the destination type. +pub fn try_cast_ref<'t, 'u, T, U>(src: &'t T) -> Option<&'u U> +where + &'t T: TransmuteInto<&'u U, NeglectAlignment>, +{ + if (src as *const T as usize) % align_of::() != 0 { + None + } else { + // Safe because we dynamically enforce the alignment + // requirement, whose static check we chose to neglect. + Some(unsafe { src.unsafe_transmute_into() }) + } +} +``` + +#### `NeglectValidity` +By default, `TransmuteFrom` and `TransmuteInto`'s methods require that all instantiations of the source type are guaranteed to be valid instantiations of the destination type. This precludes transmutations which *might* be valid depending on the source value: +```rust +#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] +#[repr(u8)] +enum Bool { + True = 1, + False = 0, +} + +/* ⚠️ This example intentionally does not compile. */ +let _ : Bool = some_u8_value.transmute_into(); // Compile Error! +``` +The `NeglectValidity` option disables this check. +```rust +pub struct NeglectValidity; + +impl TransmuteOptions for NeglectValidity {} +``` + +By using the `NeglectValidity` option, you are committing to ensure dynamically source value is a valid instance of the destination type. For instance: +```rust +#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] +#[repr(u8)] +enum Bool { + True = 1, + False = 0, +} + +pub trait TryIntoBool +{ + fn try_into_bool(self) -> Option; +} + +impl TryIntoBool for T +where + T: TransmuteInto, + u8: TransmuteInto +{ + fn try_into_bool(self) -> Option { + let val: u8 = self.transmute_into(); + + if val > 1 { + None + } else { + // Safe, because we've first verified that + // `val` is a bit-valid instance of a boolean. + Some(unsafe {val.unsafe_transmute_into()}) + } + } +} +``` + +Even with `NeglectValidity`, the compiler will statically reject transmutations that cannot possibly be valid: +```rust +#[derive(PromiseTransmutableInto)] +#[repr(C)] enum Foo { A = 24 } + +#[derive(PromiseTransmutableFrom)] +#[repr(C)] enum Bar { Z = 42 } + +let _ = ::unsafe_transmute_from(Foo::N) // Compile error! +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + + +## Implementation Guidance +The *only* item defined by this RFC requiring special compiler support is `TransmuteFrom`. To realize this RFC's proposal of safe transmutation between different types, this item will require compiler support. However, the *most* minimal acceptable implementation of `TransmuteFrom` can be achieved entirely in-language: + +```rust +/// A type is transmutable into itself. +unsafe impl TransmuteFrom for T +where + Neglect: TransmuteOptions +{} + +/// A transmutation is *stable* if... +unsafe impl TransmuteFrom for Dst +where + Src: PromiseTransmutableInto, + Dst: PromiseTransmutableFrom, + ::Archetype: + TransmuteFrom< + ::Archetype, + NeglectStability + > +{} +``` +This [minimal implementation][minimal-impl] is sufficient for convincing the compiler to accept basic stability declarations, such as those of Rust's primitive types. It is *insufficient* for making the compiler accept transmutations between *different* types (and, consequently, complex stability declarations). Implementers should use this as a starting point. + +### Listing for Initial, Minimal Implementation +[minimal-impl]: #Listing-for-Initial-Minimal-Implementation +This listing is both a minimal implementation of this RFC (excepting the automatic derives) and the **canonical specification** of this RFC's API surface ([playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=1ff1700e6dba595f1e600d20da5d6387)): +```rust +#![feature(untagged_unions,const_fn,const_fn_union)] // for the impl of transmute free functions +#![feature(const_generics)] // for stability declarations on `[T; N]` +#![feature(decl_macro)] // for stub implementations of derives +#![feature(never_type)] // for stability declarations on `!` +#![allow(warnings)] + +/// Transmutation conversions. +// suggested location: `core::convert` +pub mod transmute { + + use {options::*, stability::*}; + + /// Reinterprets the bits of a value of one type as another type, safely. + /// + /// Use `()` for `Neglect` if you do not wish to neglect any static checks. + #[inline(always)] + pub const fn safe_transmute(src: Src) -> Dst + where + Dst: TransmuteFrom, + Neglect: SafeTransmuteOptions + { + unsafe { unsafe_transmute::(src) } + } + + /// Reinterprets the bits of a value of one type as another type, potentially unsafely. + /// + /// The onus is on you to ensure that calling this method is safe. + #[inline(always)] + pub const unsafe fn unsafe_transmute(src: Src) -> Dst + where + Dst: TransmuteFrom, + Neglect: TransmuteOptions + { + use core::mem::ManuallyDrop; + + #[repr(C)] + union Transmute { + src: ManuallyDrop, + dst: ManuallyDrop, + } + + unsafe { + ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst) + } + } + + /// `Self: TransmuteInto + where + Neglect: TransmuteOptions, + { + /// Reinterpret the bits of a value of one type as another type, safely. + fn transmute_into(self) -> Dst + where + Self: Sized, + Dst: Sized, + Neglect: SafeTransmuteOptions; + + /// Reinterpret the bits of a value of one type as another type, potentially unsafely. + /// + /// The onus is on you to ensure that calling this method is safe. + unsafe fn unsafe_transmute_into(self) -> Dst + where + Self: Sized, + Dst: Sized, + Neglect: TransmuteOptions; + } + + unsafe impl TransmuteInto for Src + where + Src: ?Sized, + Dst: ?Sized + TransmuteFrom, + Neglect: TransmuteOptions, + { + #[inline(always)] + fn transmute_into(self) -> Dst + where + Self: Sized, + Dst: Sized, + Neglect: SafeTransmuteOptions, + { + Dst::transmute_from(self) + } + + #[inline(always)] + unsafe fn unsafe_transmute_into(self) -> Dst + where + Self: Sized, + Dst: Sized, + Neglect: TransmuteOptions, + { + unsafe { Dst::unsafe_transmute_from(self) } + } + } + + /// `Self: TransmuteInto + where + Neglect: TransmuteOptions, + { + /// Reinterpret the bits of a value of one type as another type, safely. + #[inline(always)] + fn transmute_from(src: Src) -> Self + where + Src: Sized, + Self: Sized, + Neglect: SafeTransmuteOptions, + { + unsafe { Self::unsafe_transmute_from(src) } + } + + /// Reinterpret the bits of a value of one type as another type, potentially unsafely. + /// + /// The onus is on you to ensure that calling this function is safe. + #[inline(always)] + unsafe fn unsafe_transmute_from(src: Src) -> Self + where + Src: Sized, + Self: Sized, + Neglect: TransmuteOptions, + { + unsafe_transmute::(src) + } + } + + /// A type is always transmutable from itself. + // This impl will be replaced with a compiler-supported for arbitrary source and destination types. + unsafe impl TransmuteFrom for T {} + + /// A type is *stably* transmutable if... + unsafe impl TransmuteFrom for Dst + where + Src: PromiseTransmutableInto, + Dst: PromiseTransmutableFrom, + ::Archetype: + TransmuteFrom< + ::Archetype, + NeglectStability + > + {} + + /// Traits for declaring the SemVer stability of types. + pub mod stability { + + use super::{TransmuteFrom, TransmuteInto, options::NeglectStability}; + + /// Declare that transmuting `Self` into `Archetype` is SemVer-stable. + pub trait PromiseTransmutableInto + where + Self::Archetype: PromiseTransmutableInto + { + /// The `Archetype` must be safely transmutable from `Self`. + type Archetype: TransmuteFrom; + } + + /// Declare that transmuting `Self` from `Archetype` is SemVer-stable. + pub trait PromiseTransmutableFrom + where + Self::Archetype: PromiseTransmutableFrom + { + /// The `Archetype` must be safely transmutable into `Self`. + type Archetype: TransmuteInto; + } + + + /// Derive macro generating an impl of the trait `PromiseTransmutableInto`. + //#[rustc_builtin_macro] + pub macro PromiseTransmutableInto($item:item) { + /* compiler built-in */ + } + + /// Derive macro generating an impl of the trait `PromiseTransmutableFrom`. + //#[rustc_builtin_macro] + pub macro PromiseTransmutableFrom($item:item) { + /* compiler built-in */ + } + + + impl PromiseTransmutableInto for ! {type Archetype = Self;} + impl PromiseTransmutableFrom for ! {type Archetype = Self;} + + impl PromiseTransmutableInto for () {type Archetype = Self;} + impl PromiseTransmutableFrom for () {type Archetype = Self;} + + impl PromiseTransmutableInto for f32 {type Archetype = Self;} + impl PromiseTransmutableFrom for f32 {type Archetype = Self;} + impl PromiseTransmutableInto for f64 {type Archetype = Self;} + impl PromiseTransmutableFrom for f64 {type Archetype = Self;} + + impl PromiseTransmutableInto for i8 {type Archetype = Self;} + impl PromiseTransmutableFrom for i8 {type Archetype = Self;} + impl PromiseTransmutableInto for i16 {type Archetype = Self;} + impl PromiseTransmutableFrom for i16 {type Archetype = Self;} + impl PromiseTransmutableInto for i32 {type Archetype = Self;} + impl PromiseTransmutableFrom for i32 {type Archetype = Self;} + impl PromiseTransmutableInto for i64 {type Archetype = Self;} + impl PromiseTransmutableFrom for i64 {type Archetype = Self;} + impl PromiseTransmutableInto for i128 {type Archetype = Self;} + impl PromiseTransmutableFrom for i128 {type Archetype = Self;} + impl PromiseTransmutableInto for isize {type Archetype = Self;} + impl PromiseTransmutableFrom for isize {type Archetype = Self;} + + impl PromiseTransmutableInto for u8 {type Archetype = Self;} + impl PromiseTransmutableFrom for u8 {type Archetype = Self;} + impl PromiseTransmutableInto for u16 {type Archetype = Self;} + impl PromiseTransmutableFrom for u16 {type Archetype = Self;} + impl PromiseTransmutableInto for u32 {type Archetype = Self;} + impl PromiseTransmutableFrom for u32 {type Archetype = Self;} + impl PromiseTransmutableInto for u64 {type Archetype = Self;} + impl PromiseTransmutableFrom for u64 {type Archetype = Self;} + impl PromiseTransmutableInto for u128 {type Archetype = Self;} + impl PromiseTransmutableFrom for u128 {type Archetype = Self;} + impl PromiseTransmutableInto for usize {type Archetype = Self;} + impl PromiseTransmutableFrom for usize {type Archetype = Self;} + + use core::marker::PhantomData; + impl PromiseTransmutableInto for PhantomData { type Archetype = Self; } + impl PromiseTransmutableFrom for PhantomData { type Archetype = Self; } + + + impl PromiseTransmutableInto for [T; N] + where + T: PromiseTransmutableInto, + [T::Archetype; N] + : TransmuteFrom + + PromiseTransmutableInto, + { + type Archetype = [T::Archetype; N]; + } + + impl PromiseTransmutableFrom for [T; N] + where + T: PromiseTransmutableFrom, + [T::Archetype; N] + : TransmuteInto + + PromiseTransmutableFrom, + { + type Archetype = [T::Archetype; N]; + } + + + impl PromiseTransmutableInto for *const T + where + T: PromiseTransmutableInto, + *const T::Archetype + : TransmuteFrom + + PromiseTransmutableInto, + { + type Archetype = *const T::Archetype; + } + + impl PromiseTransmutableFrom for *const T + where + T: PromiseTransmutableFrom, + *const T::Archetype + : TransmuteInto + + PromiseTransmutableFrom, + { + type Archetype = *const T::Archetype; + } + + + impl PromiseTransmutableInto for *mut T + where + T: PromiseTransmutableInto, + *mut T::Archetype + : TransmuteFrom + + PromiseTransmutableInto, + { + type Archetype = *mut T::Archetype; + } + + impl PromiseTransmutableFrom for *mut T + where + T: PromiseTransmutableFrom, + *mut T::Archetype + : TransmuteInto + + PromiseTransmutableFrom, + { + type Archetype = *mut T::Archetype; + } + + + impl<'a, T: ?Sized> PromiseTransmutableInto for &'a T + where + T: PromiseTransmutableInto, + &'a T::Archetype + : TransmuteFrom<&'a T, NeglectStability> + + PromiseTransmutableInto, + { + type Archetype = &'a T::Archetype; + } + + impl<'a, T: ?Sized> PromiseTransmutableFrom for &'a T + where + T: PromiseTransmutableFrom, + &'a T::Archetype + : TransmuteInto<&'a T, NeglectStability> + + PromiseTransmutableFrom, + { + type Archetype = &'a T::Archetype; + } + + impl<'a, T: ?Sized> PromiseTransmutableInto for &'a mut T + where + T: PromiseTransmutableInto, + &'a mut T::Archetype + : TransmuteFrom<&'a mut T, NeglectStability> + + PromiseTransmutableInto, + { + type Archetype = &'a mut T::Archetype; + } + + impl<'a, T: ?Sized> PromiseTransmutableFrom for &'a mut T + where + T: PromiseTransmutableFrom, + &'a mut T::Archetype + : TransmuteInto<&'a mut T, NeglectStability> + + PromiseTransmutableFrom, + { + type Archetype = &'a mut T::Archetype; + } + } + + /// Static checks that may be neglected when determining if a type is `TransmuteFrom` some other type. + pub mod options { + + /// Options that may be used with safe transmutations. + pub trait SafeTransmuteOptions: TransmuteOptions {} + + /// Options that may be used with unsafe transmutations. + pub trait TransmuteOptions: private::Sealed {} + + impl SafeTransmuteOptions for () {} + impl TransmuteOptions for () {} + + /// Neglect the stability check of `TransmuteFrom`. + pub struct NeglectStability; + + // Uncomment this if/when constructibility is fully implemented: + // impl SafeTransmuteOptions for NeglectStability {} + impl TransmuteOptions for NeglectStability {} + + // prevent third-party implementations of `TransmuteOptions` + mod private { + use super::*; + + pub trait Sealed {} + + impl Sealed for () {} + impl Sealed for NeglectStability {} + } + } +} +``` + +### Towards an Initial, Smart Implementation + +To support transmutations between different types, implementers of this RFC should begin by defining a `transmute_from` lang item to annotate libcore's definition of `TransmuteFrom`. Whether `TransmuteFrom` is implemented for a given type and parameters shall be determined within the implementation of the type system (à la [`Sized`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_sized) and [`Freeze`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_freeze)). + +This initial smart implementation may be made simpler by: + - *not* supporting enums + - *not* supporting unions + - *not* supporting `Neglect` parameters besides `()` and `NeglectStability` + - simplifying constructability + +#### Simplifying Constructability +The safety property of constructability defined in the guidance-level explanation of this RFC describes a platonic ideal of the property. + +However, we recognize that this definition poses implementation challenges: In our definition of constructability, answering the question of whether a struct or enum variant is constructible depends on *where* that question is being asked. Consequently, answering whether a given type `Src` is `TransmutableInto` a given type `Dst` will depend on *where* that question is posed. + +We recommend adopting a simplified definition of *constructability*: a type is *constructible* if its fields are marked `pub`, and those fields are constructible. With this definition, answering the question of whether a type is constructible does *not* depend on where the question is asked: we do not examine the visibility of the involved types. + +Unfortunately, with no other actions taken, this simplified definition comes... + +##### ...at the Cost of Safety +This definition is *usually* sufficient for ensuring safety: it is *generally* an error to expose a private type in a public type signature. However, these errors may be circumvented using the public-type-in-private-module trick: +```rust +pub mod crate_a { + + #[repr(C)] + pub struct NonEmptySlice<'a, T>(pub private::NonEmptySliceInner<'a, T>); + + impl<'a, T> NonEmptySlice<'a, T> { + pub fn from_array(arr: &'a [T; N], len: usize) -> Self { + assert!(len <= N && len > 0); + Self( + private::NonEmptySliceInner { + data: arr as *const T, + len, + lifetime: core::marker::PhantomData, + } + ) + } + + pub fn first(&self) -> &'a T { + unsafe { &*self.0.data } + } + } + + // introduce a private module to avoid `private_in_public` error (E0446): + pub(crate) mod private { + #[repr(C)] + pub struct NonEmptySliceInner<'a, T> { + pub data: *const T, + pub len: usize, + pub lifetime: core::marker::PhantomData<&'a ()>, + } + } + +} +``` +With this simplified definition of constructability, it is possible for a third-party to define a *safe* constructor of `NonEmptySlice` that produces a value which is *unsafe* to use: +```rust +pub evil_constructor(src: T) -> NonEmptySlice<'static, u8> +where + T: TransmuteInto, NeglectStability>, +{ + src.transmute_into() +} + +evil_constructor(0u128).first() // muahaha! +``` + +The above code is "safe" because our simplified definition of constructability fails to recognize this pattern of encapsulation, and because `NeglectStability` is a `SafeTransmutationOption`. + +The intent of `NeglectStability` is to permit the safe transmutation of types that predate the stabilization of the stability declaration traits. It also provides a convenient escape-hatch for type authors to neglect the stability of transmutations of their *own* types, without sacrificing safety. `NeglectStability` is a `SafeTransmutationOption` because, in principle, neglecting stability does not diminish safety. Our simplified definition of constructability violates this principle. + +By temporarily sacrificing these goals, we may preserve safety solely... + +##### ...at the Cost of `NeglectStability` +We may preserve safety by demoting `NeglectStability` to `UnsafeTransmutationOption`-status. + +In doing so, a third-party is forced to resort to an `unsafe` transmutation to construct `NonEmptySlice`; e.g.: + +```rust +pub evil_constructor(src: T) -> NonEmptySlice<'static, u8> +where + T: TransmuteInto, NeglectStability>, +{ + // unsafe because we `NeglectStability` + unsafe { src.unsafe_transmute_into() } +} +``` + +Demoting `NeglectStability` to unsafe-status does not stop type authors from opting-in to stable (and thus safe) transmutations; e.g., with `derive(PromiseTransmutableFrom)`: +```rust +pub mod crate_a { + + #[derive(PromiseTransmutableFrom)] + #[repr(C)] + pub struct NonEmptySlice<'a, T>(pub private::NonEmptySliceInner<'a, T>); + + impl<'a, T> NonEmptySlice<'a, T> { + pub fn from_array(arr: &'a [T; N], len: usize) -> Self { + assert!(len <= N && len > 0); + Self( + private::NonEmptySliceInner { + data: arr as *const T, + len, + lifetime: core::marker::PhantomData, + } + ) + } + + pub fn first(&self) -> &'a T { + unsafe { &*self.0.data } + } + } + + // introduce a private module to avoid `private_in_public` error (E0446): + pub(crate) mod private { + #[derive(PromiseTransmutableFrom)] + #[repr(C)] + pub struct NonEmptySliceInner<'a, T> { + pub data: *const T, + pub len: usize, + pub lifetime: core::marker::PhantomData<&'a ()>, + } + } + +} +``` +In the above example, the type author declares `NonEmptySlice` and `NonEmptySliceInner` to be stably instantiatable via transmutation. Given this, a third-party no longer needs to resort to `unsafe` code to violate the the invariants on `inner`: +```rust +pub evil_constructor(src: T) -> NonEmptySlice<'static, u8> +where + T: TransmuteInto>, +{ + src.transmute_into() +} + +evil_constructor(0u128).first() // muahaha! +``` +This safety hazard is not materially different from the one that would be induced if the type author implemented `DerefMut` for `NonEmptySlice`, or made the `private` module `pub`, or otherwise explicitly provided outsiders with unrestricted mutable access to `data`. + +##### Recommendation +We recommend that that implementers of this RFC initially simplify constructability by: + - adopting our simplified definition of constructability + - demoting `NeglectStability` to unsafe status (i.e., not implementing `SafeTransmuteOptions` for `NeglectStability`; *only* `TransmuteOptions`) + - advise users that implementing the stability declaration traits on types that are not fully-implicitly constructable will be a compiler-error will be a compiler error (i.e., these traits must not be implemented on types exploiting the pub-in-priv trick) + +If and when the implementation of `TransmuteFrom` encodes our complete definition of constructability, `NeglectStability` shall become a safe transmute option. + + +# Drawbacks +[drawbacks]: #drawbacks + +## No Notion of Platform Stability +The stability declaration traits communicate library layout stability, but not *platform* layout stability. A transmutation is platform-stable if it compiling one one platform implies it will compile on all other platforms. Unfortunately, platform-unstable types are common; e.g.: + +- All primitive number types have platform-dependent [endianness](https://en.wikipedia.org/wiki/Endianness). +- All pointer-related primitive types (`usize`, `isize`, `*const T`, `*mut T`, `&T`, `&mut T`) possess platform-dependent layouts; their sizes and alignments are well-defined, but vary between platforms. Concretely, whether `usize` is `TransmuteInto<[u8; 4]>` or `TransmuteInto<[u8; 8]>` will depend on the platform. +- The very existence of some types depends on platform, too; e.g., the contents of [`core::arch`](https://doc.rust-lang.org/stable/core/arch/), [`std::os`](https://doc.rust-lang.org/stable/std/os/), and [`core::sync::atomic`](https://doc.rust-lang.org/stable/std/sync/atomic/) all depend on platform. + +Our proposed stability system is oblivious to the inter-platform variations of these types. Expanding our stability system to be aware of inter-platform variations would introduce considerable additional complexity: +
    +
  1. + +**Cognitive Complexity:** For types whose layout varies between platforms, the [stability] declaration traits could, *perhaps*, be adapted to encode platform-related guarantees. We anticipate this would contribute substantial cognitive complexity. Type authors, even those with no interest in cross-platform stability, would nonetheless need to reason about the layout properties of their types on platforms that might not yet exist. +
    2. +
  2. + +**Ergonomic Complexity:** Platform instabilities are contagious: a type that *contains* a platform-unstable type is, itself, platform-unstable. Due to the sheer virulence of types with platform-dependent layouts, an explicit '`NeglectPlatformStability`' option would need to be used for *many* simple transmutations. The ergonomic cost of this would also be substantial. + +
    3. +
  3. + +**Implementation Complexity:** The mechanisms proposed by this RFC are, fundamentally, applications of and additions to Rust's type system (i.e., they're traits). Mechanisms that impact platform stability, namely `#[cfg(...)]` annotations, long precede type-resolution and layout computation in the compilation process. For instance, it's possible to define types with impossible layouts: +```rust +#[cfg(any())] +struct Recursive(Recursive); +``` +This program compiles successfully on all platforms because, from the perspective of later compilation stages, `Recursive` may as well not exist. + +
    4. +
+ +The issues of platform layout stability exposed by this RFC are not fundamentally different from the challenges of platform API stability. These challenges are already competently addressed by the mechanisms proposed in [RFC1868](https://github.com/rust-lang/rfcs/pull/1868). For this reason, and for the aforementioned concerns of additional complexity, we argue that communicating and enforcing platform layout stability must remain outside the scope of this RFC. + +## Stability of *Unsafe* Transmutations +[drawback-unsafe-stability]: #Stability-of-Unsafe-Transmutations + +The model of stability proposed by this RFC frames stability as a quality of *safe* transmutations. A type author cannot specify stability archetypes for *unsafe* transmutations, and it is reasonable to want to do so. + +To accommodate this, we may modify the definitions of `PromiseTransmutableFrom` and `PromiseTransmutableInto` to consume an optional `Neglect` parameter, to allow for stability declarations for unsafe transmutations: +```rust +pub trait PromiseTransmutableFrom +where + Neglect: TransmuteOptions +{ + type Archetype + : TransmuteInto> + + PromiseTransmutableInto>; +} + +pub trait PromiseTransmutableInto +where + Neglect: TransmuteOptions +{ + type Archetype + : TransmuteFrom> + + PromiseTransmutableInto>; +} +``` +Implementations of these traits for a given `Neglect` declares that a transmutation which is accepted while neglecting a particular set of checks (namely the set encoded by `Neglect`) will *continue* to be possible. + +We omit these definition from this RFC's recommendations because they are not completely satisfying. For instance, `Neglect` is a *logically* unordered set of options, but is encoded as a tuple (which *is* ordered). To declare a transmutation that requires neglecting validity and alignment checks as stable, only *one* of these impls ought to be necessary: + +```rust +impl PromiseTransmutableFrom<(NeglectAlignment, NeglectValidity)> for Foo +{ + ... +} + +impl PromiseTransmutableFrom<(NeglectValidity, NeglectAlignment)> for Foo +{ + ... +} +``` +Writing *both* impls (as we do above) is logically nonsense, but is nonetheless supported by Rust's coherence rules. + + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + + +## Rationale: `TransmuteFrom`/`TransmuteInto` + +### Why support arbitrary transmutation? +Some [prior art][prior-art], especially in the crate ecosystem, provides an API that [only supports transmutations involving particular types](#Source-and-Destination-Types-Supported) (e.g., from/into bytes). As we discuss in the [prior art][prior-art] section, we believe that the inflexibility of such approaches make them a poor basis of a language proposal. In particular, these restrictive approaches don't leave room to grow: supporting additional transmutations requires additional traits. + +The API advocated by this proposal is unopinionated about what transmutations users might wish to do, and what transmutations the compiler is able to reason about. The implementation of this RFC may be initially very simple (and perhaps support no more than the restrictive approaches allow for), but then subsequently grow in sophistication—*without* necessitating public API changes. + +### Why *two* traits? +If `TransmuteInto` is implemented in terms of `TransmuteFrom`, why provide it at all? We do so for consistency with libcore's [`From`/`Into`](https://doc.rust-lang.org/stable/rust-by-example/conversion/from_into.html) traits, and because directionality conveys intent: `TransmuteFrom` connotes *conversion*, whereas `TransmuteInto` connotes initialization. We believe that the supporting code examples of this RFC demonstrate the explanatory benefits of providing *both* traits. + +## Rationale: Transmutation Options + +### Granularity +Although the focus of our API is statically-correct, infalible transmutations, the ability to opt-out of particular static checks is essential for building safer *fallible* mechanisms, such as alignment-fallible [reference casting][ext-ref-casting], or validity-fallible transmutations (e.g., `bool` to `u8`). + +### Representation +Although transmutations options exist at a type-level, they're represented as type-level tuples, whose familiar syntax is identical to value-level tuples. An empty tuple seems like the natural choice for encoding *don't neglect anything*. + +We could not identify any advantages to representing options with const-generics. There is no clear syntactic advantage: tuples remain the most natural way to encode ad-hoc products of items. The comparative lack of default values for const-generic parameters poses an ergonomic *disadvantage*. + + +## Rationale: Stability + +### Why do we need a stability system? + +At least two requirements necessitate the presence of a stability system: + +#### Mitigating a Unique Stability Hazard +The usual rules of SemVer stability dictate that if a trait is is implemented in a version `m.a.b`, it will *continue* to be implemented for all versions `m.x.y`, where `x ≥ a` and `y ≥ b`. **`TransmuteFrom` is the exception to this rule**. It would be irresponsible to do nothing to mitigate this stability hazard. + +The compromise made by this RFC is that **`TransmuteFrom` should be stable-by-default**: + - `Dst: TransmuteFrom` follows the usual SemVer rules, + - `Dst: TransmuteFrom` *does not*. + +#### Mitigating a Possible Safety Hazard +The [simplified formulation of constructability](#simplifying-constructability) provides an initially-simpler implementation path at the cost of a soundness hole. There are three possible mitigations: + - *Pretend it does not exist.* Intentional soundness holes would not bode well for this RFC's acceptance. + - *Only provide unsafe transmutation; not safe transmutation.* This option fails to remove any `unsafe` blocks from end-users' code. + - *Allow safe transmutations only when the type authors have promised they will not create a situation that would compromise safety.* **We recommend this option.** + +This promise is inherently one of stability—the type author is vowing that they will not change the implementation of their type in a way that violates the no-pub-in-priv safety invariant of safe transmutation. + + +### Why this *particular* stability system? +The proposed stability system is both simple, flexible, and extensible. Whereas ensuring the soundness and safety of `TransmuteFrom` requires non-trivial compiler support, stability does not—it is realized as merely two normal traits and an `impl`. + +This formulation is flexible: by writing custom `Archetype`s, the [stability declaration traits][stability] make possible granular and incomplete promises of layout stability (e.g., guaranteeing the size and validity qualities of a type, but *not* its alignment. Members of the safe-transmute working group have [expressed](https://rust-lang.zulipchat.com/#narrow/stream/216762-project-safe-transmute/topic/Transmutability.20Intrinsic/near/202712834) an interest in granular stability declarations. + +Finally, this formulation is extensible. The range of advance use-cases permitted by these traits is constrained only by the set of possible `Archetype`s, which, in turn, is constrained by the completeness of `TransmuteFrom`. As the implementation of `TransmuteFrom` becomes more complete, so too will the range of advance use-cases accommodated by these traits. + + +### Couldn't `#[repr(C)]` denote stability? +In our proposal, `#[repr(C)]` does not connote any promises of transmutation stability for SemVer purposes. It has [been suggested](https://rust-lang.zulipchat.com/#narrow/stream/216762-project-safe-transmute/topic/RFC.3A.20Stability.20Declaration.20Traits/near/204011238) that the presence of `#[repr(C)]` *already* connotes total transmutation stability; i.e., that the type's author promises that the type's size and alignment and bit-validity will remain static. If this is true, then an additional stability mechanism is perhaps superfluous. However, we are unaware of any authoritative documentation indicating that `#[repr(C)]` carries this implication. Treating `#[repr(C)]` as an indicator of transmutation stability [would](https://rust-lang.zulipchat.com/#narrow/stream/216762-project-safe-transmute/topic/typic/near/201165897) thus pose a stability hazard. + + +## Alternative: Implementing this RFC in a Crate + +This RFC builds on ample [prior art][prior-art] in the crate ecosystem, but these efforts strain against the fundamental limitations of crates. Fundamentally, safe transmutation efforts use traits to expose layout information to the type system. The burden of ensuring safety [is usually either placed entirely on the end-user, or assumed by complex, incomplete proc-macro `derives`][mechanism-manual]. + +An exception to this rule is the [typic][crate-typic] crate, which utilizes complex, type-level programming to emulate a compiler-supported, "smart" `TransmuteFrom` trait (like the one proposed in this RFC). Nonetheless, [typic][crate-typic] is fundamentally limited: since Rust does not provide a type-level mechanism for reflecting over the structure of arbitrary types, even [typic][crate-typic] cannot judge the safety of a transmutation without special user-added annotations on type definitions. Although [typic][crate-typic] succeeds as a proof-of-concept, its maintainability is questionable, and the error messages it produces are [lovecraftian](https://en.wikipedia.org/wiki/Lovecraftian_horror). + +The development approaches like [typic][crate-typic]'s could, perhaps, be eased by stabilizing [frunk](https://crates.io/crates/frunk)-like structural reflection, or (better yet) by stabilizing a compiler plugin API for registering "smart" traits like `TransmuteFrom`. However, we suspect that such features would be drastically harder to design and stabilize. + +Regardless of approach, almost all [prior art][prior-art] attempts to reproduce knowledge *already* possessed by `rustc` during the compilation process (i.e., the layout qualities of a concrete type). Emulating the process of layout computation to any degree is an error-prone duplication of effort between `rustc` and the crate, in a domain where correctness is crucial. + +Finally, community-led, crate-based approaches are, inescapably, unauthoritative. These approaches are incapable of fulfilling our motivating goal of providing a *standard* mechanism for programmers to statically ensure that a transmutation is safe, sound, or stable. + +# Prior art +[prior-art]: #prior-art + +[crate-plain]: https://crates.io/crates/plain +[crate-bytemuck]: https://crates.io/crates/bytemuck +[crate-dataview]: https://crates.io/crates/dataview +[crate-safe-transmute]: https://crates.io/crates/safe-transmute +[crate-pod]: https://crates.io/crates/pod +[crate-uncon]: https://crates.io/crates/uncon +[crate-typic]: https://crates.io/crates/typic +[crate-zerocopy]: https://crates.io/crates/zerocopy +[crate-convute]: https://crates.io/crates/convute +[crate-byterepr]: https://crates.io/crates/byterepr + +[2017-02]: https://internals.rust-lang.org/t/pre-rfc-safe-coercions/4823 +[2018-03]: https://internals.rust-lang.org/t/pre-rfc-frombits-intobits/7071 +[2018-03-18]: https://internals.rust-lang.org/t/pre-rfc-frombits-intobits/7071/23 +[2018-05-18]: https://internals.rust-lang.org/t/pre-rfc-trait-for-deserializing-untrusted-input/7519 +[2018-05-23]: https://github.com/joshlf/rfcs/blob/joshlf/from-bytes/text/0000-from-bytes.md +[2019-09]: https://internals.rust-lang.org/t/specifying-a-set-of-transmutes-from-struct-t-to-struct-u-which-are-not-ub/10917 +[2019-11]: https://internals.rust-lang.org/t/pre-rfc-safe-transmute/11347 +[2019-12-05-gnzlbg]: https://gist.github.com/gnzlbg/4ee5a49cc3053d8d20fddb04bc546000 +[2019-12-05-v2]: https://internals.rust-lang.org/t/pre-rfc-v2-safe-transmute/11431 +[2020-07]: https://internals.rust-lang.org/t/pre-rfc-explicit-opt-in-oibit-for-truly-pod-data-and-safe-transmutes/2361 + +## Prior Art: Rust +[prior-art-rust]: #prior-art-rust + +A handful of dimensions of variation characterize the distinctions between prior art in Rust: + - conversion complexity + - conversion fallibility + - source and destination types supported + - implementation mechanism + - stability hazards + +We review each of these dimensions in turn, along with this proposal's location along these dimensions: + +### Conversion Complexity +Prior work differs in whether it supports complex conversions, or only simple transmutation. [*Pre-RFC FromBits/IntoBits*][2018-03]'s proposed traits include conversion methods that are implemented by type authors. Because end-users provide their own definitions of these methods, they can be defined to do more than just transmutation (e.g., slice casting). (This approach is similar to the [uncon][crate-uncon] crate's [`FromUnchecked`](https://docs.rs/uncon/1.*/uncon/trait.FromUnchecked.html) and [`IntoUnchecked`](https://docs.rs/uncon/1.*/uncon/trait.IntoUnchecked.html) traits, which provide unsafe conversions between types. These traits are safe to implement, but their conversion methods are not.) + +In contrast, our RFC focuses only on transmutation. Our `TransmutableFrom` and `TransmutableInto` traits serve as both a marker *and* a mechanism: if `Dst: TransmuteFrom`, it is sound to transmute from `Dst` into `Src` using `mem::transmute`. However, these traits *also* provide transmutation methods that are guaranteed to compile into nothing more complex than a `memcpy`. These methods cannot be overridden by end-users to implement more complex behavior. + +The signal and transmutability and mechanism are, in principle, separable. The [convute][crate-convute] crate's [`Transmute`](https://docs.rs/convute/0.2.0/convute/marker/trait.Transmute.html) trait is an unsafe marker trait representing types that can be transmuted into `T`. This is *just* a marker trait; the actual conversion mechanisms are provided by a [separate suite](https://docs.rs/convute/0.2.0/convute/convert/index.html) of traits and functions. Our RFC combines marker with mechanism because we feel that separating these aspects introduces additional complexity with little added value. + +### Conversion Fallibility +Prior work differs in whether it supports only infallible conversions, or fallible conversions, too. The [convute][crate-convute] crate's [`TryTransmute`](https://docs.rs/convute/0.2.0/convute/marker/trait.TryTransmute.html) trait provides a method, `can_transmute`, that returns true a transmutation from `Self` to `T` is valid for a particular value of `&self`. An early version of [typic][crate-typic] abstracted a similar mechanism into an [`Invariants`](https://docs.rs/typic/0.1.0/typic/transmute/trait.Invariants.html) trait, with additional facilities for error reporting. [*Draft-RFC: `Compatible`/`TryCompatible`*][2019-12-05-gnzlbg] employs a similar mechanism to typic. + +Typic removed support for fallible transmutation after reckoning with several challenges: +- The causes of uncertain failure could be language-imposed (e.g., alignment or validity requirements), or library imposed (i.e., invariants placed on a structure's private fields). +- The points of uncertain failures could be arbitrarily 'deep' into the fields of a type. +- Error reporting incurs a runtime cost commensurate with the detail of the reporting, but the detail of reporting required by end-user depends on use-case, not just type. For instance: for some use-cases it may be necessary to know where and why a byte was not a valid `bool`; in others it may be sufficient to know simply *whether* an error occurred. + +Finally, we observed that the mechanisms of fallible transmutation were basically separable from the mechanisms of infallible transmutation, and thus these challenges could be addressed at a later date. For these reasons, our RFC *only* addresses infallible transmutation. + +While this RFC does not provide a grand, all-encompassing mechanism for fallible transmutation, the fundamental mechanisms of our RFC are useful for constructing safer, purpose-built fallible conversion mechanisms; e.g.: +```rust +/// Try to convert a `&T` into `&U`. +/// +/// This produces `None` if the referent isn't appropriately +/// aligned, as required by the destination type. +pub fn try_cast_ref<'t, 'u, T, U>(src: &'t T) -> Option<&'u U> +where + &'t T: TransmuteInto<&'u U, NeglectAlignment>, +{ + if (src as *const T as usize) % align_of::() != 0 { + None + } else { + // Safe because we dynamically enforce the alignment + // requirement, whose static check we chose to neglect. + Some(unsafe { src.unsafe_transmute_into() }) + } +} +``` +In this approach, our RFC is joined by crates such as [plain](https://docs.rs/plain/0.2.3/plain/#functions), [bytemuck](https://docs.rs/bytemuck/1.*/bytemuck/#functions), [dataview](https://docs.rs/dataview/0.1.1/dataview/struct.DataView.html#methods), [safe-transmute](https://docs.rs/safe-transmute/0.11.0/safe_transmute/fn.transmute_one.html), [zerocopy](https://docs.rs/zerocopy/0.3.0/zerocopy/struct.LayoutVerified.html#methods), and [byterepr](https://docs.rs/byterepr/0.1.0/byterepr/trait.ByteRepr.html#provided-methods), and several pre-RFCs (such as [this][2018-05-18] and [this](https://github.com/joshlf/rfcs/blob/joshlf/from-bits/text/0000-from-bits.md#library-functions)). The ubiquity of these mechanisms makes a strong case for their inclusion in libcore. + +### Source and Destination Types Supported +Prior work differs in whether its API surface is flexible enough to support transmutation between arbitrary types, or something less. + +#### Arbitrary Types +Approaches supporting transmutations between arbitrary types invariably define traits akin to: +```rust +/// Indicates that `Self` may be transmuted into `Dst`. +pub unsafe trait TransmuteInto +{ ... } + +/// Indicates that `Self` may be transmuted from `Dst`. +pub unsafe trait TransmuteFrom +{ ... } +``` +This approach, taken by our RFC, is used by at least two crates: +- The [convute][crate-convute] crate's [`Transmute`](https://docs.rs/convute/0.2.0/convute/marker/trait.Transmute.html) trait is akin to the above definition of `TransmuteInto`. +- The [typic][crate-typic] crate's [`TransmuteInto`](https://docs.rs/typic/0.3.0/typic/transmute/trait.TransmuteInto.html) and [`TransmuteFrom`](https://docs.rs/typic/0.3.0/typic/transmute/trait.TransmuteFrom.html) traits almost exactly mirror the above definitions. + +...and several proposals: +- [*Pre-RFC: Safe coercions*][2017-02] proposes a `Coercible` trait that is implemented if `A` is safely transmutable into `B`. +- [*Pre-RFC: `FromBits`/`IntoBits`*][2018-03] proposes the traits `IntoBits` and `FromBits.` +- [*Draft-RFC: `FromBytes`*][2018-05-23] proposes the traits `IntoBytes` and `FromBytes.` +- [*Draft-RFC: `Compatible`/`TryCompatible`*][2019-12-05-gnzlbg] proposes the trait `Compatible`, akin to the above definition of `TransmuteInto`. + +##### From/Into Bytes Transmutations +Other approaches adopt an API that only supports transmutation of a type into initialized bytes, and from initialized bytes. These approaches invariably define traits akin to: +```rust +/// Indicates that a type may be transmuted into an appropriately-sized array of bytes. +pub unsafe trait IntoBytes +{} + +/// Indicates that a type may be transmuted from an appropriately-sized array of bytes. +pub unsafe trait FromBytes +{} +``` +This is the approach taken by the [zerocopy][crate-zerocopy] crate, and the [*Pre-RFC: Safe Transmute*][2019-11] and [*Pre-RFC: Safe Transmute v2*][2019-12-05-v2] proposals. + +This approach is strictly less flexible than an API supporting transmutation between arbitrary types. It is incapable of representing transmutations of bytes into types with validity constraints, and incapable of representing transmutations of types with padding bytes into bytes. + +Supporting additional transmutation source and destination types requires a commensurate addition of conversion traits. For instance, some of [zerocopy][crate-zerocopy]'s users [require](https://fuchsia-review.googlesource.com/c/fuchsia/+/306036/2#message-a1a0c9cf16e3dec24e7b0548e3c09382f63783f0) a trait that reflects types which can be transmuted from a buffer of zeroed bytes. This would require introducing an additional trait, `FromZeros`. + +An advantage of this API is that it gives descriptive names to perhaps the two most common transmutations. However, an API providing transmutation between arbitrary types can encode `FromBytes` and `IntoBytes`: +```rust +// `Dst` is `FromBytes` if it can be safely transmuted *from* an +// equivalently sized array of `u8`. +unsafe impl FromBytes for Dst +where + Dst: TransmuteFrom<[u8; size_of::()]>, +{} + +// `Src` is `IntoBytes` if it can be safely transmuted *into* an +// equivalently sized array of `u8`. +unsafe impl IntoBytes for Src +where + Src: TransmuteInto<[u8; size_of::()]>, +{} +``` +For these reasons, we argue that a `FromBytes`/`ToBytes` style API is a poor foundation for in-language safe transmutation. + +##### Bytes-to-Bytes Transmutations (aka "Plain Old Data") +Finally, many approaches (especially crates) supply a marker trait that represents "plain old data"; e.g.: +```rust +/// Implemented by types that are "plain old data": +pub unsafe trait PlainOldData +{} +``` +This sort of trait is present in crates such as [plain](https://docs.rs/plain/0.2.3/plain/trait.Plain.html), [bytemuck](https://docs.rs/bytemuck/1.*/bytemuck/trait.Pod.html), [dataview](https://docs.rs/dataview/0.1.1/dataview/trait.Pod.html), [safe-transmute](https://docs.rs/safe-transmute/0.11.0/safe_transmute/trivial/trait.TriviallyTransmutable.html), and [pod](https://docs.rs/pod/0.5.0/pod/trait.Pod.html), and at least two language proposals ([here][2018-05-18] and [here][2020-07]). + +The exact definition of what constitutes "plain old data" varies between crates. One simple definition is that a type `T` is "plain old data" if it can be transmuted both from and into initialized bytes; i.e.: +```rust +unsafe impl PlainOldData for T +where + T: FromBytes + IntoBytes, +{} +``` + +This definition precludes useful transmutations. For instance, `MaybeUninit` is transmutable from a `u8`, but not *into* a `u8`. + +Given this inflexibility, we argue that this approach is a poor foundation for in-language safe transmutation. + + +### Implementation Mechanism +Not only does prior work differ in which traits are used to encode valid transmutations, they differ in the level of user intervention required to take advantage of the traits. + +#### Manual +[mechanism-manual]: #Manual +Fully manual approaches require type authors to implement the transmutation traits manually. The involved traits are `unsafe`, so it is up to type authors to verify for themselves that their hand-written implementations are sound. This is the approach taken by crates such as [plain][crate-plain], [bytemuck][crate-bytemuck], [safe-transmute][crate-safe-transmute], and [pod][crate-pod], and at least one language proposal: [*Pre-RFC: Safe Transmute*][2019-12-05-v2] (which advocates for a "plain old data" API). + +In semi-manual approaches, type authors simply `derive` the applicable traits, using `derive` macros that produce a compile-error if the implementation is not sound. This approach is realized by crates such as ([zerocopy](https://docs.rs/zerocopy/0.3.0/zerocopy/#derives), [zeroable](https://docs.rs/zeroable/0.2.0/zeroable/) and [dataview](https://docs.rs/dataview/0.1.1/dataview/derive.Pod.html)) and advocated by at least two language proposals: [*Pre-RFC: Safe Transmute v2*][2019-12-05-v2] (which advocates for a `FromBytes`/`IntoBytes`-style API), and [*Pre-RFC FromBits/IntoBits*][2018-03] (which advocates for a general-transmutation API). + +We believe that the implementation burden these approaches place on end-users, and their inflexibility, make them a poor foundation for in-language safe transmutation: +- These approaches require authors to implement and, potentially, verify a large number of `unsafe` traits, ranging from *O(n)* implementations for plain-old-data trait approaches, to potentially [*many* more](https://internals.rust-lang.org/t/pre-rfc-frombits-intobits/7071/28). +- These approaches are generally impractical for APIs that permit truly general transmutation, as type authors can only construct implementations of the transmutation traits for types they have at their disposal. +- These approaches conflate transmutation stability with transmutation safety. An end-user wishing to transmute a type for which its author has *not* manually implemented the applicable traits must resort to the wildly unsafe `mem::transmute`. + + +#### Automatic +Automatic approaches implement the transmutation traits without user intervention, whenever it is sound to do so. This is the approach taken by our RFC. Automatic mechanisms appear in at least four prior language proposals: +- [*Pre-RFC: Safe coercions*][2017-02] +- [*Draft-RFC: `from_bytes`*][2018-05-23] +- [*Pre-RFC: Trait for deserializing untrusted input*][2018-05-18] +- [*Draft-RFC: `compatible_trait`*][2019-12-05-gnzlbg] + +The [typic][crate-typic] crate mocks a fully-automatic approach: its `TransmuteFrom` trait is usable with any types that are `repr(C)`, or otherwise have a well-defined memory layout. (In practice, since Rust lacks reflection over type definitions, `repr(C)` annotations much be changed to `typic::repr(C)`.) + +### Stability Hazards +Fully automatic approaches introduce, at the very least, a stability hazard: they supply a safe constructor for types, without the consent of those types' authors. If a type author hid the internals of their type because they do not wish for its implementation details to become a part of the type's API for SemVer purposes, an automatic transmutation mechanism subverts that intent. + +No attempt to avoid this hazard is made by most of the proposals featuring automatic mechanisms; e.g.: +- [*Draft-RFC: `from_bytes`*][2018-05-23] +- [*Pre-RFC: Trait for deserializing untrusted input*][2018-05-18] +- [*Draft-RFC: `compatible_trait`*][2019-12-05-gnzlbg] + +#### Hazard-Avoidant +The automatic mechanism proposed by [*Pre-RFC: Safe coercions*][2017-02] exploits field visibility, requiring that all fields that have different types in `Src` and `Dst` are visible at the location where the coercion is made. This approach falls short in three respects: +1. Confining the visibility requirement only to fields of *different* types is insufficient; two different types with identical field types may subject those fields to different invariants. +2. The 'location' where the coercion is made is ill-defined; the presence of the proposed `Coercible` trait may be far-removed from the location of the actual conversion (if any conversion occurs at all). +3. Field visibility stabilizes the structure of a type, but *not* its layout (e.e., its size). + +Our RFC exploits the related concept of *constructability*, which is a property of a struct, or enum variant (rather than solely a property of fields). However, we recognize that it may be difficult to test for constructability within the trait resolution process. + +The simplified definition of *constructability* we propose is the same employed by [typic][crate-typic] (which uses the term "visibility"). [Typic][crate-typic] regards the pub-in-priv soundness hole of the simplified definition to be sufficiently niche that `NeglectStability` remains "safe". However, unlike [typic][crate-typic], we believe that this simplified definition imposes a safety hazard substantial enough to warrant making `NeglectStability` initially usable with *only* unsafe transmutes. + +Our RFC separates *constructability*, which concerns what aspects of a type's structure are part of its public API, and *stability*, which concerns the aspects of a type's layout that are part of its public API for SemVer purposes. This distinction does not appear in prior work. + + +## Prior Art: Haskell + +Haskell's [`Coercible`](https://hackage.haskell.org/package/base-4.14.0.0/docs/Data-Coerce.html#t:Coercible) typeclass is implemented for all types `A` and `B` when the compiler can infer that they have the same representation. As with our proposal's `TransmuteFrom` trait, instances of this typeclass are created "on-the-fly" by the compiler. `Coercible` primarily provides a safe means to convert to-and-from newtypes, and does not seek to answer, for instance, if two `u8`s are interchangeable with a `u16`. + +Haskell takes an algebraic approach to this problem, reasoning at the level of type definitions, not type layouts. However, not all type parameters have an impact on1 a type's layout; for instance: +```rust +#[repr(C)] +struct Bar(PhantomData); + +#[repr(transparent)] +struct Foo(T, Bar); +``` +`Foo`'s layout is impacted solely by `T`, not `U`, but this isn't necessarily clear by looking at the definition of `Foo`. To reason about these scenarios, Haskell introduces the concept of type parameter [*roles*](https://gitlab.haskell.org/ghc/ghc/-/wikis/roles)—labels that denote the relationship of a type parameter to coercibility. + +Our RFC does not need the concept of roles, because it does not attempt to abstractly reason about type definitions. Rather, it reasons about type *layouts*. This example, for instance, does not pose a challenge to our proposal: +```rust +trait SomeTrait { type AssociatedType; } + +#[repr(C)] +struct MyStruct(pub T, pub T::AssociatedType); +``` +For a *particular* `T`, `MyStruct` will have a *particular* layout. Our proposed `TransmuteFrom` trait reasons about the +*layouts* of types (which are fully concrete), not the *definitions* (which may be somewhat abstract). + + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +### Questions To Be Resolved Before RFC Acceptance +The following unresolved questions should be resolved during the RFC process: + +##### Unhandled Use-Cases? +We endeavored to design an API surface with ([nearly][drawbacks]) zero compromises. However, if you have a use-case that you believe is neither satisfied outright by our proposal, nor [aided][future-possibilities] by our proposal, we would *urgently* like to hear of it. + +##### Extensions for Inclusion? +In [*Future Possibilities*][future-possibilities], we propose a number of additional abstractions that are aided by this RFC. Some of these abstractions are commonplace in [prior art][prior-art] and should perhaps be included with this RFC. Some of our proposed extensions could begin their crates that work on stable Rust; others, such as [generic atomics][future-possibility-generic-atomics], require nightly-only intrinsics. + +### Questions To Be Resolved Before Feature Stabilization +The following unresolved questions should be resolved before feature stabilization: + +##### Layout-Stability for Unsafe Transmutations? +We [observe][drawback-unsafe-stability] that our proposed model for stability declaration, although very expressive, does not permit type authors to declare the stability of *unsafe* transmutations. Alongside that observation, we suggest a [SemVer-compatible](https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md#minor-change-adding-a-defaulted-type-parameter) upgrade of the stability declaration traits that may resolve this shortcoming. + +While it is unclear if there is any demand for this degree of flexibility, this upgrade-path should be carefully considered *before* stabilizing (and thus committing) to this RFC's layout stability declaration traits. + + +### Questions Out of Scope +We consider the following unresolved questions to be out-of-scope of *this* RFC process: + +##### Design of `NeglectConstructability`? +`TransmuteFrom` and `TransmuteInto` require that the destination type has a matching constructor in which all fields are marked `pub`. Conspicuously *missing* from this RFC is a `NeglectConstructability` unsafe option to disable this check. + +The omission is intentional. The consequences of such an option are suprising in both their subtlety and their unsafety. Some of unsafe Rust's hairiest interactions lie at the intersections of `!Send`, `!Sync`, `UnsafeCell` and restricted field visibility. These building blocks are used to build safe, public abstractions that encapsulate unsafe, hidden internals. + +# Future possibilities +[future-possibilities]: #future-possibilities + +## Extension: `PromiseTransmutable` Shorthand +[extension-promisetransmutable-shorthand]: #extension-promisetransmutable-shorthand +[0000-ext-promise-transmutable.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-promise-transmutable.md + +See [here][0000-ext-promise-transmutable.md]. + +## Extension: Layout Property Traits +[0000-ext-layout-traits.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-layout-traits.md + +See [here][0000-ext-layout-traits.md]. + +## Extension: Byte Transmutation Traits and Safe Initialization +[extension-zerocopy]: #extension-byte-transmutation-traits-and-safe-initialization +[0000-ext-byte-transmutation.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-byte-transmutation.md + +See [here][0000-ext-byte-transmutation.md]. + + +## Extension: Slice and `Vec` Casting +[ext-slice-casting]: #extension-slice-and-vec-casting +[ext-vec-casting]: #extension-slice-and-vec-casting +[0000-ext-container-casting.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-container-casting.md + +See [here][0000-ext-container-casting.md]. + + +## Extension: `include_data!` +[future-possibility-include_data]: #Extension-include_data +[0000-ext-include-data.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-include-data.md + +See [here][0000-ext-include-data.md]. + + +## Extension: Generic Atomics +[future-possibility-generic-atomics]: #extension-generic-atomics +[0000-ext-generic-atomic.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-generic-atomic.md + +See [here][0000-ext-generic-atomic.md]. From 72e24314a0acc2495e5e82121ce80c1b531f5fa6 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 31 Aug 2020 20:04:04 -0400 Subject: [PATCH 02/18] update rfc PR# --- text/0000-safer-transmute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index f6d11b2d8a4..2138b16f252 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -2,7 +2,7 @@ - Feature Name: `safer_transmute` - Start Date: 2020-08-31 -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/2981) - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) From 9548ed724be46ef3d72b060a12038934c3ccb7e8 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 31 Aug 2020 20:04:30 -0400 Subject: [PATCH 03/18] update rfc PR# --- text/0000-safer-transmute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 2138b16f252..d356e4a3ba1 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -2,7 +2,7 @@ - Feature Name: `safer_transmute` - Start Date: 2020-08-31 -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/2981) +- RFC PR: [rust-lang/rfcs#2981](https://github.com/rust-lang/rfcs/pull/2981) - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) From 1fb17cbc0bb315cb66d5d8cf8854f379c7de2e02 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Thu, 3 Sep 2020 16:50:53 -0400 Subject: [PATCH 04/18] describe minimal-but-useful stable API, and further stabilization milestones --- text/0000-safer-transmute.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index d356e4a3ba1..3657976c7cf 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -1320,6 +1320,34 @@ We recommend that that implementers of this RFC initially simplify constructabil If and when the implementation of `TransmuteFrom` encodes our complete definition of constructability, `NeglectStability` shall become a safe transmute option. +### Minimal Useful Stabilization Surface + +Stabilizing *only* these three items of the Initial Smart Implementation will cover most use-cases: + - `TransmuteFrom` + - `#[derive(PromiseTransmutableFrom)]` + - `#[derive(PromiseTransmutableInto)]` + +(If the [`PromiseTransmutable` shorthand extension](extension-promisetransmutable-shorthand) is accepted, this may be further reduced to just *two* items: `TransmuteFrom` and `#[derive(PromiseTransmutable)]`.) + +Stabilizing `TransmuteOptions` and `SafeTransmuteOptions` will additionally allow end-users to build generic abstractions over `TransmuteFrom` (e.g., slice casting abstractions). + +Stabilizing `PromiseTransmutableFrom` and `PromiseTransmutableInto` will additionally allow end-users make limited stability promises, and to make stability promises for types where `derive` is too restrictive (namely, types containing `PhantomData`). + +Stabilizing `TransmuteInto` and `NeglectStability` will additionally allow end-users to implement `PromiseTransmutableFrom` in cases where the `Archetype`'s trait bounds must be repeated for lack of [implied bounds](https://github.com/rust-lang/rust/issues/44491); e.g.: +```rust +impl PromiseTransmutableFrom for GenericArray +where + T: PromiseTransmutableFrom, + N: ArrayLength, + + // for lack of implied bounds, we must repeat the bounds on `Archetype`: + GenericArray + : TransmuteInto + + PromiseTransmutableFrom, +{ + type Archetype = GenericArray; +} +``` # Drawbacks [drawbacks]: #drawbacks From 4f0ec015bd5e782ea1dc3b9d57eb75bf673704bc Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Sat, 5 Sep 2020 09:32:52 -0400 Subject: [PATCH 05/18] fix dead intradocument link --- text/0000-safer-transmute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 3657976c7cf..27d81eb573a 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -1327,7 +1327,7 @@ Stabilizing *only* these three items of the Initial Smart Implementation will co - `#[derive(PromiseTransmutableFrom)]` - `#[derive(PromiseTransmutableInto)]` -(If the [`PromiseTransmutable` shorthand extension](extension-promisetransmutable-shorthand) is accepted, this may be further reduced to just *two* items: `TransmuteFrom` and `#[derive(PromiseTransmutable)]`.) +(If the [`PromiseTransmutable` shorthand extension][extension-promisetransmutable-shorthand] is accepted, this may be further reduced to just *two* items: `TransmuteFrom` and `#[derive(PromiseTransmutable)]`.) Stabilizing `TransmuteOptions` and `SafeTransmuteOptions` will additionally allow end-users to build generic abstractions over `TransmuteFrom` (e.g., slice casting abstractions). From 7646f4890281e070974605d7c63da8f56ff3d251 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Wed, 9 Sep 2020 14:00:41 -0400 Subject: [PATCH 06/18] Remove 'Compromises' column of table in 'Neglecting Static Checks'. https://github.com/rust-lang/rfcs/pull/2981#discussion_r484088700 --- text/0000-safer-transmute.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 27d81eb573a..ac337d9bff6 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -640,11 +640,11 @@ In the above definitions, `Src` represents the source type of the transmutation, The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are unsafe, unsound, or unstable. However, you may explicitly opt-out of some static checks: -| Transmute Option | Compromises | Usable With | -|---------------------|-------------|---------------------------------------------------------| -| `NeglectStabilty` | Stability | `transmute_{from,into}`, `unsafe_transmute_{from,into}` | -| `NeglectAlignment` | Safety | `unsafe_transmute_{from,into}` | -| `NeglectValidity` | Soundness | `unsafe_transmute_{from,into}` | +| Transmute Option | Usable With | +|---------------------|---------------------------------------------------------| +| `NeglectStabilty` | `transmute_{from,into}`, `unsafe_transmute_{from,into}` | +| `NeglectAlignment` | `unsafe_transmute_{from,into}` | +| `NeglectValidity` | `unsafe_transmute_{from,into}` | `NeglectStabilty` implements the `SafeTransmuteOptions` and `TransmuteOptions` marker traits, as it can be used in both safe and unsafe code. The selection of multiple options is encoded by grouping them as a tuple; e.g., `(NeglectAlignment, NeglectValidity)` is a selection of both the `NeglectAlignment` and `NeglectValidity` options. From 01d1f9ce9edb30fcb4d829db2e7fdb4ea69b646d Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Wed, 9 Sep 2020 14:11:25 -0400 Subject: [PATCH 07/18] 'soundness' -> 'well-definedness' --- text/0000-safer-transmute.md | 44 ++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index ac337d9bff6..665b5773563 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -98,42 +98,42 @@ unsafe fn transmute(src: Src) -> Dst ``` ### 📖 Safer Transmutation -By **safer transmutation** we mean: *what `where` bound could we add to `transmute` restricts its type parameters `Src` and `Dst` in ways that statically limit the function's misuse?* Our answer to this question will ensure that transmutations are, by default, *sound*, *safe* and *stable*. +By **safer transmutation** we mean: *what `where` bound could we add to `transmute` restricts its type parameters `Src` and `Dst` in ways that statically limit the function's misuse?* Our answer to this question will ensure that transmutations are, by default, *well-defined*, *safe* and *stable*. -### 📖 Soundness -A transmutation is ***sound*** if the mere act of transmuting a value from one type to another is not unspecified or undefined behavior. +### 📖 Well-Definedness +A transmutation is ***well-defined*** if the mere act of transmuting a value from one type to another is not unspecified or undefined behavior. ### 📖 Safety -A sound transmutation is ***safe*** if *using* the transmuted value cannot violate memory safety. +A well-defined transmutation is ***safe*** if *using* the transmuted value cannot violate memory safety. ### 📖 Stability A safe transmutation is ***stable*** if the authors of the source type and destination types have indicated that the layouts of those types is part of their libraries' stability guarantees. ## Concepts in Depth -***Disclaimer:** While the high-level definitions of transmutation soundness, safety and stability are a core component of this RFC, the detailed rules and examples in this section are **not**. We expect that the initial implementation of `TransmuteFrom` may initially be considerably less sophisticated than the examples in this section (and thus forbid valid transmutations). Nonetheless, this section explores nuanced cases of transmutation soundness and safety to demonstrate that the APIs we propose can grow to handle that nuance.* +***Disclaimer:** While the high-level definitions of transmutation well-definedness, safety and stability are a core component of this RFC, the detailed rules and examples in this section are **not**. We expect that the initial implementation of `TransmuteFrom` may initially be considerably less sophisticated than the examples in this section (and thus forbid valid transmutations). Nonetheless, this section explores nuanced cases of transmutation well-definedness and safety to demonstrate that the APIs we propose can grow to handle that nuance.* -### 📖 When is a transmutation sound? -[sound transmutation]: #-when-is-a-transmutation-sound -A transmutation is ***sound*** if the mere act of transmuting a value from one type to another is not unspecified or undefined behavior. +### 📖 When is a transmutation well-defined? +[sound transmutation]: #-when-is-a-transmutation-well-defined +A transmutation is ***well-defined*** if the mere act of transmuting a value from one type to another is not unspecified or undefined behavior. #### Well-Defined Representation [`u8`]: core::u8 [`f32`]: core::f32 -Transmutation is *always unsound* if it occurs between types with unspecified representations. +Transmutation is ill-defined if it occurs between types with unspecified representations. Most of Rust's primitive types have specified representations. That is, the precise layout characteristics of [`u8`], [`f32`] is a documented and guaranteed aspect of those types. In contrast, most `struct` and `enum` types defined without an explicit `#[repr(C)]` attribute do ***not*** have well-specified layout characteristics. -To ensure that types you've define are soundly transmutable, you almost always (with very few exceptions) must mark them with the `#[repr(C)]` attribute. +To ensure that types you've define are transmutable, you almost always (with very few exceptions) must mark them with the `#[repr(C)]` attribute. #### Requirements on Owned Values [transmute-owned]: #requirements-on-owned-values -Transmutations involving owned values must adhere to two rules to be sound. They must: +Transmutations involving owned values must adhere to two rules to be well-defined. They must: * [preserve or broaden the bit validity][owned-validity], and * [preserve or shrink the size][owned-size]. @@ -193,7 +193,7 @@ let _ : Packed = Padded::default().transmute_into(); // Compile Error! ##### Preserve or Shrink Size [owned-size]: #Preserve-or-Shrink-Size -It's completely sound to transmute into a type with fewer bytes than the source type; e.g.: +It's well-defined to transmute into a type with fewer bytes than the source type; e.g.: ```rust let _ : [u8; 16] = [u8; 32]::default().transmute_into(); ``` @@ -208,9 +208,9 @@ let _ : [u8; 32] = [u8; 16]::default().transmute_into(); // Compile Error! #### Requirements on References [transmute-references]: #requirements-on-references -The [restrictions above that apply to transmuting owned values][transmute-owned] also apply to transmuting references. However, references carry a few additional restrictions. +The [restrictions above that apply to transmuting owned values][transmute-owned] also apply to transmuting references. However, references carry a few *additional* restrictions. -A [sound transmutation] must: +A [well-defined transmutation] of references must: - [preserve or shrink size][reference-size], - [preserve or relax alignment][reference-alignment], - [preserve or shrink lifetimes][reference-lifetimes], @@ -301,7 +301,7 @@ If this example did not produce a compile error, the value of `z` would not be a ### 📖 When is a transmutation safe? -A sound transmutation is ***safe*** if *using* the transmuted value safely cannot violate memory safety. Whereas soundness solely concerns the act of transmutation, *safety* is concerned with what might happen with a value *after* transmutation occurs. +A well-defined transmutation is ***safe*** if *using* the transmuted value safely cannot violate memory safety. Whereas well-definedness solely concerns the act of transmutation, *safety* is concerned with what might happen with a value *after* transmutation occurs. #### Implicit Constructability A struct or enum variant is *fully implicitly constructable* at a given location only if, at that location, that type can be instantiated via its *implicit constructor*, and its fields are also *implicitly constructable*. @@ -409,7 +409,7 @@ If this example did not produce a compile error, the value of `z` would not be a ### 📖 When is a transmutation stable? [stability]: #-when-is-a-transmutation-stable -Since the soundness and safety of a transmutation is affected by the layouts of the source and destination types, changes to those types' layouts may cause code which previously compiled to produce errors. In other words, transmutation causes a type's layout to become part of that type's API for the purposes of SemVer stability. +Since the well-definedness and safety of a transmutation is affected by the layouts of the source and destination types, changes to those types' layouts may cause code which previously compiled to produce errors. In other words, transmutation causes a type's layout to become part of that type's API for the purposes of SemVer stability. The question is, then: *how can the author of a type reason about transmutations they did not write, from-or-to types they did not write?* We address this problem by introducing two traits which both allow an author to opt-in to stability guarantees for their types, and allow third-parties to reason at compile-time about what guarantees are provided for such types. @@ -638,7 +638,7 @@ In the above definitions, `Src` represents the source type of the transmutation, ### Neglecting Static Checks [options]: #Neglecting-Static-Checks -The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are unsafe, unsound, or unstable. However, you may explicitly opt-out of some static checks: +The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are unsafe, ill-defined, or unstable. However, you may explicitly opt-out of some static checks: | Transmute Option | Usable With | |---------------------|---------------------------------------------------------| @@ -682,7 +682,7 @@ where } ``` -Neglecting stability over-eagerly cannot cause unsoundness or unsafety. For this reason, it is the only transmutation option available on the safe methods `transmute_from` and `transmute_into`. However, neglecting stability over-eagerly may cause your code to cease compiling if the authors of the source and destination types make changes that affect their layout. +Neglecting stability over-eagerly cannot cause ill-definedness or unsafety. For this reason, it is the only transmutation option available on the safe methods `transmute_from` and `transmute_into`. However, neglecting stability over-eagerly may cause your code to cease compiling if the authors of the source and destination types make changes that affect their layout. By using the `NeglectStability` option to transmute types you do not own, you are committing to ensure that your reliance on these types' layouts is consistent with their documented stability guarantees. @@ -859,7 +859,7 @@ pub mod transmute { } /// `Self: TransmuteInto where @@ -910,7 +910,7 @@ pub mod transmute { } /// `Self: TransmuteInto where @@ -1496,7 +1496,7 @@ The development approaches like [typic][crate-typic]'s could, perhaps, be eased Regardless of approach, almost all [prior art][prior-art] attempts to reproduce knowledge *already* possessed by `rustc` during the compilation process (i.e., the layout qualities of a concrete type). Emulating the process of layout computation to any degree is an error-prone duplication of effort between `rustc` and the crate, in a domain where correctness is crucial. -Finally, community-led, crate-based approaches are, inescapably, unauthoritative. These approaches are incapable of fulfilling our motivating goal of providing a *standard* mechanism for programmers to statically ensure that a transmutation is safe, sound, or stable. +Finally, community-led, crate-based approaches are, inescapably, unauthoritative. These approaches are incapable of fulfilling our motivating goal of providing a *standard* mechanism for programmers to statically ensure that a transmutation is safe, well-defined, or stable. # Prior art [prior-art]: #prior-art @@ -1538,7 +1538,7 @@ We review each of these dimensions in turn, along with this proposal's location ### Conversion Complexity Prior work differs in whether it supports complex conversions, or only simple transmutation. [*Pre-RFC FromBits/IntoBits*][2018-03]'s proposed traits include conversion methods that are implemented by type authors. Because end-users provide their own definitions of these methods, they can be defined to do more than just transmutation (e.g., slice casting). (This approach is similar to the [uncon][crate-uncon] crate's [`FromUnchecked`](https://docs.rs/uncon/1.*/uncon/trait.FromUnchecked.html) and [`IntoUnchecked`](https://docs.rs/uncon/1.*/uncon/trait.IntoUnchecked.html) traits, which provide unsafe conversions between types. These traits are safe to implement, but their conversion methods are not.) -In contrast, our RFC focuses only on transmutation. Our `TransmutableFrom` and `TransmutableInto` traits serve as both a marker *and* a mechanism: if `Dst: TransmuteFrom`, it is sound to transmute from `Dst` into `Src` using `mem::transmute`. However, these traits *also* provide transmutation methods that are guaranteed to compile into nothing more complex than a `memcpy`. These methods cannot be overridden by end-users to implement more complex behavior. +In contrast, our RFC focuses only on transmutation. Our `TransmutableFrom` and `TransmutableInto` traits serve as both a marker *and* a mechanism: if `Dst: TransmuteFrom`, it is safe to transmute from `Dst` into `Src` using `mem::transmute`. However, these traits *also* provide transmutation methods that are guaranteed to compile into nothing more complex than a `memcpy`. These methods cannot be overridden by end-users to implement more complex behavior. The signal and transmutability and mechanism are, in principle, separable. The [convute][crate-convute] crate's [`Transmute`](https://docs.rs/convute/0.2.0/convute/marker/trait.Transmute.html) trait is an unsafe marker trait representing types that can be transmuted into `T`. This is *just* a marker trait; the actual conversion mechanisms are provided by a [separate suite](https://docs.rs/convute/0.2.0/convute/convert/index.html) of traits and functions. Our RFC combines marker with mechanism because we feel that separating these aspects introduces additional complexity with little added value. From 4bb42ee51ad6562b52b6ccab79c831e0bf336227 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Sat, 12 Sep 2020 20:36:56 -0400 Subject: [PATCH 08/18] revamp constructability Spurred by https://github.com/rust-lang/rfcs/pull/2981#discussion_r487319138 --- text/0000-safer-transmute.md | 818 +++++++++++++++-------------------- 1 file changed, 355 insertions(+), 463 deletions(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 665b5773563..d9bedc57b92 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -12,7 +12,7 @@ We propose traits, namely `TransmuteInto` and `TransmuteFrom`, that are implemented *automatically* for combinations of types that may be safely transmuted. In other words, this RFC makes safe transmutation *as easy as 1..., 2..., `repr(C)`!* ```rust use core::transmute::{ - TransmuteInto, + transmute, stability::{PromiseTransmutableInto, PromiseTransmutableFrom}, }; @@ -22,18 +22,16 @@ pub struct Foo(pub u8, pub u16); // ^ there's a padding byte here, between these fields // Transmute fearlessly! -let _ : Foo = 64u32.transmute_into(); // Alchemy Achieved! -// ^^^^^^^^^^^^^^ provided by the `TransmuteInto` trait +let _ : Foo = transmute!(64u32); // Alchemy Achieved! -let _ : u32 = Foo(16, 12).transmute_into(); // Compile Error! +let _ : u32 = transmute!(Foo(16, 12)); // Compile Error! -// error[E0277]: the trait bound `u32: TransmuteFrom` is not satisfied -// --> src/demo.rs:15:27 +// error[E0277]: the trait bound `u32: TransmuteFrom` is not satisfied +// --> src/demo.rs:14:27 // | -// 15 | let _ : u32 = Foo(16, 12).transmute_into(); // Compile Error! -// | ^^^^^^^^^^^^^^ the trait `TransmuteFrom` is not implemented for `u32` +// 14 | let _ : u32 = transmute!(Foo(16, 12)); // Compile Error! +// | ^^^^^^^^^^^ the trait `TransmuteFrom` is not implemented for `u32` // | -// = note: required because of the requirements on the impl of `TransmuteInto` for `foo::Foo` // = note: byte 8 of the source type may be uninitialized; byte 8 of the destination type cannot be uninitialized. ``` @@ -60,7 +58,7 @@ Given the expressive foundation provided by this RFC, we also envision a range o ## Codifying Language Layout Guarantees Documentation of Rust's layout guarantees for a type are often spread across countless issues, pull requests, RFCs and various official resources. It can be very difficult to get a straight answer. When transmutation is involved, users must reason about the *combined* layout properties of the source and destination types. -This RFC proposes mechanisms that programmers will use to confidently answer such questions—by checking whether the `TransmuteFrom` and `TransmuteInto` traits are implemented, or (equivalently) by checking whether the `can_transmute` predicate (a `const fn`) is satisfied. +This RFC proposes mechanisms that programmers will use to confidently answer such questions—by checking whether the `TransmuteFrom` and `TransmuteInto` traits are implemented. ## Expressing Library Layout Guarantees There is no canonical way for crate authors to declare the SemVer layout guarantees of their types. Crate authors currently must state their layout guarantees using prose in their documentation. In contrast to structural stability (e.g., the declared visibility of fields), layout stability is expressed extra-linguistically. @@ -152,8 +150,8 @@ enum Bool { False = 0, } -let _ : u8 = Bool::True.transmute_into(); -let _ : u8 = Bool::False.transmute_into(); +let _ : u8 = transmute!(Bool::True); +let _ : u8 = transmute!(Bool::False); ``` (Note: #[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] annotation connotes that all aspects of Bool's layout are part of its library stability guarantee.) @@ -162,7 +160,7 @@ let _ : u8 = Bool::False.transmute_into(); ...because all possible instances of `Bool` are also valid instances of [`u8`]. However, transmuting a [`u8`] into a `Bool` is forbidden: ```rust /* ⚠️ This example intentionally does not compile. */ -let _ : Bool = u8::default().transmute_into(); // Compile Error! +let _ : Bool = transmute!(u8::default()); // Compile Error! ``` ...because not all instances of [`u8`] are valid instances of `Bool`. @@ -181,12 +179,12 @@ assert_eq!(mem::size_of::(), mem::size_of::()); We may safely transmute from `Packed` to `Padded`: ```rust -let _ : Padded = Packed::default().transmute_into(); +let _ : Padded = transmute!(Packed::default()); ``` ...but not from `Padded` to `Packed`: ```rust /* ⚠️ This example intentionally does not compile. */ -let _ : Packed = Padded::default().transmute_into(); // Compile Error! +let _ : Packed = transmute!(Padded::default()); // Compile Error! ``` ...because doing so would expose two uninitialized padding bytes in `Padded` as if they were initialized bytes in `Packed`. @@ -195,14 +193,14 @@ let _ : Packed = Padded::default().transmute_into(); // Compile Error! It's well-defined to transmute into a type with fewer bytes than the source type; e.g.: ```rust -let _ : [u8; 16] = [u8; 32]::default().transmute_into(); +let _ : [u8; 16] = transmute!([u8; 32]::default()); ``` This transmute truncates away the final sixteen bytes of the `[u8; 32]` value. A value may ***not*** be transmuted into a type of greater size, if doing so would expose uninitialized bytes as initialized: ```rust /* ⚠️ This example intentionally does not compile. */ -let _ : [u8; 32] = [u8; 16]::default().transmute_into(); // Compile Error! +let _ : [u8; 32] = transmute!([u8; 16]::default()); // Compile Error! ``` #### Requirements on References @@ -222,26 +220,26 @@ A [well-defined transmutation] of references must: You may preserve or decrease the size of the referent type via transmutation: ```rust -let _: &[u8; 3] = (&[0u8; 9]).transmute_into(); +let _: &[u8; 3] = transmute!(&[0u8; 9]); ``` However, you may **not**, under any circumstances, *increase* the size of the referent type: ```rust /* ⚠️ This example intentionally does not compile. */ -let _: &[u8; 9] = (&[0u8; 3]).transmute_into(); // Compile Error! +let _: &[u8; 9] = transmute!(&[0u8; 3]); // Compile Error! ``` ##### Preserve or Relax Alignment [reference-alignment]: #Preserve-or-Relax-Alignment Unaligned loads are undefined behavior. You may transmute a reference into reference of more relaxed alignment: ```rust -let _: &[u8; 0] = (&[0u16; 0]).transmute_into(); +let _: &[u8; 0] = transmute!(&[0u16; 0]); ``` However, you may **not** transmute a reference into a reference of more-restrictive alignment: ```rust /* ⚠️ This example intentionally does not compile. */ -let _: &[u16; 0] = (&[0u8; 0]).transmute_into(); // Compile Error! +let _: &[u16; 0] = transmute!(&[0u8; 0]); // Compile Error! ``` ##### Preserve or Shrink Lifetimes @@ -251,7 +249,7 @@ You may transmute a reference into a reference of lesser lifetime: ```rust fn shrink<'a>() -> &'a u8 { static long : &'static u8 = &16; - long.transmute_into() + transmute!(long) } ``` @@ -259,7 +257,7 @@ However, you may **not** transmute a reference into a reference of greater lifet ```rust /* ⚠️ This example intentionally does not compile. */ fn extend<'a>(short: &'a u8) -> &'static u8 { - short.transmute_into() // Compile Error! + transmute!(short) // Compile Error! } ``` @@ -268,14 +266,14 @@ fn extend<'a>(short: &'a u8) -> &'static u8 { You may preserve or decrease the uniqueness of a reference through transmutation: ```rust -let _: &u8 = (&42u8).transmute_into(); -let _: &u8 = (&mut 42u8).transmute_into(); +let _: &u8 = transmute!(&42u8); +let _: &u8 = transmute!(&mut 42u8); ``` However, you may **not** transmute a shared reference into a unique reference: ```rust /* ⚠️ This example intentionally does not compile. */ -let _: &mut u8 = (&42u8).transmute_into(); // Compile Error! +let _: &mut u8 = transmute!(&42u8); // Compile Error! ``` ##### Mutate-able References Must Preserve Validity @@ -290,7 +288,7 @@ Unlike transmutations of owned values, the transmutation of a mutate-able refere /* ⚠️ This example intentionally does not compile. */ let mut x = NonZeroU8::new(42).unwrap(); { - let y : &mut u8 = (&mut x).transmute_into(); // Compile Error! + let y : &mut u8 = transmute!(&mut x); // Compile Error! *y = 0; } @@ -301,7 +299,7 @@ If this example did not produce a compile error, the value of `z` would not be a ### 📖 When is a transmutation safe? -A well-defined transmutation is ***safe*** if *using* the transmuted value safely cannot violate memory safety. Whereas well-definedness solely concerns the act of transmutation, *safety* is concerned with what might happen with a value *after* transmutation occurs. +A well-defined transmutation is ***safe*** if *using* the transmuted value safely cannot violate memory safety. Whereas well-definedness solely concerns the act of transmutation, *safety* is concerned with what might happen with a value *after* transmutation occurs. A well-defined transmutation must be safe if the involved types are *implicitly constructable*. #### Implicit Constructability A struct or enum variant is *fully implicitly constructable* at a given location only if, at that location, that type can be instantiated via its *implicit constructor*, and its fields are also *implicitly constructable*. @@ -388,24 +386,100 @@ pub mod crate_a { In the above example, the definitions of both `NonEmptySlice` and its field `NonEmptySliceInner` are marked `pub`, and all fields of these types are marked `pub`. However, `NonEmptySlice` is *not* fully implicitly constructible outside of `crate_a`, because the module containing `NonEmptySliceInner` is not visibile outside of `crate_a`. #### Constructability and Transmutation -Transmutation supplies a mechanism for constructing instances of a type *without* invoking its implicit constructor, nor any constructors defined by the type's author. - -In the previous examples, it would be *unsafe* to transmute `0u128` into `NonEmptySlice` outside `crate_a`, because subsequent *safe* use of that value (namely, calling `first`) would violate memory safety. (However, it's completely safe to transmute `NonEmptySlice` into a `u128`.) +Transmutation supplies a mechanism for constructing instances of a type *without* invoking its implicit constructor, nor any constructors defined by the type's author. In the previous examples, it would be *unsafe* to transmute `[usize; 2]` into `NonEmptySlice` outside `crate_a`, because subsequent *safe* use of that value (namely, calling `first`) would violate memory safety: +```rust +/* ⚠️ This example intentionally does not compile. */ +// [usize; 2] ⟶ NonEmptySlice +let _: NonEmptySlice<'static, u8> = transmute!([0usize; 2]); // Compile Error: `NonEmptySlice<_, _>` is not constructible here. +``` +If a field is private, then instantiating or modifying it via transmutation is not, generally speaking, safe. -For transmutations where the destination type involves mutate-able references, the constructability of the source type is also relevant. Consider: +For transmutations where the destination type involves mutate-able references, the constructability of the *source* type is also relevant. Consider: ```rust /* ⚠️ This example intentionally does not compile. */ let arr = [0u8, 1u8, 2u8]; let mut x = NonEmptySlice::from_array(&arr, 2); { - let y : &mut u128 = (&mut x).transmute_into(); // Compile Error! - *y = 0u128; + // &mut NonEmptySlice ⟶ &mut [usize; 2] + let y : &mut u128 = transmute!(&mut x) // Compile Error! `&mut NonEmptySlice` is not constructible here. + *y[0] = 0; + *y[1] = 0; } let z : NonEmptySlice = x; ``` If this example did not produce a compile error, the value of `z` would not be a safe instance of its type, `NonEmptySlice`, because `z.first()` would dereference a null pointer. +#### Constructability and Scope +Whether a type is fully implicitly constructable will depends on the *scope* in which that question is asked. Consider: +```rust +pub mod a { + + #[repr(C)] pub struct Foo(private::Bar); + + mod private { + #[repr(C)] pub struct Bar; + } + + // `Foo` is fully implicitly constructible in this module. + const _: Foo = Foo { private::Bar }; + + // Thus, `Foo` is transmutable in this module! + const _: Foo = transmute!(()); +} + +pub mod b { + use super::a; + + // `Foo` is NOT fully implicitly constructible in this module. + const _: Foo = a::Foo { a::private::Bar }; // Compile Error: the module `a::private` is private. + + // Thus, `Foo` is NOT transmutable in this module: + const _: Foo = transmute!(()); // Compile Error: `Foo` is not constructible here. +} +``` + +The `Constructible` marker trait makes the relationship between scope and constructability explicit: +```rust +pub unsafe trait Constructible +{} +``` +A type `T` is `Constructible`, if it is fully implicitly constructible in the scope represented by the type `Scope`. These `Scope` types are generated by the `Here!()` macro, which produces a type that uniquely identifies its invocation scope. + +The `transmute!` macro provides a shorthand for safely transmuting a value using its invocation scope as its reference frame: +```rust +pub macro transmute($expr: expr) { + TransmuteFrom::<_, Here!()>::transmute_from($expr) + // ┯ ━━━┯━━━ + // │ ┕ check constructability from `transmute!`'s invocation scope + // ┕ the destination type of the transmute (`_` used to infer the type from context) +} +``` + +The explicit `Scope` parameter of `TransmuteFrom` makes possible the creation of generic abstractions over it. For instance, consider a hypothetical `FromBytes` trait that indicates whether `Self` is safely initializable from a a sufficiently large buffer of `u8`s: +```rust +pub unsafe trait FromBytes +{} + +unsafe impl FromBytes for Dst +where + Dst: TransmuteFrom<[u8; usize::MAX], ??? >, +{} +``` +The above definition leaves ambiguous (`???`) the scope in which the constructability of `Dst` is checked: is it the from the perspective of where this trait is defined, or where this trait is *used*? You probably do *not* intend for this trait to *only* be usable with `Dst` types that are defined in the same scope as the `FromBytes` trait! + +Adding an explicit `Scope` parameter to `FromBytes` makes this unambiguous; the transmutability of `Dst` should be assessed from where the trait is used, *not* where it is defined: +```rust +pub unsafe trait FromBytes +{} + +unsafe impl FromBytes for Dst +where + Dst: TransmuteFrom<[u8; usize::MAX], Scope>, +{} +``` + + ### 📖 When is a transmutation stable? [stability]: #-when-is-a-transmutation-stable @@ -417,18 +491,14 @@ The question is, then: *how can the author of a type reason about transmutations You may declare the stability guarantees of your type by implementing one or both of two traits: ```rust -pub trait PromiseTransmutableFrom -where - Self::Archetype: PromiseTransmutableFrom +pub trait PromiseTransmutableFrom { - type Archetype: TransmuteInto + type Archetype: TransmuteInto; } -pub trait PromiseTransmutableInto -where - Self::Archetype: PromiseTransmutableInto, +pub trait PromiseTransmutableInto { - type Archetype: TransmuteFrom + type Archetype: TransmuteFrom; } ``` @@ -441,18 +511,18 @@ You are free to change the layout of your type however you like between minor cr #### Stability & Transmutation Together with the `PromiseTransmutableFrom` and `PromiseTransmutableInto` traits, this impl of `TransmuteFrom` constitutes the formal definition of transmutation stability: ```rust -unsafe impl TransmuteFrom for Dst +unsafe impl TransmuteFrom for Dst where - Src: PromiseTransmutableInto, - Dst: PromiseTransmutableFrom, + Src: PromiseTransmutableInto, + Dst: PromiseTransmutableFrom, - Dst::Archetype: TransmuteFrom + Dst::Archetype: TransmuteFrom {} ``` Why is this safe? Can we really safely judge whether `Dst` is transmutable from `Src` by assessing the transmutability of two different types? Yes! Transmutability is *transitive*. Concretely, if we can safely transmute: - `Src` to `Src::Archetype` (enforced by `Src: PromiseTransmutableInto`), and - `Dst::Archetype` to `Dst` (enforced by `Dst: PromiseTransmutableFrom`), and - - `Src::Archetype` to `Dst::Archetype` (enforced by `Dst::Archetype: TransmuteFrom`), + - `Src::Archetype` to `Dst::Archetype` (enforced by `Dst::Archetype: TransmuteFrom`), ...then it follows that we can safely transmute `Src` to `Dst` in three steps: 1. we transmute `Src` to `Src::Archetype`, @@ -477,17 +547,17 @@ const _: () = { use core::transmute::stability::PromiseTransmutableFrom; #[repr(C)] - pub struct TransmutableFromArchetype( - pub ::Archetype, - pub ::Archetype, + pub struct TransmutableFromArchetype( + pub >::Archetype, + pub >::Archetype, ); - impl PromiseTransmutableFrom for TransmutableFromArchetype { + impl PromiseTransmutableFrom for TransmutableFromArchetype { type Archetype = Self; } - impl PromiseTransmutableFrom for Foo { - type Archetype = TransmutableFromArchetype; + impl PromiseTransmutableFrom for Foo { + type Archetype = TransmutableFromArchetype; } }; @@ -496,17 +566,17 @@ const _: () = { use core::transmute::stability::PromiseTransmutableInto; #[repr(C)] - pub struct TransmutableIntoArchetype( - pub ::Archetype, - pub ::Archetype, + pub struct TransmutableIntoArchetype( + pub >::Archetype, + pub >::Archetype, ); - impl PromiseTransmutableFrom for TransmutableIntoArchetype { + impl PromiseTransmutableInto for TransmutableIntoArchetype { type Archetype = Self; } - impl PromiseTransmutableInto for Foo { - type Archetype = TransmutableIntoArchetype; + impl PromiseTransmutableInto for Foo { + type Archetype = TransmutableIntoArchetype; } }; ``` @@ -526,18 +596,15 @@ The alignment of `Foo` affects transmutability of `&Foo`. A `&Foo` cannot be saf const _: () = { use core::transmute::stability::PromiseTransmutableFrom; + // 2^29 is currently the maximum alignment #[repr(C, align(536870912))] - pub struct TransmutableFromArchetype( - pub ::Archetype, - pub ::Archetype, + pub struct TransmutableFromArchetype( + pub >::Archetype, + pub >::Archetype, ); - impl PromiseTransmutableFrom for TransmutableFromArchetype { - type Archetype = Self; - } - - impl PromiseTransmutableFrom for Foo { - type Archetype = TransmutableFromArchetype; + impl PromiseTransmutableFrom for Foo { + type Archetype = TransmutableFromArchetype; } }; ``` @@ -547,17 +614,13 @@ const _: () = { use core::transmute::stability::PromiseTransmutableInto; #[repr(C, packed(1))] - pub struct TransmutableIntoArchetype( - pub ::Archetype, - pub ::Archetype, + pub struct TransmutableIntoArchetype( + pub >::Archetype, + pub >::Archetype, ); - impl PromiseTransmutableInto for TransmutableIntoArchetype { - type Archetype = Self; - } - - impl PromiseTransmutableInto for Foo { - type Archetype = TransmutableIntoArchetype; + impl PromiseTransmutableInto for Foo { + type Archetype = TransmutableIntoArchetype; } }; ``` @@ -570,7 +633,7 @@ Two traits provide mechanisms for transmutation between types: ```rust // this trait is implemented automagically by the compiler #[lang = "transmute_from"] -pub unsafe trait TransmuteFrom +pub unsafe trait TransmuteFrom where Neglect: TransmuteOptions, { @@ -606,7 +669,7 @@ where } // implemented in terms of `TransmuteFrom` -pub unsafe trait TransmuteInto +pub unsafe trait TransmuteInto where Neglect: TransmuteOptions, { @@ -623,10 +686,10 @@ where Neglect: TransmuteOptions; } -unsafe impl TransmuteInto for Src +unsafe impl TransmuteInto for Src where Src: ?Sized, - Dst: ?Sized + TransmuteFrom, + Dst: ?Sized + TransmuteFrom, Neglect: TransmuteOptions, { ... @@ -638,7 +701,7 @@ In the above definitions, `Src` represents the source type of the transmutation, ### Neglecting Static Checks [options]: #Neglecting-Static-Checks -The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are unsafe, ill-defined, or unstable. However, you may explicitly opt-out of some static checks: +The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are unsafe, ill-defined, or unstable. However, you may explicitly opt-out of some static checks; e.g.: | Transmute Option | Usable With | |---------------------|---------------------------------------------------------| @@ -674,15 +737,15 @@ impl TransmuteOptions for NeglectStability {} Prior to the adoption of the [stability declaration traits][stability], crate authors documented the layout guarantees of their types with doc comments. The `TransmuteFrom` and `TransmuteInto` traits and methods may be used with these types by requesting that the stability check is neglected; for instance: ```rust -fn serialize(val : LibraryType, dst: W) -> std::io::Result<()> +fn serialize(val : LibraryType, dst: W) -> std::io::Result<()> where - LibraryType: TransmuteInto<[u8; size_of::()], NeglectStability> + LibraryType: TransmuteInto<[u8; size_of::()], Scope, NeglectStability> { ... } ``` -Neglecting stability over-eagerly cannot cause ill-definedness or unsafety. For this reason, it is the only transmutation option available on the safe methods `transmute_from` and `transmute_into`. However, neglecting stability over-eagerly may cause your code to cease compiling if the authors of the source and destination types make changes that affect their layout. +Neglecting stability over-eagerly cannot cause ill-definedness or unsafety. For this reason, it is the only transmutation option available on the safe methods `transmute_from` and `transmute_into`. However, neglecting stability over-eagerly may cause your code to behave incorrectly or cease compiling if the authors of the source and destination types make changes that affect their layout. By using the `NeglectStability` option to transmute types you do not own, you are committing to ensure that your reliance on these types' layouts is consistent with their documented stability guarantees. @@ -702,9 +765,9 @@ By using the `NeglectAlignment` option, you are committing to ensure that the tr /// /// This produces `None` if the referent isn't appropriately /// aligned, as required by the destination type. -pub fn try_cast_ref<'t, 'u, T, U>(src: &'t T) -> Option<&'u U> +pub fn try_cast_ref<'t, 'u, Scope, T, U>(src: &'t T) -> Option<&'u U> where - &'t T: TransmuteInto<&'u U, NeglectAlignment>, + &'t T: TransmuteInto<&'u U, Scope, NeglectAlignment>, { if (src as *const T as usize) % align_of::() != 0 { None @@ -727,7 +790,7 @@ enum Bool { } /* ⚠️ This example intentionally does not compile. */ -let _ : Bool = some_u8_value.transmute_into(); // Compile Error! +let _ : Bool = transmute!(some_u8_value); // Compile Error! ``` The `NeglectValidity` option disables this check. ```rust @@ -745,15 +808,15 @@ enum Bool { False = 0, } -pub trait TryIntoBool +pub trait TryIntoBool { fn try_into_bool(self) -> Option; } -impl TryIntoBool for T +impl TryIntoBool for T where T: TransmuteInto, - u8: TransmuteInto + u8: TransmuteInto { fn try_into_bool(self) -> Option { let val: u8 = self.transmute_into(); @@ -777,42 +840,114 @@ Even with `NeglectValidity`, the compiler will statically reject transmutations #[derive(PromiseTransmutableFrom)] #[repr(C)] enum Bar { Z = 42 } -let _ = ::unsafe_transmute_from(Foo::N) // Compile error! +let _ = ::unsafe_transmute_from(Foo::N) // Compile error! ``` # Reference-level explanation [reference-level-explanation]: #reference-level-explanation - ## Implementation Guidance -The *only* item defined by this RFC requiring special compiler support is `TransmuteFrom`. To realize this RFC's proposal of safe transmutation between different types, this item will require compiler support. However, the *most* minimal acceptable implementation of `TransmuteFrom` can be achieved entirely in-language: +Three items in this RFC require special compiler support: + - `Here!()` + - `Constructible` + - `TransmuteFrom` +### Implementing `Here!()` +The `Here!` macro produces a type that uniquely identifies its invocation scope. For instance: ```rust -/// A type is transmutable into itself. -unsafe impl TransmuteFrom for T -where - Neglect: TransmuteOptions -{} +use static_assertions::*; + +type A = Here!(); +type B = Here!(); + +/* Types A and !::B are equal */ +assert_type_eq_all!(A, B); + +trait Foo { + type C; + const CONST: (); +} + +impl Foo for ! { + type C = Here!(); -/// A transmutation is *stable* if... -unsafe impl TransmuteFrom for Dst + const CONST: () = { + type D = Here!(); + + /* A, B and !::C are equal */ + assert_type_eq_all!(A, B, !::C); + + /* C and D are NOT equal; C and D inhabit different scopes */ + assert_type_ne_all!(C, D); + }; +} +``` +Scope types should (as much as possible) pretty print in compiler error messages as their definition path. + +### Implementing `Constructible` +The compiler implements `Constructible` for `T` if `T` is fully implicitly constructible in the scope uniquely identified by the type `Scope`. + +A type `T` is fully implicitly constructible in a particular scope if: + - `T`'s implicit constructor is reachable from the scope, and either: + - `T` has no fields, or + - `T`'s fields are fully implicitly constructible from the scope. + +The `Constructible` trait does not ever need to be made stable, or even visible (à la the virtual `Freeze` trait). It is merely a useful device for implementing `TransmuteFrom`. + +### Implementing `TransmuteFrom` +The implementation of `TransmuteFrom` is completely internal to the compiler (à la [`Sized`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_sized) and [`Freeze`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_freeze)). + +#### Stability and Transmutation +Unless `NeglectStability` is used as a `Neglect` option, a `Src` is *stably* transmutable into `Dst` in a given `Scope` if: + 1. `Src: PromiseTransmutableInto` + 2. `Dst: PromiseTransmutableFrom` + 3. `Dst::Archetype: TransmuteFrom` + +#### Constructability and Transmutation +Unless `NeglectConstructability` is used as `Neglect` option, a `Src` is *safely* transmutable into `Dst` in a given `Scope` if: + 1. `Dst: Constructible` + 2. `Dst: TransmuteFrom` + +If `Src` is a mutatable reference, then additionally: + 1. `Src: Constructible` + + +### Minimal Useful Stabilization Surface +Stabilizing *only* these items of the Initial Smart Implementation will cover many use-cases: + - `transmute!()` + - `#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)]` + - `#[derive(PromiseTransmutableInto)]` + +(If the [`PromiseTransmutable` shorthand extension][extension-promisetransmutable-shorthand] is accepted, this may be further reduced to just *two* items: `transmute!()` and `#[derive(PromiseTransmutable)]`.) + +To define traits that generically abstract over `TransmuteFrom`, these items must be stabilized: + - `Here!()` + - `TransmuteFrom` + - `TransmuteOptions` and `SafeTransmuteOptions` + +Stabilizing `PromiseTransmutableFrom` and `PromiseTransmutableInto` will additionally allow end-users make limited stability promises, and to make stability promises for types where `derive` is too restrictive (namely, types containing `PhantomData`). + +Additionally stabilizing `TransmuteInto` and `NeglectStability` will additionally allow end-users to implement `PromiseTransmutableFrom` and `PromiseTransmutableInto` in cases where the `Archetype`'s trait bounds must be repeated for lack of [implied bounds](https://github.com/rust-lang/rust/issues/44491); e.g.: +```rust +impl PromiseTransmutableFrom for GenericArray where - Src: PromiseTransmutableInto, - Dst: PromiseTransmutableFrom, - ::Archetype: - TransmuteFrom< - ::Archetype, - NeglectStability - > -{} + T: PromiseTransmutableFrom, + N: ArrayLength, + + // for lack of implied bounds, we must repeat the bounds on `Archetype`, + // which requires naming `NeglectStability` + GenericArray: TransmuteInto +{ + type Archetype = GenericArray; +} ``` -This [minimal implementation][minimal-impl] is sufficient for convincing the compiler to accept basic stability declarations, such as those of Rust's primitive types. It is *insufficient* for making the compiler accept transmutations between *different* types (and, consequently, complex stability declarations). Implementers should use this as a starting point. -### Listing for Initial, Minimal Implementation + +### Complete API Surface [minimal-impl]: #Listing-for-Initial-Minimal-Implementation -This listing is both a minimal implementation of this RFC (excepting the automatic derives) and the **canonical specification** of this RFC's API surface ([playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=1ff1700e6dba595f1e600d20da5d6387)): +This listing is both a **canonical specification** of this RFC's API surface ([playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=ec5c618d843acd7c40855ebfb386d49a)): ```rust -#![feature(untagged_unions,const_fn,const_fn_union)] // for the impl of transmute free functions +#![feature(untagged_unions,const_fn,const_fn_union)] // for the impl of unsafe_transmute_from #![feature(const_generics)] // for stability declarations on `[T; N]` #![feature(decl_macro)] // for stub implementations of derives #![feature(never_type)] // for stability declarations on `!` @@ -824,44 +959,15 @@ pub mod transmute { use {options::*, stability::*}; - /// Reinterprets the bits of a value of one type as another type, safely. - /// - /// Use `()` for `Neglect` if you do not wish to neglect any static checks. - #[inline(always)] - pub const fn safe_transmute(src: Src) -> Dst - where - Dst: TransmuteFrom, - Neglect: SafeTransmuteOptions - { - unsafe { unsafe_transmute::(src) } - } - - /// Reinterprets the bits of a value of one type as another type, potentially unsafely. - /// - /// The onus is on you to ensure that calling this method is safe. - #[inline(always)] - pub const unsafe fn unsafe_transmute(src: Src) -> Dst - where - Dst: TransmuteFrom, - Neglect: TransmuteOptions - { - use core::mem::ManuallyDrop; - - #[repr(C)] - union Transmute { - src: ManuallyDrop, - dst: ManuallyDrop, - } - - unsafe { - ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst) - } + /// Safely and stably transmute $expr + pub macro transmute($expr: expr) { + TransmuteFrom::<_, Here!()>::transmute_from($expr) } /// `Self: TransmuteInto + pub unsafe trait TransmuteInto where Neglect: TransmuteOptions, { @@ -882,10 +988,10 @@ pub mod transmute { Neglect: TransmuteOptions; } - unsafe impl TransmuteInto for Src + unsafe impl TransmuteInto for Src where Src: ?Sized, - Dst: ?Sized + TransmuteFrom, + Dst: ?Sized + TransmuteFrom, Neglect: TransmuteOptions, { #[inline(always)] @@ -910,9 +1016,9 @@ pub mod transmute { } /// `Self: TransmuteInto + pub unsafe trait TransmuteFrom where Neglect: TransmuteOptions, { @@ -937,24 +1043,25 @@ pub mod transmute { Self: Sized, Neglect: TransmuteOptions, { - unsafe_transmute::(src) + use core::mem::ManuallyDrop; + + #[repr(C)] + union Transmute { + src: ManuallyDrop, + dst: ManuallyDrop, + } + + unsafe { + ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst) + } } } /// A type is always transmutable from itself. // This impl will be replaced with a compiler-supported for arbitrary source and destination types. - unsafe impl TransmuteFrom for T {} - - /// A type is *stably* transmutable if... - unsafe impl TransmuteFrom for Dst + unsafe impl TransmuteFrom for T where - Src: PromiseTransmutableInto, - Dst: PromiseTransmutableFrom, - ::Archetype: - TransmuteFrom< - ::Archetype, - NeglectStability - > + Neglect: TransmuteOptions {} /// Traits for declaring the SemVer stability of types. @@ -963,21 +1070,16 @@ pub mod transmute { use super::{TransmuteFrom, TransmuteInto, options::NeglectStability}; /// Declare that transmuting `Self` into `Archetype` is SemVer-stable. - pub trait PromiseTransmutableInto - where - Self::Archetype: PromiseTransmutableInto - { + pub trait PromiseTransmutableInto { /// The `Archetype` must be safely transmutable from `Self`. - type Archetype: TransmuteFrom; + type Archetype: TransmuteFrom; } /// Declare that transmuting `Self` from `Archetype` is SemVer-stable. - pub trait PromiseTransmutableFrom - where - Self::Archetype: PromiseTransmutableFrom + pub trait PromiseTransmutableFrom { /// The `Archetype` must be safely transmutable into `Self`. - type Archetype: TransmuteInto; + type Archetype: TransmuteInto; } @@ -994,147 +1096,131 @@ pub mod transmute { } - impl PromiseTransmutableInto for ! {type Archetype = Self;} - impl PromiseTransmutableFrom for ! {type Archetype = Self;} - - impl PromiseTransmutableInto for () {type Archetype = Self;} - impl PromiseTransmutableFrom for () {type Archetype = Self;} - - impl PromiseTransmutableInto for f32 {type Archetype = Self;} - impl PromiseTransmutableFrom for f32 {type Archetype = Self;} - impl PromiseTransmutableInto for f64 {type Archetype = Self;} - impl PromiseTransmutableFrom for f64 {type Archetype = Self;} - - impl PromiseTransmutableInto for i8 {type Archetype = Self;} - impl PromiseTransmutableFrom for i8 {type Archetype = Self;} - impl PromiseTransmutableInto for i16 {type Archetype = Self;} - impl PromiseTransmutableFrom for i16 {type Archetype = Self;} - impl PromiseTransmutableInto for i32 {type Archetype = Self;} - impl PromiseTransmutableFrom for i32 {type Archetype = Self;} - impl PromiseTransmutableInto for i64 {type Archetype = Self;} - impl PromiseTransmutableFrom for i64 {type Archetype = Self;} - impl PromiseTransmutableInto for i128 {type Archetype = Self;} - impl PromiseTransmutableFrom for i128 {type Archetype = Self;} - impl PromiseTransmutableInto for isize {type Archetype = Self;} - impl PromiseTransmutableFrom for isize {type Archetype = Self;} - - impl PromiseTransmutableInto for u8 {type Archetype = Self;} - impl PromiseTransmutableFrom for u8 {type Archetype = Self;} - impl PromiseTransmutableInto for u16 {type Archetype = Self;} - impl PromiseTransmutableFrom for u16 {type Archetype = Self;} - impl PromiseTransmutableInto for u32 {type Archetype = Self;} - impl PromiseTransmutableFrom for u32 {type Archetype = Self;} - impl PromiseTransmutableInto for u64 {type Archetype = Self;} - impl PromiseTransmutableFrom for u64 {type Archetype = Self;} - impl PromiseTransmutableInto for u128 {type Archetype = Self;} - impl PromiseTransmutableFrom for u128 {type Archetype = Self;} - impl PromiseTransmutableInto for usize {type Archetype = Self;} - impl PromiseTransmutableFrom for usize {type Archetype = Self;} + impl PromiseTransmutableInto for ! {type Archetype = Self;} + impl PromiseTransmutableFrom for ! {type Archetype = Self;} + impl PromiseTransmutableInto for () {type Archetype = Self;} + impl PromiseTransmutableFrom for () {type Archetype = Self;} + + impl PromiseTransmutableInto for f32 {type Archetype = Self;} + impl PromiseTransmutableFrom for f32 {type Archetype = Self;} + impl PromiseTransmutableInto for f64 {type Archetype = Self;} + impl PromiseTransmutableFrom for f64 {type Archetype = Self;} + + impl PromiseTransmutableInto for i8 {type Archetype = Self;} + impl PromiseTransmutableFrom for i8 {type Archetype = Self;} + impl PromiseTransmutableInto for i16 {type Archetype = Self;} + impl PromiseTransmutableFrom for i16 {type Archetype = Self;} + impl PromiseTransmutableInto for i32 {type Archetype = Self;} + impl PromiseTransmutableFrom for i32 {type Archetype = Self;} + impl PromiseTransmutableInto for i64 {type Archetype = Self;} + impl PromiseTransmutableFrom for i64 {type Archetype = Self;} + impl PromiseTransmutableInto for i128 {type Archetype = Self;} + impl PromiseTransmutableFrom for i128 {type Archetype = Self;} + impl PromiseTransmutableInto for isize {type Archetype = Self;} + impl PromiseTransmutableFrom for isize {type Archetype = Self;} + + impl PromiseTransmutableInto for u8 {type Archetype = Self;} + impl PromiseTransmutableFrom for u8 {type Archetype = Self;} + impl PromiseTransmutableInto for u16 {type Archetype = Self;} + impl PromiseTransmutableFrom for u16 {type Archetype = Self;} + impl PromiseTransmutableInto for u32 {type Archetype = Self;} + impl PromiseTransmutableFrom for u32 {type Archetype = Self;} + impl PromiseTransmutableInto for u64 {type Archetype = Self;} + impl PromiseTransmutableFrom for u64 {type Archetype = Self;} + impl PromiseTransmutableInto for u128 {type Archetype = Self;} + impl PromiseTransmutableFrom for u128 {type Archetype = Self;} + impl PromiseTransmutableInto for usize {type Archetype = Self;} + impl PromiseTransmutableFrom for usize {type Archetype = Self;} use core::marker::PhantomData; - impl PromiseTransmutableInto for PhantomData { type Archetype = Self; } - impl PromiseTransmutableFrom for PhantomData { type Archetype = Self; } + impl PromiseTransmutableInto for PhantomData { type Archetype = Self; } + impl PromiseTransmutableFrom for PhantomData { type Archetype = Self; } - impl PromiseTransmutableInto for [T; N] + impl PromiseTransmutableInto for [T; N] where - T: PromiseTransmutableInto, - [T::Archetype; N] - : TransmuteFrom - + PromiseTransmutableInto, + T: PromiseTransmutableInto, + [T::Archetype; N]: TransmuteFrom { type Archetype = [T::Archetype; N]; } - impl PromiseTransmutableFrom for [T; N] + impl PromiseTransmutableFrom for [T; N] where - T: PromiseTransmutableFrom, - [T::Archetype; N] - : TransmuteInto - + PromiseTransmutableFrom, + T: PromiseTransmutableFrom, + [T::Archetype; N]: TransmuteInto { type Archetype = [T::Archetype; N]; } - impl PromiseTransmutableInto for *const T + impl PromiseTransmutableInto for *const T where - T: PromiseTransmutableInto, - *const T::Archetype - : TransmuteFrom - + PromiseTransmutableInto, + T: PromiseTransmutableInto, + *const T::Archetype: TransmuteFrom { type Archetype = *const T::Archetype; } - impl PromiseTransmutableFrom for *const T + impl PromiseTransmutableFrom for *const T where - T: PromiseTransmutableFrom, - *const T::Archetype - : TransmuteInto - + PromiseTransmutableFrom, + T: PromiseTransmutableFrom, + *const T::Archetype: TransmuteInto { type Archetype = *const T::Archetype; } - impl PromiseTransmutableInto for *mut T + impl PromiseTransmutableInto for *mut T where - T: PromiseTransmutableInto, - *mut T::Archetype - : TransmuteFrom - + PromiseTransmutableInto, + T: PromiseTransmutableInto, + *mut T::Archetype: TransmuteFrom { type Archetype = *mut T::Archetype; } - impl PromiseTransmutableFrom for *mut T + impl PromiseTransmutableFrom for *mut T where - T: PromiseTransmutableFrom, - *mut T::Archetype - : TransmuteInto - + PromiseTransmutableFrom, + T: PromiseTransmutableFrom, + *mut T::Archetype: TransmuteInto, { type Archetype = *mut T::Archetype; } - impl<'a, T: ?Sized> PromiseTransmutableInto for &'a T + impl<'a, T: ?Sized, Scope> PromiseTransmutableInto for &'a T where - T: PromiseTransmutableInto, - &'a T::Archetype - : TransmuteFrom<&'a T, NeglectStability> - + PromiseTransmutableInto, + T: PromiseTransmutableInto, + T::Archetype: 'a, + &'a T::Archetype: TransmuteFrom<&'a T, Scope, NeglectStability> { type Archetype = &'a T::Archetype; } - impl<'a, T: ?Sized> PromiseTransmutableFrom for &'a T + impl<'a, T: ?Sized, Scope> PromiseTransmutableFrom for &'a T where - T: PromiseTransmutableFrom, - &'a T::Archetype - : TransmuteInto<&'a T, NeglectStability> - + PromiseTransmutableFrom, + T: PromiseTransmutableFrom, + T::Archetype: 'a, + &'a T::Archetype: TransmuteInto<&'a T, Scope, NeglectStability> { type Archetype = &'a T::Archetype; } - impl<'a, T: ?Sized> PromiseTransmutableInto for &'a mut T + + impl<'a, T: ?Sized, Scope> PromiseTransmutableInto for &'a mut T where - T: PromiseTransmutableInto, - &'a mut T::Archetype - : TransmuteFrom<&'a mut T, NeglectStability> - + PromiseTransmutableInto, + T: PromiseTransmutableInto, + T::Archetype: 'a, + &'a mut T::Archetype: TransmuteFrom<&'a mut T, Scope, NeglectStability> { type Archetype = &'a mut T::Archetype; } - impl<'a, T: ?Sized> PromiseTransmutableFrom for &'a mut T + impl<'a, T: ?Sized, Scope> PromiseTransmutableFrom for &'a mut T where - T: PromiseTransmutableFrom, - &'a mut T::Archetype - : TransmuteInto<&'a mut T, NeglectStability> - + PromiseTransmutableFrom, + T: PromiseTransmutableFrom, + T::Archetype: 'a, + &'a mut T::Archetype: TransmuteInto<&'a mut T, Scope, NeglectStability> { type Archetype = &'a mut T::Archetype; } @@ -1155,8 +1241,7 @@ pub mod transmute { /// Neglect the stability check of `TransmuteFrom`. pub struct NeglectStability; - // Uncomment this if/when constructibility is fully implemented: - // impl SafeTransmuteOptions for NeglectStability {} + impl SafeTransmuteOptions for NeglectStability {} impl TransmuteOptions for NeglectStability {} // prevent third-party implementations of `TransmuteOptions` @@ -1172,182 +1257,6 @@ pub mod transmute { } ``` -### Towards an Initial, Smart Implementation - -To support transmutations between different types, implementers of this RFC should begin by defining a `transmute_from` lang item to annotate libcore's definition of `TransmuteFrom`. Whether `TransmuteFrom` is implemented for a given type and parameters shall be determined within the implementation of the type system (à la [`Sized`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_sized) and [`Freeze`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_freeze)). - -This initial smart implementation may be made simpler by: - - *not* supporting enums - - *not* supporting unions - - *not* supporting `Neglect` parameters besides `()` and `NeglectStability` - - simplifying constructability - -#### Simplifying Constructability -The safety property of constructability defined in the guidance-level explanation of this RFC describes a platonic ideal of the property. - -However, we recognize that this definition poses implementation challenges: In our definition of constructability, answering the question of whether a struct or enum variant is constructible depends on *where* that question is being asked. Consequently, answering whether a given type `Src` is `TransmutableInto` a given type `Dst` will depend on *where* that question is posed. - -We recommend adopting a simplified definition of *constructability*: a type is *constructible* if its fields are marked `pub`, and those fields are constructible. With this definition, answering the question of whether a type is constructible does *not* depend on where the question is asked: we do not examine the visibility of the involved types. - -Unfortunately, with no other actions taken, this simplified definition comes... - -##### ...at the Cost of Safety -This definition is *usually* sufficient for ensuring safety: it is *generally* an error to expose a private type in a public type signature. However, these errors may be circumvented using the public-type-in-private-module trick: -```rust -pub mod crate_a { - - #[repr(C)] - pub struct NonEmptySlice<'a, T>(pub private::NonEmptySliceInner<'a, T>); - - impl<'a, T> NonEmptySlice<'a, T> { - pub fn from_array(arr: &'a [T; N], len: usize) -> Self { - assert!(len <= N && len > 0); - Self( - private::NonEmptySliceInner { - data: arr as *const T, - len, - lifetime: core::marker::PhantomData, - } - ) - } - - pub fn first(&self) -> &'a T { - unsafe { &*self.0.data } - } - } - - // introduce a private module to avoid `private_in_public` error (E0446): - pub(crate) mod private { - #[repr(C)] - pub struct NonEmptySliceInner<'a, T> { - pub data: *const T, - pub len: usize, - pub lifetime: core::marker::PhantomData<&'a ()>, - } - } - -} -``` -With this simplified definition of constructability, it is possible for a third-party to define a *safe* constructor of `NonEmptySlice` that produces a value which is *unsafe* to use: -```rust -pub evil_constructor(src: T) -> NonEmptySlice<'static, u8> -where - T: TransmuteInto, NeglectStability>, -{ - src.transmute_into() -} - -evil_constructor(0u128).first() // muahaha! -``` - -The above code is "safe" because our simplified definition of constructability fails to recognize this pattern of encapsulation, and because `NeglectStability` is a `SafeTransmutationOption`. - -The intent of `NeglectStability` is to permit the safe transmutation of types that predate the stabilization of the stability declaration traits. It also provides a convenient escape-hatch for type authors to neglect the stability of transmutations of their *own* types, without sacrificing safety. `NeglectStability` is a `SafeTransmutationOption` because, in principle, neglecting stability does not diminish safety. Our simplified definition of constructability violates this principle. - -By temporarily sacrificing these goals, we may preserve safety solely... - -##### ...at the Cost of `NeglectStability` -We may preserve safety by demoting `NeglectStability` to `UnsafeTransmutationOption`-status. - -In doing so, a third-party is forced to resort to an `unsafe` transmutation to construct `NonEmptySlice`; e.g.: - -```rust -pub evil_constructor(src: T) -> NonEmptySlice<'static, u8> -where - T: TransmuteInto, NeglectStability>, -{ - // unsafe because we `NeglectStability` - unsafe { src.unsafe_transmute_into() } -} -``` - -Demoting `NeglectStability` to unsafe-status does not stop type authors from opting-in to stable (and thus safe) transmutations; e.g., with `derive(PromiseTransmutableFrom)`: -```rust -pub mod crate_a { - - #[derive(PromiseTransmutableFrom)] - #[repr(C)] - pub struct NonEmptySlice<'a, T>(pub private::NonEmptySliceInner<'a, T>); - - impl<'a, T> NonEmptySlice<'a, T> { - pub fn from_array(arr: &'a [T; N], len: usize) -> Self { - assert!(len <= N && len > 0); - Self( - private::NonEmptySliceInner { - data: arr as *const T, - len, - lifetime: core::marker::PhantomData, - } - ) - } - - pub fn first(&self) -> &'a T { - unsafe { &*self.0.data } - } - } - - // introduce a private module to avoid `private_in_public` error (E0446): - pub(crate) mod private { - #[derive(PromiseTransmutableFrom)] - #[repr(C)] - pub struct NonEmptySliceInner<'a, T> { - pub data: *const T, - pub len: usize, - pub lifetime: core::marker::PhantomData<&'a ()>, - } - } - -} -``` -In the above example, the type author declares `NonEmptySlice` and `NonEmptySliceInner` to be stably instantiatable via transmutation. Given this, a third-party no longer needs to resort to `unsafe` code to violate the the invariants on `inner`: -```rust -pub evil_constructor(src: T) -> NonEmptySlice<'static, u8> -where - T: TransmuteInto>, -{ - src.transmute_into() -} - -evil_constructor(0u128).first() // muahaha! -``` -This safety hazard is not materially different from the one that would be induced if the type author implemented `DerefMut` for `NonEmptySlice`, or made the `private` module `pub`, or otherwise explicitly provided outsiders with unrestricted mutable access to `data`. - -##### Recommendation -We recommend that that implementers of this RFC initially simplify constructability by: - - adopting our simplified definition of constructability - - demoting `NeglectStability` to unsafe status (i.e., not implementing `SafeTransmuteOptions` for `NeglectStability`; *only* `TransmuteOptions`) - - advise users that implementing the stability declaration traits on types that are not fully-implicitly constructable will be a compiler-error will be a compiler error (i.e., these traits must not be implemented on types exploiting the pub-in-priv trick) - -If and when the implementation of `TransmuteFrom` encodes our complete definition of constructability, `NeglectStability` shall become a safe transmute option. - -### Minimal Useful Stabilization Surface - -Stabilizing *only* these three items of the Initial Smart Implementation will cover most use-cases: - - `TransmuteFrom` - - `#[derive(PromiseTransmutableFrom)]` - - `#[derive(PromiseTransmutableInto)]` - -(If the [`PromiseTransmutable` shorthand extension][extension-promisetransmutable-shorthand] is accepted, this may be further reduced to just *two* items: `TransmuteFrom` and `#[derive(PromiseTransmutable)]`.) - -Stabilizing `TransmuteOptions` and `SafeTransmuteOptions` will additionally allow end-users to build generic abstractions over `TransmuteFrom` (e.g., slice casting abstractions). - -Stabilizing `PromiseTransmutableFrom` and `PromiseTransmutableInto` will additionally allow end-users make limited stability promises, and to make stability promises for types where `derive` is too restrictive (namely, types containing `PhantomData`). - -Stabilizing `TransmuteInto` and `NeglectStability` will additionally allow end-users to implement `PromiseTransmutableFrom` in cases where the `Archetype`'s trait bounds must be repeated for lack of [implied bounds](https://github.com/rust-lang/rust/issues/44491); e.g.: -```rust -impl PromiseTransmutableFrom for GenericArray -where - T: PromiseTransmutableFrom, - N: ArrayLength, - - // for lack of implied bounds, we must repeat the bounds on `Archetype`: - GenericArray - : TransmuteInto - + PromiseTransmutableFrom, -{ - type Archetype = GenericArray; -} -``` # Drawbacks [drawbacks]: #drawbacks @@ -1391,22 +1300,18 @@ The model of stability proposed by this RFC frames stability as a quality of *sa To accommodate this, we may modify the definitions of `PromiseTransmutableFrom` and `PromiseTransmutableInto` to consume an optional `Neglect` parameter, to allow for stability declarations for unsafe transmutations: ```rust -pub trait PromiseTransmutableFrom +pub trait PromiseTransmutableFrom where Neglect: TransmuteOptions { - type Archetype - : TransmuteInto> - + PromiseTransmutableInto>; + type Archetype: TransmuteInto>; } -pub trait PromiseTransmutableInto +pub trait PromiseTransmutableInto where Neglect: TransmuteOptions { - type Archetype - : TransmuteFrom> - + PromiseTransmutableInto>; + type Archetype: TransmuteFrom>; } ``` Implementations of these traits for a given `Neglect` declares that a transmutation which is accepted while neglecting a particular set of checks (namely the set encoded by `Neglect`) will *continue* to be possible. @@ -1414,12 +1319,12 @@ Implementations of these traits for a given `Neglect` declares that a transmutat We omit these definition from this RFC's recommendations because they are not completely satisfying. For instance, `Neglect` is a *logically* unordered set of options, but is encoded as a tuple (which *is* ordered). To declare a transmutation that requires neglecting validity and alignment checks as stable, only *one* of these impls ought to be necessary: ```rust -impl PromiseTransmutableFrom<(NeglectAlignment, NeglectValidity)> for Foo +impl PromiseTransmutableFrom for Foo { ... } -impl PromiseTransmutableFrom<(NeglectValidity, NeglectAlignment)> for Foo +impl PromiseTransmutableFrom for Foo { ... } @@ -1465,21 +1370,12 @@ The compromise made by this RFC is that **`TransmuteFrom` should be stable-by-de - `Dst: TransmuteFrom` follows the usual SemVer rules, - `Dst: TransmuteFrom` *does not*. -#### Mitigating a Possible Safety Hazard -The [simplified formulation of constructability](#simplifying-constructability) provides an initially-simpler implementation path at the cost of a soundness hole. There are three possible mitigations: - - *Pretend it does not exist.* Intentional soundness holes would not bode well for this RFC's acceptance. - - *Only provide unsafe transmutation; not safe transmutation.* This option fails to remove any `unsafe` blocks from end-users' code. - - *Allow safe transmutations only when the type authors have promised they will not create a situation that would compromise safety.* **We recommend this option.** - -This promise is inherently one of stability—the type author is vowing that they will not change the implementation of their type in a way that violates the no-pub-in-priv safety invariant of safe transmutation. - - ### Why this *particular* stability system? The proposed stability system is both simple, flexible, and extensible. Whereas ensuring the soundness and safety of `TransmuteFrom` requires non-trivial compiler support, stability does not—it is realized as merely two normal traits and an `impl`. This formulation is flexible: by writing custom `Archetype`s, the [stability declaration traits][stability] make possible granular and incomplete promises of layout stability (e.g., guaranteeing the size and validity qualities of a type, but *not* its alignment. Members of the safe-transmute working group have [expressed](https://rust-lang.zulipchat.com/#narrow/stream/216762-project-safe-transmute/topic/Transmutability.20Intrinsic/near/202712834) an interest in granular stability declarations. -Finally, this formulation is extensible. The range of advance use-cases permitted by these traits is constrained only by the set of possible `Archetype`s, which, in turn, is constrained by the completeness of `TransmuteFrom`. As the implementation of `TransmuteFrom` becomes more complete, so too will the range of advance use-cases accommodated by these traits. +Finally, this formulation is extensible. The range of advance use-cases permitted by these traits is constrained only by the set of possible `Archetype`s, which, in turn, is constrained by the expressiveness of `TransmuteFrom`. As the implementation of `TransmuteFrom` becomes more complete, so too will the range of advance use-cases accommodated by these traits. ### Couldn't `#[repr(C)]` denote stability? @@ -1692,11 +1588,7 @@ The automatic mechanism proposed by [*Pre-RFC: Safe coercions*][2017-02] exploit 2. The 'location' where the coercion is made is ill-defined; the presence of the proposed `Coercible` trait may be far-removed from the location of the actual conversion (if any conversion occurs at all). 3. Field visibility stabilizes the structure of a type, but *not* its layout (e.e., its size). -Our RFC exploits the related concept of *constructability*, which is a property of a struct, or enum variant (rather than solely a property of fields). However, we recognize that it may be difficult to test for constructability within the trait resolution process. - -The simplified definition of *constructability* we propose is the same employed by [typic][crate-typic] (which uses the term "visibility"). [Typic][crate-typic] regards the pub-in-priv soundness hole of the simplified definition to be sufficiently niche that `NeglectStability` remains "safe". However, unlike [typic][crate-typic], we believe that this simplified definition imposes a safety hazard substantial enough to warrant making `NeglectStability` initially usable with *only* unsafe transmutes. - -Our RFC separates *constructability*, which concerns what aspects of a type's structure are part of its public API, and *stability*, which concerns the aspects of a type's layout that are part of its public API for SemVer purposes. This distinction does not appear in prior work. +Our RFC, [typic][crate-typic], and Haskell exploit the related concept of *constructability*. Typic uses a simplified, scope-unaware formulation of constructability that suffers from a soundness hole induced by the pub-in-priv trick. ## Prior Art: Haskell From 39e8bf8f75179699f60a6533d967b9a5ea170ebd Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Sat, 12 Sep 2020 22:13:18 -0400 Subject: [PATCH 09/18] fix error in minimal useful stabilization surface --- text/0000-safer-transmute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index d9bedc57b92..2bb6a304924 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -915,7 +915,7 @@ If `Src` is a mutatable reference, then additionally: ### Minimal Useful Stabilization Surface Stabilizing *only* these items of the Initial Smart Implementation will cover many use-cases: - `transmute!()` - - `#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)]` + - `#[derive(PromiseTransmutableFrom)]` - `#[derive(PromiseTransmutableInto)]` (If the [`PromiseTransmutable` shorthand extension][extension-promisetransmutable-shorthand] is accepted, this may be further reduced to just *two* items: `transmute!()` and `#[derive(PromiseTransmutable)]`.) From 0473af99285a5daafa2df1b61012b7d934da9634 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Sun, 13 Sep 2020 14:14:32 -0400 Subject: [PATCH 10/18] use `FromZeros` in constructability example, rather than `FromBytes` --- text/0000-safer-transmute.md | 72 +++++++++++++++++++++++++----------- 1 file changed, 50 insertions(+), 22 deletions(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 2bb6a304924..9ecbfa2662f 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -439,13 +439,6 @@ pub mod b { } ``` -The `Constructible` marker trait makes the relationship between scope and constructability explicit: -```rust -pub unsafe trait Constructible -{} -``` -A type `T` is `Constructible`, if it is fully implicitly constructible in the scope represented by the type `Scope`. These `Scope` types are generated by the `Here!()` macro, which produces a type that uniquely identifies its invocation scope. - The `transmute!` macro provides a shorthand for safely transmuting a value using its invocation scope as its reference frame: ```rust pub macro transmute($expr: expr) { @@ -455,28 +448,63 @@ pub macro transmute($expr: expr) { // ┕ the destination type of the transmute (`_` used to infer the type from context) } ``` +The `Here!()` macro produces a type that uniquely identifies its invocation scope. -The explicit `Scope` parameter of `TransmuteFrom` makes possible the creation of generic abstractions over it. For instance, consider a hypothetical `FromBytes` trait that indicates whether `Self` is safely initializable from a a sufficiently large buffer of `u8`s: +This explicit `Scope` parameter of `TransmuteFrom` makes possible the creation of generic abstractions over it. For instance, consider a hypothetical `FromZeros` trait that indicates whether `Self` is safely initializable from a a sufficiently large buffer of zero-initialized bytes: ```rust -pub unsafe trait FromBytes -{} +pub mod zerocopy { + pub unsafe trait FromZeros { + /// Safely initialize `Self` from zeroed bytes. + fn zeroed() -> Self; + } -unsafe impl FromBytes for Dst -where - Dst: TransmuteFrom<[u8; usize::MAX], ??? >, -{} + #[derive(Copy, Clone, PromiseTransmutableInto)] + #[repr(u8)] + pub enum Zero { + Zero = 0u8 + } + + unsafe impl FromZeros for Dst + where + Dst: TransmuteFrom<[Zero; usize::MAX], ??? >, + { + fn zeroed() -> Self { + [Zero; size_of::].transmute_into() + } + } +} ``` -The above definition leaves ambiguous (`???`) the scope in which the constructability of `Dst` is checked: is it the from the perspective of where this trait is defined, or where this trait is *used*? You probably do *not* intend for this trait to *only* be usable with `Dst` types that are defined in the same scope as the `FromBytes` trait! +The above definition leaves ambiguous (`???`) the scope in which the constructability of `Dst` is checked: is it the from the perspective of where this trait is defined, or where this trait is *used*? You probably do *not* intend for this trait to *only* be usable with `Dst` types that are defined in the same scope as the `FromZeros` trait! -Adding an explicit `Scope` parameter to `FromBytes` makes this unambiguous; the transmutability of `Dst` should be assessed from where the trait is used, *not* where it is defined: +Adding an explicit `Scope` parameter to `FromZeros` makes this unambiguous; the transmutability of `Dst` should be assessed from where the trait is used, *not* where it is defined: ```rust -pub unsafe trait FromBytes -{} +pub unsafe trait FromZeros { + /// Safely initialize `Self` from zeroed bytes. + fn zeroed() -> Self; +} -unsafe impl FromBytes for Dst +unsafe impl FromZeros for Dst where - Dst: TransmuteFrom<[u8; usize::MAX], Scope>, -{} + Dst: TransmuteFrom<[Zero; usize::MAX], Scope> +{ + fn zeroed() -> Self { + [Zero; size_of::].transmute_into() + } +} +``` + +A thid-party could then use `FromZeros` like so: +```rust +use zerocopy::FromZeros; + +#[derive(PromiseTransmutableInto)] +#[repr(C)] +struct Foo { + ... +} + +// Initialize `Foo` from zero-initialized bytes. +let _: Foo = FromZeros::<_, Here!()>::zeroed(); ``` @@ -849,7 +877,7 @@ let _ = ::unsafe_transmute_f ## Implementation Guidance Three items in this RFC require special compiler support: - `Here!()` - - `Constructible` + - `Constructible` (a private implementation detail of `TransmuteFrom`) - `TransmuteFrom` ### Implementing `Here!()` From 44a2d41d88ca937d2f2b69e8ebedd78b47e57c76 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Sun, 13 Sep 2020 14:16:56 -0400 Subject: [PATCH 11/18] remove unnecessary `pub` --- text/0000-safer-transmute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 9ecbfa2662f..73f1003d488 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -460,7 +460,7 @@ pub mod zerocopy { #[derive(Copy, Clone, PromiseTransmutableInto)] #[repr(u8)] - pub enum Zero { + enum Zero { Zero = 0u8 } From be8c6efc00efbb0d31369a48d7a80bba482fdf6a Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 14 Sep 2020 10:56:40 -0400 Subject: [PATCH 12/18] fix typo Co-authored-by: Christopher Durham --- text/0000-safer-transmute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 73f1003d488..f8ee738f226 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -474,7 +474,7 @@ pub mod zerocopy { } } ``` -The above definition leaves ambiguous (`???`) the scope in which the constructability of `Dst` is checked: is it the from the perspective of where this trait is defined, or where this trait is *used*? You probably do *not* intend for this trait to *only* be usable with `Dst` types that are defined in the same scope as the `FromZeros` trait! +The above definition leaves ambiguous (`???`) the scope in which the constructability of `Dst` is checked: is it from the perspective of where this trait is defined, or where this trait is *used*? You probably do *not* intend for this trait to *only* be usable with `Dst` types that are defined in the same scope as the `FromZeros` trait! Adding an explicit `Scope` parameter to `FromZeros` makes this unambiguous; the transmutability of `Dst` should be assessed from where the trait is used, *not* where it is defined: ```rust From a3250788d84dd3afbc5dede08ec9602fedd634a0 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 12 Oct 2020 12:46:54 -0400 Subject: [PATCH 13/18] generate scope types with `pub(self)` visibility https://github.com/rust-lang/rfcs/pull/2981#discussion_r502977870 --- text/0000-safer-transmute.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index f8ee738f226..c8f016d7453 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -912,6 +912,8 @@ impl Foo for ! { ``` Scope types should (as much as possible) pretty print in compiler error messages as their definition path. +These scope types should generated with `pub(self)` visibility. We are not currently aware of any reason why publicly re-exporting a scope type via a type alias would be a good idea; restricting the visibility of these types will warn users against doing so. If compelling use-cases for re-exported scope types are discovered in the future, a broader visibility could be used instead without breaking backwards compatibility. + ### Implementing `Constructible` The compiler implements `Constructible` for `T` if `T` is fully implicitly constructible in the scope uniquely identified by the type `Scope`. From bb9b988c10a60d714ad52c9804aad36961960736 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 12 Oct 2020 13:15:46 -0400 Subject: [PATCH 14/18] introduce `differing_sizes` lint https://github.com/rust-lang/rfcs/pull/2981#discussion_r481408491 --- text/0000-safer-transmute.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index c8f016d7453..6f59ef9f297 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -203,6 +203,8 @@ A value may ***not*** be transmuted into a type of greater size, if doing so wou let _ : [u8; 32] = transmute!([u8; 16]::default()); // Compile Error! ``` +A `differing_sizes` lint reports warnings for invocations of `transmute!()` where the source and destination types are different sizes. + #### Requirements on References [transmute-references]: #requirements-on-references @@ -941,6 +943,8 @@ Unless `NeglectConstructability` is used as `Neglect` option, a `Src` is *safely If `Src` is a mutatable reference, then additionally: 1. `Src: Constructible` +### Implementing `differing_sizes` +The `differing_sizes` lint reports a compiler warning when the source and destination types of a `transmute!()`, `transmute_into` or `transmute_from` invocation differ. This lint shall be warn-by-default. ### Minimal Useful Stabilization Surface Stabilizing *only* these items of the Initial Smart Implementation will cover many use-cases: From ec9eb45a07477ca7f8438d072854bf8699a2e3bb Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Fri, 16 Oct 2020 14:10:52 -0400 Subject: [PATCH 15/18] Remove the stability system and `TransmuteInto` (for now). The stability system fails to match the usual rules of stability: https://github.com/rust-lang/rfcs/pull/2981#issuecomment-710033947 https://github.com/rust-lang/rfcs/pull/2981#issuecomment-710048751 Per the rationale here: https://github.com/rust-lang/rfcs/pull/2981#issuecomment-710263596 this commit merely removes the stability system, rather than attempting to replace it. It's hard to craft a stability system that isn't overly-prescriptive. I anticipate a not-insignificant amount of design work will be required to overcome the pitfalls I describe in that comment. It might very well happen, but it will take time. Separately, this commit removes `TransmutableInto` (for now). `TransmuteInto` was necessitated by the `PromiseTransmutableFrom` trait. Without that trait, it's just an ergonomic nicety. It *is* nice to have and may very well make a reappearance, but in this time of flux, it's just an extra trait that requires labor to keep it up-to-date with the current state of the RFC. --- text/0000-safer-transmute.md | 687 +++-------------------------------- 1 file changed, 51 insertions(+), 636 deletions(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 6f59ef9f297..c7586dbe26e 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -9,14 +9,8 @@ # Summary [summary]: #summary -We propose traits, namely `TransmuteInto` and `TransmuteFrom`, that are implemented *automatically* for combinations of types that may be safely transmuted. In other words, this RFC makes safe transmutation *as easy as 1..., 2..., `repr(C)`!* +We propose traits, namely `TransmuteFrom`, that are implemented *automatically* for combinations of types that may be safely transmuted. In other words, this RFC makes safe transmutation *as easy as 1..., 2..., `repr(C)`!* ```rust -use core::transmute::{ - transmute, - stability::{PromiseTransmutableInto, PromiseTransmutableFrom}, -}; - -#[derive(PromiseTransmutableInto, PromiseTransmutableFrom)] // declare `Foo` to be *stably* transmutable #[repr(C)] pub struct Foo(pub u8, pub u16); // ^ there's a padding byte here, between these fields @@ -27,9 +21,9 @@ let _ : Foo = transmute!(64u32); // Alchemy Achieved! let _ : u32 = transmute!(Foo(16, 12)); // Compile Error! // error[E0277]: the trait bound `u32: TransmuteFrom` is not satisfied -// --> src/demo.rs:14:27 +// --> src/demo.rs:7:27 // | -// 14 | let _ : u32 = transmute!(Foo(16, 12)); // Compile Error! +// 7 | let _ : u32 = transmute!(Foo(16, 12)); // Compile Error! // | ^^^^^^^^^^^ the trait `TransmuteFrom` is not implemented for `u32` // | // = note: byte 8 of the source type may be uninitialized; byte 8 of the destination type cannot be uninitialized. @@ -43,7 +37,6 @@ Byte-reinterpretation conversions (such as those performed by `mem::transmute`, This RFC's comprehensive approach provides additional benefits beyond the mere act of transmutation; namely: - [authoritatively codifies language layout guarantees](#codifying-language-layout-guarantees) - - [allows crate authors to codify their types' layout stability guarantees](#expressing-library-layout-guarantees) - [allows crate authors to codify their abstractions' layout requirements](#expressing-layout-requirements) Given the expressive foundation provided by this RFC, we also envision a range of future possibilities that will *not* require additional compiler support, including: @@ -58,14 +51,7 @@ Given the expressive foundation provided by this RFC, we also envision a range o ## Codifying Language Layout Guarantees Documentation of Rust's layout guarantees for a type are often spread across countless issues, pull requests, RFCs and various official resources. It can be very difficult to get a straight answer. When transmutation is involved, users must reason about the *combined* layout properties of the source and destination types. -This RFC proposes mechanisms that programmers will use to confidently answer such questions—by checking whether the `TransmuteFrom` and `TransmuteInto` traits are implemented. - -## Expressing Library Layout Guarantees -There is no canonical way for crate authors to declare the SemVer layout guarantees of their types. Crate authors currently must state their layout guarantees using prose in their documentation. In contrast to structural stability (e.g., the declared visibility of fields), layout stability is expressed extra-linguistically. - -This isn't satisfactory: guarantees expressed in prose outside of the Rust programming language are guarantees that cannot be reasoned about *inside* the language. Whereas `rustc` can dutifully deny programmers access to private fields, it is unable to prevent programmers from making unfounded expectations of types' in-memory layouts. - -This RFC proposes simple-but-powerful [mechanisms][stability] for declaring layout stability guarantees. +This RFC proposes mechanisms that programmers will use to confidently answer such questions—by checking whether the `TransmuteFrom` trait is implemented. ## Expressing Layout Requirements Similarly, there is no canonical way for crate authors to declare the layout requirements of generic abstractions over types that have certain layout properties. @@ -96,7 +82,7 @@ unsafe fn transmute(src: Src) -> Dst ``` ### 📖 Safer Transmutation -By **safer transmutation** we mean: *what `where` bound could we add to `transmute` restricts its type parameters `Src` and `Dst` in ways that statically limit the function's misuse?* Our answer to this question will ensure that transmutations are, by default, *well-defined*, *safe* and *stable*. +By **safer transmutation** we mean: *what `where` bound could we add to `transmute` restricts its type parameters `Src` and `Dst` in ways that statically limit the function's misuse?* Our answer to this question will ensure that transmutations are, by default, *well-defined* and *safe*. ### 📖 Well-Definedness A transmutation is ***well-defined*** if the mere act of transmuting a value from one type to another is not unspecified or undefined behavior. @@ -104,12 +90,9 @@ A transmutation is ***well-defined*** if the mere act of transmuting a value fro ### 📖 Safety A well-defined transmutation is ***safe*** if *using* the transmuted value cannot violate memory safety. -### 📖 Stability -A safe transmutation is ***stable*** if the authors of the source type and destination types have indicated that the layouts of those types is part of their libraries' stability guarantees. - ## Concepts in Depth -***Disclaimer:** While the high-level definitions of transmutation well-definedness, safety and stability are a core component of this RFC, the detailed rules and examples in this section are **not**. We expect that the initial implementation of `TransmuteFrom` may initially be considerably less sophisticated than the examples in this section (and thus forbid valid transmutations). Nonetheless, this section explores nuanced cases of transmutation well-definedness and safety to demonstrate that the APIs we propose can grow to handle that nuance.* +***Disclaimer:** While the high-level definitions of transmutation well-definedness and safety is a core component of this RFC, the detailed rules and examples in this section are **not**. We expect that the initial implementation of `TransmuteFrom` may initially be considerably less sophisticated than the examples in this section (and thus forbid valid transmutations). Nonetheless, this section explores nuanced cases of transmutation well-definedness and safety to demonstrate that the APIs we propose can grow to handle that nuance.* ### 📖 When is a transmutation well-defined? @@ -143,7 +126,6 @@ The bits of any valid instance of the source type must be a bit-valid instance o For example, we are permitted to transmute a `Bool` into a [`u8`]: ```rust -#[derive(Default, PromiseTransmutableFrom, PromiseTransmutableInto)] #[repr(u8)] enum Bool { True = 1, @@ -153,9 +135,6 @@ enum Bool { let _ : u8 = transmute!(Bool::True); let _ : u8 = transmute!(Bool::False); ``` - -(Note: #[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] annotation connotes that all aspects of Bool's layout are part of its library stability guarantee.) - ...because all possible instances of `Bool` are also valid instances of [`u8`]. However, transmuting a [`u8`] into a `Bool` is forbidden: ```rust @@ -167,11 +146,11 @@ let _ : Bool = transmute!(u8::default()); // Compile Error! Another example: While laying out certain types, Rust may insert padding bytes between the layouts of fields. In the below example `Padded` has two padding bytes, while `Packed` has none: ```rust #[repr(C)] -#[derive(Default, PromiseTransmutableFrom, PromiseTransmutableInto)] +#[derive(Default)] struct Padded(pub u8, pub u16, pub u8); #[repr(C)] -#[derive(Default, PromiseTransmutableFrom, PromiseTransmutableInto)] +#[derive(Default)] struct Packed(pub u16, pub u16, pub u16); assert_eq!(mem::size_of::(), mem::size_of::()); @@ -460,7 +439,7 @@ pub mod zerocopy { fn zeroed() -> Self; } - #[derive(Copy, Clone, PromiseTransmutableInto)] + #[derive(Copy, Clone)] #[repr(u8)] enum Zero { Zero = 0u8 @@ -499,7 +478,6 @@ A thid-party could then use `FromZeros` like so: ```rust use zerocopy::FromZeros; -#[derive(PromiseTransmutableInto)] #[repr(C)] struct Foo { ... @@ -510,156 +488,9 @@ let _: Foo = FromZeros::<_, Here!()>::zeroed(); ``` -### 📖 When is a transmutation stable? -[stability]: #-when-is-a-transmutation-stable - -Since the well-definedness and safety of a transmutation is affected by the layouts of the source and destination types, changes to those types' layouts may cause code which previously compiled to produce errors. In other words, transmutation causes a type's layout to become part of that type's API for the purposes of SemVer stability. - -The question is, then: *how can the author of a type reason about transmutations they did not write, from-or-to types they did not write?* We address this problem by introducing two traits which both allow an author to opt-in to stability guarantees for their types, and allow third-parties to reason at compile-time about what guarantees are provided for such types. - -#### `PromiseTransmutableFrom` and `PromiseTransmutableInto` - -You may declare the stability guarantees of your type by implementing one or both of two traits: -```rust -pub trait PromiseTransmutableFrom -{ - type Archetype: TransmuteInto; -} - -pub trait PromiseTransmutableInto -{ - type Archetype: TransmuteFrom; -} -``` - -To implement each of these traits, you must specify an `Archetype`. An `Archetype` is a type whose layout exemplifies the extremities of your stability promise (i.e., the least/most constrained type for which it is valid to transmute your type into/from). - -By implementing `PromiseTransmutableFrom`, you promise that your type is guaranteed to be safely transmutable *from* `PromiseTransmutableFrom::Archetype`. Conversely, by implementing `PromiseTransmutableInto`, you promise that your type is guaranteed to be safely transmutable *into* `PromiseTransmutableInto::Archetype`. - -You are free to change the layout of your type however you like between minor crate versions so long as that change does not violates these promises. These two traits are capable of expressing simple and complex stability guarantees. - -#### Stability & Transmutation -Together with the `PromiseTransmutableFrom` and `PromiseTransmutableInto` traits, this impl of `TransmuteFrom` constitutes the formal definition of transmutation stability: -```rust -unsafe impl TransmuteFrom for Dst -where - Src: PromiseTransmutableInto, - Dst: PromiseTransmutableFrom, - - Dst::Archetype: TransmuteFrom -{} -``` -Why is this safe? Can we really safely judge whether `Dst` is transmutable from `Src` by assessing the transmutability of two different types? Yes! Transmutability is *transitive*. Concretely, if we can safely transmute: - - `Src` to `Src::Archetype` (enforced by `Src: PromiseTransmutableInto`), and - - `Dst::Archetype` to `Dst` (enforced by `Dst: PromiseTransmutableFrom`), and - - `Src::Archetype` to `Dst::Archetype` (enforced by `Dst::Archetype: TransmuteFrom`), - -...then it follows that we can safely transmute `Src` to `Dst` in three steps: - 1. we transmute `Src` to `Src::Archetype`, - 2. we transmute `Src::Archetype` to `Dst::Archetype`, - 3. we transmute `Dst::Archetype` to `Dst`. - -#### Common Use-Case: As-Stable-As-Possible -[stability-common]: #common-use-case-as-stable-as-possible -To promise that all transmutations which are currently safe for your type will remain so in the future, simply annotate your type with: -```rust -#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] -#[repr(C)] -pub struct Foo(pub Bar, pub Baz); -``` -This expands to: -```rust -#[repr(C)] -pub struct Foo(pub Bar, pub Baz); - -/// Generated `PromiseTransmutableFrom` for `Foo` -const _: () = { - use core::transmute::stability::PromiseTransmutableFrom; - - #[repr(C)] - pub struct TransmutableFromArchetype( - pub >::Archetype, - pub >::Archetype, - ); - - impl PromiseTransmutableFrom for TransmutableFromArchetype { - type Archetype = Self; - } - - impl PromiseTransmutableFrom for Foo { - type Archetype = TransmutableFromArchetype; - } -}; - -/// Generated `PromiseTransmutableInto` for `Foo` -const _: () = { - use core::transmute::stability::PromiseTransmutableInto; - - #[repr(C)] - pub struct TransmutableIntoArchetype( - pub >::Archetype, - pub >::Archetype, - ); - - impl PromiseTransmutableInto for TransmutableIntoArchetype { - type Archetype = Self; - } - - impl PromiseTransmutableInto for Foo { - type Archetype = TransmutableIntoArchetype; - } -}; -``` -Since deriving *both* of these traits together is, by far, the most common use-case, we [propose][extension-promisetransmutable-shorthand] `#[derive(PromiseTransmutable)]` as an ergonomic shortcut. - - -#### Uncommon Use-Case: Weak Stability Guarantees -[stability-uncommon]: #uncommon-use-case-weak-stability-guarantees - -We also can specify *custom* `Archetype`s to finely constrain the set of transmutations we are willing to make stability promises for. Consider, for instance, if we want to leave ourselves the future leeway to change the alignment of a type `Foo` without making a SemVer major change: -```rust -#[repr(C)] -pub struct Foo(pub Bar, pub Baz); -``` -The alignment of `Foo` affects transmutability of `&Foo`. A `&Foo` cannot be safely transmuted from a `&Bar` if the alignment requirements of `Foo` exceed those of `Bar`. If we don't want to promise that `&Foo` is stably transmutable from virtually *any* `Bar`, we simply make `Foo`'s `PromiseTransmutableFrom::Archetype` a type with maximally strict alignment requirements: -```rust -const _: () = { - use core::transmute::stability::PromiseTransmutableFrom; - - // 2^29 is currently the maximum alignment - #[repr(C, align(536870912))] - pub struct TransmutableFromArchetype( - pub >::Archetype, - pub >::Archetype, - ); - - impl PromiseTransmutableFrom for Foo { - type Archetype = TransmutableFromArchetype; - } -}; -``` -Conversely, a `&Foo` cannot be safely transmuted *into* a `&Bar` if the alignment requirements of `Bar` exceed those of `Foo`. We reduce this set of stable transmutations by making `PromiseTransmutableFrom::Archetype` a type with minimal alignment requirements: -```rust -const _: () = { - use core::transmute::stability::PromiseTransmutableInto; - - #[repr(C, packed(1))] - pub struct TransmutableIntoArchetype( - pub >::Archetype, - pub >::Archetype, - ); - - impl PromiseTransmutableInto for Foo { - type Archetype = TransmutableIntoArchetype; - } -}; -``` -Given these two stability promises, we are free to modify the alignment of `Foo` in SemVer-minor changes without running any risk of breaking dependent crates. - - ## Mechanisms of Transmutation -Two traits provide mechanisms for transmutation between types: +The `TransmuteFrom` trait provides the fundamental mechanism checking the transmutability of types: ```rust // this trait is implemented automagically by the compiler #[lang = "transmute_from"] @@ -697,33 +528,6 @@ where } } } - -// implemented in terms of `TransmuteFrom` -pub unsafe trait TransmuteInto -where - Neglect: TransmuteOptions, -{ - fn transmute_into(self) -> Dst - where - Self: Sized, - Dst: Sized, - Neglect: SafeTransmuteOptions; - - unsafe fn unsafe_transmute_into(self) -> Dst - where - Self: Sized, - Dst: Sized, - Neglect: TransmuteOptions; -} - -unsafe impl TransmuteInto for Src -where - Src: ?Sized, - Dst: ?Sized + TransmuteFrom, - Neglect: TransmuteOptions, -{ - ... -} ``` In the above definitions, `Src` represents the source type of the transmutation, `Dst` represents the destination type of the transmutation, and `Neglect` is a parameter that [encodes][options] which static checks the compiler ought to neglect when considering if a transmutation is valid. The default value of `Neglect` is `()`, which reflects that, by default, the compiler does not neglect *any* static checks. @@ -731,15 +535,14 @@ In the above definitions, `Src` represents the source type of the transmutation, ### Neglecting Static Checks [options]: #Neglecting-Static-Checks -The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are unsafe, ill-defined, or unstable. However, you may explicitly opt-out of some static checks; e.g.: +The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are ill-defined or unsafe. However, you may explicitly opt-out of some static checks; e.g.: | Transmute Option | Usable With | |---------------------|---------------------------------------------------------| -| `NeglectStabilty` | `transmute_{from,into}`, `unsafe_transmute_{from,into}` | | `NeglectAlignment` | `unsafe_transmute_{from,into}` | | `NeglectValidity` | `unsafe_transmute_{from,into}` | -`NeglectStabilty` implements the `SafeTransmuteOptions` and `TransmuteOptions` marker traits, as it can be used in both safe and unsafe code. The selection of multiple options is encoded by grouping them as a tuple; e.g., `(NeglectAlignment, NeglectValidity)` is a selection of both the `NeglectAlignment` and `NeglectValidity` options. +The selection of multiple options is encoded by grouping them as a tuple; e.g., `(NeglectAlignment, NeglectValidity)` is a selection of both the `NeglectAlignment` and `NeglectValidity` options. We introduce two marker traits which serve to group together the options that may be used with safe transmutes, and those which may be used with `unsafe` transmutes: ```rust @@ -753,36 +556,10 @@ impl SafeTransmuteOptions for () {} impl TransmuteOptions for () {} ``` -#### `NeglectStability` -[`NeglectStability`]: #neglectstability - -By default, `TransmuteFrom` and `TransmuteInto`'s methods require that the [layouts of the source and destination types are SemVer-stable][stability]. The `NeglectStability` option disables this requirement. -```rust -pub struct NeglectStability; - -impl SafeTransmuteOptions for NeglectStability {} -impl TransmuteOptions for NeglectStability {} -``` - -Prior to the adoption of the [stability declaration traits][stability], crate authors documented the layout guarantees of their types with doc comments. The `TransmuteFrom` and `TransmuteInto` traits and methods may be used with these types by requesting that the stability check is neglected; for instance: - -```rust -fn serialize(val : LibraryType, dst: W) -> std::io::Result<()> -where - LibraryType: TransmuteInto<[u8; size_of::()], Scope, NeglectStability> -{ - ... -} -``` - -Neglecting stability over-eagerly cannot cause ill-definedness or unsafety. For this reason, it is the only transmutation option available on the safe methods `transmute_from` and `transmute_into`. However, neglecting stability over-eagerly may cause your code to behave incorrectly or cease compiling if the authors of the source and destination types make changes that affect their layout. - -By using the `NeglectStability` option to transmute types you do not own, you are committing to ensure that your reliance on these types' layouts is consistent with their documented stability guarantees. - #### `NeglectAlignment` [ext-ref-casting]: #NeglectAlignment -By default, `TransmuteFrom` and `TransmuteInto`'s methods require that, when transmuting references, the minimum alignment of the destination's referent type is no greater than the minimum alignment of the source's referent type. The `NeglectAlignment` option disables this requirement. +By default, `TransmuteFrom`'s methods require that, when transmuting references, the minimum alignment of the destination's referent type is no greater than the minimum alignment of the source's referent type. The `NeglectAlignment` option disables this requirement. ```rust pub struct NeglectAlignment; @@ -795,24 +572,23 @@ By using the `NeglectAlignment` option, you are committing to ensure that the tr /// /// This produces `None` if the referent isn't appropriately /// aligned, as required by the destination type. -pub fn try_cast_ref<'t, 'u, Scope, T, U>(src: &'t T) -> Option<&'u U> +pub fn try_cast_ref<'t, 'u, T, U, Scope>(src: &'t T) -> Option<&'u U> where - &'t T: TransmuteInto<&'u U, Scope, NeglectAlignment>, + &'t T: TransmuteFrom<&'u U, Scope, NeglectAlignment>, { if (src as *const T as usize) % align_of::() != 0 { None } else { // Safe because we dynamically enforce the alignment // requirement, whose static check we chose to neglect. - Some(unsafe { src.unsafe_transmute_into() }) + Some(unsafe { TransmuteFrom::unsafe_transmute_from(src) }) } } ``` #### `NeglectValidity` -By default, `TransmuteFrom` and `TransmuteInto`'s methods require that all instantiations of the source type are guaranteed to be valid instantiations of the destination type. This precludes transmutations which *might* be valid depending on the source value: +By default, `TransmuteFrom`'s methods require that all instantiations of the source type are guaranteed to be valid instantiations of the destination type. This precludes transmutations which *might* be valid depending on the source value: ```rust -#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] #[repr(u8)] enum Bool { True = 1, @@ -831,7 +607,6 @@ impl TransmuteOptions for NeglectValidity {} By using the `NeglectValidity` option, you are committing to ensure dynamically source value is a valid instance of the destination type. For instance: ```rust -#[derive(PromiseTransmutableFrom, PromiseTransmutableInto)] #[repr(u8)] enum Bool { True = 1, @@ -845,18 +620,18 @@ pub trait TryIntoBool impl TryIntoBool for T where - T: TransmuteInto, - u8: TransmuteInto + u8: TransmuteFrom, + Bool: TransmuteFrom { fn try_into_bool(self) -> Option { - let val: u8 = self.transmute_into(); + let val: u8 = TransmuteFrom::transmute_from(self); if val > 1 { None } else { // Safe, because we've first verified that // `val` is a bit-valid instance of a boolean. - Some(unsafe {val.unsafe_transmute_into()}) + Some(unsafe {TransmuteFrom::unsafe_transmute_from(val)}) } } } @@ -864,10 +639,8 @@ where Even with `NeglectValidity`, the compiler will statically reject transmutations that cannot possibly be valid: ```rust -#[derive(PromiseTransmutableInto)] #[repr(C)] enum Foo { A = 24 } -#[derive(PromiseTransmutableFrom)] #[repr(C)] enum Bar { Z = 42 } let _ = ::unsafe_transmute_from(Foo::N) // Compile error! @@ -929,12 +702,6 @@ The `Constructible` trait does not ever need to be made stable, or even visible ### Implementing `TransmuteFrom` The implementation of `TransmuteFrom` is completely internal to the compiler (à la [`Sized`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_sized) and [`Freeze`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_freeze)). -#### Stability and Transmutation -Unless `NeglectStability` is used as a `Neglect` option, a `Src` is *stably* transmutable into `Dst` in a given `Scope` if: - 1. `Src: PromiseTransmutableInto` - 2. `Dst: PromiseTransmutableFrom` - 3. `Dst::Archetype: TransmuteFrom` - #### Constructability and Transmutation Unless `NeglectConstructability` is used as `Neglect` option, a `Src` is *safely* transmutable into `Dst` in a given `Scope` if: 1. `Dst: Constructible` @@ -947,110 +714,36 @@ If `Src` is a mutatable reference, then additionally: The `differing_sizes` lint reports a compiler warning when the source and destination types of a `transmute!()`, `transmute_into` or `transmute_from` invocation differ. This lint shall be warn-by-default. ### Minimal Useful Stabilization Surface -Stabilizing *only* these items of the Initial Smart Implementation will cover many use-cases: +Stabilizing *only* this subset of the Initial Smart Implementation will cover many use-cases: - `transmute!()` - - `#[derive(PromiseTransmutableFrom)]` - - `#[derive(PromiseTransmutableInto)]` - -(If the [`PromiseTransmutable` shorthand extension][extension-promisetransmutable-shorthand] is accepted, this may be further reduced to just *two* items: `transmute!()` and `#[derive(PromiseTransmutable)]`.) To define traits that generically abstract over `TransmuteFrom`, these items must be stabilized: - `Here!()` - `TransmuteFrom` - `TransmuteOptions` and `SafeTransmuteOptions` -Stabilizing `PromiseTransmutableFrom` and `PromiseTransmutableInto` will additionally allow end-users make limited stability promises, and to make stability promises for types where `derive` is too restrictive (namely, types containing `PhantomData`). - -Additionally stabilizing `TransmuteInto` and `NeglectStability` will additionally allow end-users to implement `PromiseTransmutableFrom` and `PromiseTransmutableInto` in cases where the `Archetype`'s trait bounds must be repeated for lack of [implied bounds](https://github.com/rust-lang/rust/issues/44491); e.g.: -```rust -impl PromiseTransmutableFrom for GenericArray -where - T: PromiseTransmutableFrom, - N: ArrayLength, - - // for lack of implied bounds, we must repeat the bounds on `Archetype`, - // which requires naming `NeglectStability` - GenericArray: TransmuteInto -{ - type Archetype = GenericArray; -} -``` - ### Complete API Surface [minimal-impl]: #Listing-for-Initial-Minimal-Implementation -This listing is both a **canonical specification** of this RFC's API surface ([playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=ec5c618d843acd7c40855ebfb386d49a)): +This listing is the **canonical specification** of this RFC's API surface ([playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0a4dda2760244110bcf829c298c45c34)): ```rust #![feature(untagged_unions,const_fn,const_fn_union)] // for the impl of unsafe_transmute_from -#![feature(const_generics)] // for stability declarations on `[T; N]` #![feature(decl_macro)] // for stub implementations of derives -#![feature(never_type)] // for stability declarations on `!` #![allow(warnings)] /// Transmutation conversions. // suggested location: `core::convert` pub mod transmute { - use {options::*, stability::*}; + use options::*; - /// Safely and stably transmute $expr + /// Safely transmute $expr pub macro transmute($expr: expr) { TransmuteFrom::<_, Here!()>::transmute_from($expr) } - /// `Self: TransmuteInto - where - Neglect: TransmuteOptions, - { - /// Reinterpret the bits of a value of one type as another type, safely. - fn transmute_into(self) -> Dst - where - Self: Sized, - Dst: Sized, - Neglect: SafeTransmuteOptions; - - /// Reinterpret the bits of a value of one type as another type, potentially unsafely. - /// - /// The onus is on you to ensure that calling this method is safe. - unsafe fn unsafe_transmute_into(self) -> Dst - where - Self: Sized, - Dst: Sized, - Neglect: TransmuteOptions; - } - - unsafe impl TransmuteInto for Src - where - Src: ?Sized, - Dst: ?Sized + TransmuteFrom, - Neglect: TransmuteOptions, - { - #[inline(always)] - fn transmute_into(self) -> Dst - where - Self: Sized, - Dst: Sized, - Neglect: SafeTransmuteOptions, - { - Dst::transmute_from(self) - } - - #[inline(always)] - unsafe fn unsafe_transmute_into(self) -> Dst - where - Self: Sized, - Dst: Sized, - Neglect: TransmuteOptions, - { - unsafe { Dst::unsafe_transmute_from(self) } - } - } - - /// `Self: TransmuteInto where @@ -1084,7 +777,7 @@ pub mod transmute { src: ManuallyDrop, dst: ManuallyDrop, } - + unsafe { ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst) } @@ -1098,168 +791,6 @@ pub mod transmute { Neglect: TransmuteOptions {} - /// Traits for declaring the SemVer stability of types. - pub mod stability { - - use super::{TransmuteFrom, TransmuteInto, options::NeglectStability}; - - /// Declare that transmuting `Self` into `Archetype` is SemVer-stable. - pub trait PromiseTransmutableInto { - /// The `Archetype` must be safely transmutable from `Self`. - type Archetype: TransmuteFrom; - } - - /// Declare that transmuting `Self` from `Archetype` is SemVer-stable. - pub trait PromiseTransmutableFrom - { - /// The `Archetype` must be safely transmutable into `Self`. - type Archetype: TransmuteInto; - } - - - /// Derive macro generating an impl of the trait `PromiseTransmutableInto`. - //#[rustc_builtin_macro] - pub macro PromiseTransmutableInto($item:item) { - /* compiler built-in */ - } - - /// Derive macro generating an impl of the trait `PromiseTransmutableFrom`. - //#[rustc_builtin_macro] - pub macro PromiseTransmutableFrom($item:item) { - /* compiler built-in */ - } - - - impl PromiseTransmutableInto for ! {type Archetype = Self;} - impl PromiseTransmutableFrom for ! {type Archetype = Self;} - impl PromiseTransmutableInto for () {type Archetype = Self;} - impl PromiseTransmutableFrom for () {type Archetype = Self;} - - impl PromiseTransmutableInto for f32 {type Archetype = Self;} - impl PromiseTransmutableFrom for f32 {type Archetype = Self;} - impl PromiseTransmutableInto for f64 {type Archetype = Self;} - impl PromiseTransmutableFrom for f64 {type Archetype = Self;} - - impl PromiseTransmutableInto for i8 {type Archetype = Self;} - impl PromiseTransmutableFrom for i8 {type Archetype = Self;} - impl PromiseTransmutableInto for i16 {type Archetype = Self;} - impl PromiseTransmutableFrom for i16 {type Archetype = Self;} - impl PromiseTransmutableInto for i32 {type Archetype = Self;} - impl PromiseTransmutableFrom for i32 {type Archetype = Self;} - impl PromiseTransmutableInto for i64 {type Archetype = Self;} - impl PromiseTransmutableFrom for i64 {type Archetype = Self;} - impl PromiseTransmutableInto for i128 {type Archetype = Self;} - impl PromiseTransmutableFrom for i128 {type Archetype = Self;} - impl PromiseTransmutableInto for isize {type Archetype = Self;} - impl PromiseTransmutableFrom for isize {type Archetype = Self;} - - impl PromiseTransmutableInto for u8 {type Archetype = Self;} - impl PromiseTransmutableFrom for u8 {type Archetype = Self;} - impl PromiseTransmutableInto for u16 {type Archetype = Self;} - impl PromiseTransmutableFrom for u16 {type Archetype = Self;} - impl PromiseTransmutableInto for u32 {type Archetype = Self;} - impl PromiseTransmutableFrom for u32 {type Archetype = Self;} - impl PromiseTransmutableInto for u64 {type Archetype = Self;} - impl PromiseTransmutableFrom for u64 {type Archetype = Self;} - impl PromiseTransmutableInto for u128 {type Archetype = Self;} - impl PromiseTransmutableFrom for u128 {type Archetype = Self;} - impl PromiseTransmutableInto for usize {type Archetype = Self;} - impl PromiseTransmutableFrom for usize {type Archetype = Self;} - - use core::marker::PhantomData; - impl PromiseTransmutableInto for PhantomData { type Archetype = Self; } - impl PromiseTransmutableFrom for PhantomData { type Archetype = Self; } - - - impl PromiseTransmutableInto for [T; N] - where - T: PromiseTransmutableInto, - [T::Archetype; N]: TransmuteFrom - { - type Archetype = [T::Archetype; N]; - } - - impl PromiseTransmutableFrom for [T; N] - where - T: PromiseTransmutableFrom, - [T::Archetype; N]: TransmuteInto - { - type Archetype = [T::Archetype; N]; - } - - - impl PromiseTransmutableInto for *const T - where - T: PromiseTransmutableInto, - *const T::Archetype: TransmuteFrom - { - type Archetype = *const T::Archetype; - } - - impl PromiseTransmutableFrom for *const T - where - T: PromiseTransmutableFrom, - *const T::Archetype: TransmuteInto - { - type Archetype = *const T::Archetype; - } - - - impl PromiseTransmutableInto for *mut T - where - T: PromiseTransmutableInto, - *mut T::Archetype: TransmuteFrom - { - type Archetype = *mut T::Archetype; - } - - impl PromiseTransmutableFrom for *mut T - where - T: PromiseTransmutableFrom, - *mut T::Archetype: TransmuteInto, - { - type Archetype = *mut T::Archetype; - } - - - impl<'a, T: ?Sized, Scope> PromiseTransmutableInto for &'a T - where - T: PromiseTransmutableInto, - T::Archetype: 'a, - &'a T::Archetype: TransmuteFrom<&'a T, Scope, NeglectStability> - { - type Archetype = &'a T::Archetype; - } - - impl<'a, T: ?Sized, Scope> PromiseTransmutableFrom for &'a T - where - T: PromiseTransmutableFrom, - T::Archetype: 'a, - &'a T::Archetype: TransmuteInto<&'a T, Scope, NeglectStability> - { - type Archetype = &'a T::Archetype; - } - - - impl<'a, T: ?Sized, Scope> PromiseTransmutableInto for &'a mut T - where - T: PromiseTransmutableInto, - T::Archetype: 'a, - &'a mut T::Archetype: TransmuteFrom<&'a mut T, Scope, NeglectStability> - { - type Archetype = &'a mut T::Archetype; - } - - impl<'a, T: ?Sized, Scope> PromiseTransmutableFrom for &'a mut T - where - T: PromiseTransmutableFrom, - T::Archetype: 'a, - &'a mut T::Archetype: TransmuteInto<&'a mut T, Scope, NeglectStability> - { - type Archetype = &'a mut T::Archetype; - } - } - /// Static checks that may be neglected when determining if a type is `TransmuteFrom` some other type. pub mod options { @@ -1272,11 +803,18 @@ pub mod transmute { impl SafeTransmuteOptions for () {} impl TransmuteOptions for () {} - /// Neglect the stability check of `TransmuteFrom`. - pub struct NeglectStability; + /// Neglect the alignment check of `TransmuteFrom`. + pub struct NeglectAlignment; + + /// Neglect the validity check of `TransmuteFrom`. + pub struct NeglectValidity; + + /* additional options */ - impl SafeTransmuteOptions for NeglectStability {} - impl TransmuteOptions for NeglectStability {} + impl TransmuteOptions for NeglectAlignment {} + impl TransmuteOptions for NeglectValidity {} + + impl TransmuteOptions for (NeglectAlignment, NeglectValidity) {} // prevent third-party implementations of `TransmuteOptions` mod private { @@ -1285,7 +823,9 @@ pub mod transmute { pub trait Sealed {} impl Sealed for () {} - impl Sealed for NeglectStability {} + impl Sealed for NeglectAlignment {} + impl Sealed for NeglectValidity {} + impl Sealed for (NeglectAlignment, NeglectValidity) {} } } } @@ -1295,127 +835,19 @@ pub mod transmute { # Drawbacks [drawbacks]: #drawbacks -## No Notion of Platform Stability -The stability declaration traits communicate library layout stability, but not *platform* layout stability. A transmutation is platform-stable if it compiling one one platform implies it will compile on all other platforms. Unfortunately, platform-unstable types are common; e.g.: - -- All primitive number types have platform-dependent [endianness](https://en.wikipedia.org/wiki/Endianness). -- All pointer-related primitive types (`usize`, `isize`, `*const T`, `*mut T`, `&T`, `&mut T`) possess platform-dependent layouts; their sizes and alignments are well-defined, but vary between platforms. Concretely, whether `usize` is `TransmuteInto<[u8; 4]>` or `TransmuteInto<[u8; 8]>` will depend on the platform. -- The very existence of some types depends on platform, too; e.g., the contents of [`core::arch`](https://doc.rust-lang.org/stable/core/arch/), [`std::os`](https://doc.rust-lang.org/stable/std/os/), and [`core::sync::atomic`](https://doc.rust-lang.org/stable/std/sync/atomic/) all depend on platform. - -Our proposed stability system is oblivious to the inter-platform variations of these types. Expanding our stability system to be aware of inter-platform variations would introduce considerable additional complexity: -
    -
  1. - -**Cognitive Complexity:** For types whose layout varies between platforms, the [stability] declaration traits could, *perhaps*, be adapted to encode platform-related guarantees. We anticipate this would contribute substantial cognitive complexity. Type authors, even those with no interest in cross-platform stability, would nonetheless need to reason about the layout properties of their types on platforms that might not yet exist. -
    2. -
  2. - -**Ergonomic Complexity:** Platform instabilities are contagious: a type that *contains* a platform-unstable type is, itself, platform-unstable. Due to the sheer virulence of types with platform-dependent layouts, an explicit '`NeglectPlatformStability`' option would need to be used for *many* simple transmutations. The ergonomic cost of this would also be substantial. - -
    3. -
  3. - -**Implementation Complexity:** The mechanisms proposed by this RFC are, fundamentally, applications of and additions to Rust's type system (i.e., they're traits). Mechanisms that impact platform stability, namely `#[cfg(...)]` annotations, long precede type-resolution and layout computation in the compilation process. For instance, it's possible to define types with impossible layouts: -```rust -#[cfg(any())] -struct Recursive(Recursive); -``` -This program compiles successfully on all platforms because, from the perspective of later compilation stages, `Recursive` may as well not exist. - -
    4. -
- -The issues of platform layout stability exposed by this RFC are not fundamentally different from the challenges of platform API stability. These challenges are already competently addressed by the mechanisms proposed in [RFC1868](https://github.com/rust-lang/rfcs/pull/1868). For this reason, and for the aforementioned concerns of additional complexity, we argue that communicating and enforcing platform layout stability must remain outside the scope of this RFC. - -## Stability of *Unsafe* Transmutations -[drawback-unsafe-stability]: #Stability-of-Unsafe-Transmutations - -The model of stability proposed by this RFC frames stability as a quality of *safe* transmutations. A type author cannot specify stability archetypes for *unsafe* transmutations, and it is reasonable to want to do so. - -To accommodate this, we may modify the definitions of `PromiseTransmutableFrom` and `PromiseTransmutableInto` to consume an optional `Neglect` parameter, to allow for stability declarations for unsafe transmutations: -```rust -pub trait PromiseTransmutableFrom -where - Neglect: TransmuteOptions -{ - type Archetype: TransmuteInto>; -} - -pub trait PromiseTransmutableInto -where - Neglect: TransmuteOptions -{ - type Archetype: TransmuteFrom>; -} -``` -Implementations of these traits for a given `Neglect` declares that a transmutation which is accepted while neglecting a particular set of checks (namely the set encoded by `Neglect`) will *continue* to be possible. - -We omit these definition from this RFC's recommendations because they are not completely satisfying. For instance, `Neglect` is a *logically* unordered set of options, but is encoded as a tuple (which *is* ordered). To declare a transmutation that requires neglecting validity and alignment checks as stable, only *one* of these impls ought to be necessary: - -```rust -impl PromiseTransmutableFrom for Foo -{ - ... -} - -impl PromiseTransmutableFrom for Foo -{ - ... -} -``` -Writing *both* impls (as we do above) is logically nonsense, but is nonetheless supported by Rust's coherence rules. +TODO # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives - -## Rationale: `TransmuteFrom`/`TransmuteInto` +## Rationale: `TransmuteFrom` ### Why support arbitrary transmutation? Some [prior art][prior-art], especially in the crate ecosystem, provides an API that [only supports transmutations involving particular types](#Source-and-Destination-Types-Supported) (e.g., from/into bytes). As we discuss in the [prior art][prior-art] section, we believe that the inflexibility of such approaches make them a poor basis of a language proposal. In particular, these restrictive approaches don't leave room to grow: supporting additional transmutations requires additional traits. The API advocated by this proposal is unopinionated about what transmutations users might wish to do, and what transmutations the compiler is able to reason about. The implementation of this RFC may be initially very simple (and perhaps support no more than the restrictive approaches allow for), but then subsequently grow in sophistication—*without* necessitating public API changes. -### Why *two* traits? -If `TransmuteInto` is implemented in terms of `TransmuteFrom`, why provide it at all? We do so for consistency with libcore's [`From`/`Into`](https://doc.rust-lang.org/stable/rust-by-example/conversion/from_into.html) traits, and because directionality conveys intent: `TransmuteFrom` connotes *conversion*, whereas `TransmuteInto` connotes initialization. We believe that the supporting code examples of this RFC demonstrate the explanatory benefits of providing *both* traits. - -## Rationale: Transmutation Options - -### Granularity -Although the focus of our API is statically-correct, infalible transmutations, the ability to opt-out of particular static checks is essential for building safer *fallible* mechanisms, such as alignment-fallible [reference casting][ext-ref-casting], or validity-fallible transmutations (e.g., `bool` to `u8`). - -### Representation -Although transmutations options exist at a type-level, they're represented as type-level tuples, whose familiar syntax is identical to value-level tuples. An empty tuple seems like the natural choice for encoding *don't neglect anything*. - -We could not identify any advantages to representing options with const-generics. There is no clear syntactic advantage: tuples remain the most natural way to encode ad-hoc products of items. The comparative lack of default values for const-generic parameters poses an ergonomic *disadvantage*. - - -## Rationale: Stability - -### Why do we need a stability system? - -At least two requirements necessitate the presence of a stability system: - -#### Mitigating a Unique Stability Hazard -The usual rules of SemVer stability dictate that if a trait is is implemented in a version `m.a.b`, it will *continue* to be implemented for all versions `m.x.y`, where `x ≥ a` and `y ≥ b`. **`TransmuteFrom` is the exception to this rule**. It would be irresponsible to do nothing to mitigate this stability hazard. - -The compromise made by this RFC is that **`TransmuteFrom` should be stable-by-default**: - - `Dst: TransmuteFrom` follows the usual SemVer rules, - - `Dst: TransmuteFrom` *does not*. - -### Why this *particular* stability system? -The proposed stability system is both simple, flexible, and extensible. Whereas ensuring the soundness and safety of `TransmuteFrom` requires non-trivial compiler support, stability does not—it is realized as merely two normal traits and an `impl`. - -This formulation is flexible: by writing custom `Archetype`s, the [stability declaration traits][stability] make possible granular and incomplete promises of layout stability (e.g., guaranteeing the size and validity qualities of a type, but *not* its alignment. Members of the safe-transmute working group have [expressed](https://rust-lang.zulipchat.com/#narrow/stream/216762-project-safe-transmute/topic/Transmutability.20Intrinsic/near/202712834) an interest in granular stability declarations. - -Finally, this formulation is extensible. The range of advance use-cases permitted by these traits is constrained only by the set of possible `Archetype`s, which, in turn, is constrained by the expressiveness of `TransmuteFrom`. As the implementation of `TransmuteFrom` becomes more complete, so too will the range of advance use-cases accommodated by these traits. - - -### Couldn't `#[repr(C)]` denote stability? -In our proposal, `#[repr(C)]` does not connote any promises of transmutation stability for SemVer purposes. It has [been suggested](https://rust-lang.zulipchat.com/#narrow/stream/216762-project-safe-transmute/topic/RFC.3A.20Stability.20Declaration.20Traits/near/204011238) that the presence of `#[repr(C)]` *already* connotes total transmutation stability; i.e., that the type's author promises that the type's size and alignment and bit-validity will remain static. If this is true, then an additional stability mechanism is perhaps superfluous. However, we are unaware of any authoritative documentation indicating that `#[repr(C)]` carries this implication. Treating `#[repr(C)]` as an indicator of transmutation stability [would](https://rust-lang.zulipchat.com/#narrow/stream/216762-project-safe-transmute/topic/typic/near/201165897) thus pose a stability hazard. - - ## Alternative: Implementing this RFC in a Crate This RFC builds on ample [prior art][prior-art] in the crate ecosystem, but these efforts strain against the fundamental limitations of crates. Fundamentally, safe transmutation efforts use traits to expose layout information to the type system. The burden of ensuring safety [is usually either placed entirely on the end-user, or assumed by complex, incomplete proc-macro `derives`][mechanism-manual]. @@ -1426,7 +858,7 @@ The development approaches like [typic][crate-typic]'s could, perhaps, be eased Regardless of approach, almost all [prior art][prior-art] attempts to reproduce knowledge *already* possessed by `rustc` during the compilation process (i.e., the layout qualities of a concrete type). Emulating the process of layout computation to any degree is an error-prone duplication of effort between `rustc` and the crate, in a domain where correctness is crucial. -Finally, community-led, crate-based approaches are, inescapably, unauthoritative. These approaches are incapable of fulfilling our motivating goal of providing a *standard* mechanism for programmers to statically ensure that a transmutation is safe, well-defined, or stable. +Finally, community-led, crate-based approaches are, inescapably, unauthoritative. These approaches are incapable of fulfilling our motivating goal of providing a *standard* mechanism for programmers to statically ensure that a transmutation is well-defined or safe. # Prior art [prior-art]: #prior-art @@ -1488,16 +920,16 @@ While this RFC does not provide a grand, all-encompassing mechanism for fallible /// /// This produces `None` if the referent isn't appropriately /// aligned, as required by the destination type. -pub fn try_cast_ref<'t, 'u, T, U>(src: &'t T) -> Option<&'u U> +pub fn try_cast_ref<'t, 'u, T, U, Scope>(src: &'t T) -> Option<&'u U> where - &'t T: TransmuteInto<&'u U, NeglectAlignment>, + &'t T: TransmuteFrom<&'u U, Scope, NeglectAlignment>, { if (src as *const T as usize) % align_of::() != 0 { None } else { // Safe because we dynamically enforce the alignment // requirement, whose static check we chose to neglect. - Some(unsafe { src.unsafe_transmute_into() }) + Some(unsafe { TransmuteFrom::unsafe_transmute_from(src) }) } } ``` @@ -1507,7 +939,7 @@ In this approach, our RFC is joined by crates such as [plain](https://docs.rs/pl Prior work differs in whether its API surface is flexible enough to support transmutation between arbitrary types, or something less. #### Arbitrary Types -Approaches supporting transmutations between arbitrary types invariably define traits akin to: +Approaches supporting transmutations between arbitrary types invariably define traits akin to either or both: ```rust /// Indicates that `Self` may be transmuted into `Dst`. pub unsafe trait TransmuteInto @@ -1608,8 +1040,8 @@ Automatic approaches implement the transmutation traits without user interventio The [typic][crate-typic] crate mocks a fully-automatic approach: its `TransmuteFrom` trait is usable with any types that are `repr(C)`, or otherwise have a well-defined memory layout. (In practice, since Rust lacks reflection over type definitions, `repr(C)` annotations much be changed to `typic::repr(C)`.) -### Stability Hazards -Fully automatic approaches introduce, at the very least, a stability hazard: they supply a safe constructor for types, without the consent of those types' authors. If a type author hid the internals of their type because they do not wish for its implementation details to become a part of the type's API for SemVer purposes, an automatic transmutation mechanism subverts that intent. +### Safety Hazards +Fully automatic approaches introduce, at the very least, a safety hazard: they supply a safe constructor for types, without the consent of those types' authors. If a type author hid the internals of their type because they do not wish for its implementation details to become a part of the type's API for SemVer for safety purposes, an automatic transmutation mechanism subverts that intent. No attempt to avoid this hazard is made by most of the proposals featuring automatic mechanisms; e.g.: - [*Draft-RFC: `from_bytes`*][2018-05-23] @@ -1665,29 +1097,12 @@ In [*Future Possibilities*][future-possibilities], we propose a number of additi ### Questions To Be Resolved Before Feature Stabilization The following unresolved questions should be resolved before feature stabilization: -##### Layout-Stability for Unsafe Transmutations? -We [observe][drawback-unsafe-stability] that our proposed model for stability declaration, although very expressive, does not permit type authors to declare the stability of *unsafe* transmutations. Alongside that observation, we suggest a [SemVer-compatible](https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md#minor-change-adding-a-defaulted-type-parameter) upgrade of the stability declaration traits that may resolve this shortcoming. - -While it is unclear if there is any demand for this degree of flexibility, this upgrade-path should be carefully considered *before* stabilizing (and thus committing) to this RFC's layout stability declaration traits. - - ### Questions Out of Scope We consider the following unresolved questions to be out-of-scope of *this* RFC process: -##### Design of `NeglectConstructability`? -`TransmuteFrom` and `TransmuteInto` require that the destination type has a matching constructor in which all fields are marked `pub`. Conspicuously *missing* from this RFC is a `NeglectConstructability` unsafe option to disable this check. - -The omission is intentional. The consequences of such an option are suprising in both their subtlety and their unsafety. Some of unsafe Rust's hairiest interactions lie at the intersections of `!Send`, `!Sync`, `UnsafeCell` and restricted field visibility. These building blocks are used to build safe, public abstractions that encapsulate unsafe, hidden internals. - # Future possibilities [future-possibilities]: #future-possibilities -## Extension: `PromiseTransmutable` Shorthand -[extension-promisetransmutable-shorthand]: #extension-promisetransmutable-shorthand -[0000-ext-promise-transmutable.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-promise-transmutable.md - -See [here][0000-ext-promise-transmutable.md]. - ## Extension: Layout Property Traits [0000-ext-layout-traits.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-layout-traits.md From af9f3012de8a642a9fcaa2f2f334e68f682742ad Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 19 Oct 2020 11:32:36 -0400 Subject: [PATCH 16/18] Add some intro text to Future Possibilities + sentence on safe union access. --- text/0000-safer-transmute.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index c7586dbe26e..f567bc2c7a3 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -1103,16 +1103,20 @@ We consider the following unresolved questions to be out-of-scope of *this* RFC # Future possibilities [future-possibilities]: #future-possibilities +## Safe Union Access + +Given `TransmuteFrom`, the compiler can determine whether an access of a union variant of type `V` from a union `U` is safe by checking `V: TransmuteFrom`. In accesses where that bound is satisfied, the compiler can omit the requirement that the access occur in an `unsafe` block. + ## Extension: Layout Property Traits [0000-ext-layout-traits.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-layout-traits.md -See [here][0000-ext-layout-traits.md]. +Given `TransmuteFrom`, crates can define traits that are implemented only when size and alignment invariants are satisfied, such as `SizeEq` or `AlignLtEq`. For additional details, see [here][0000-ext-layout-traits.md]. ## Extension: Byte Transmutation Traits and Safe Initialization [extension-zerocopy]: #extension-byte-transmutation-traits-and-safe-initialization [0000-ext-byte-transmutation.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-byte-transmutation.md -See [here][0000-ext-byte-transmutation.md]. +Given `TransmuteFrom`, crates can define zerocopy-style traits. For additional details, see [here][0000-ext-byte-transmutation.md]. ## Extension: Slice and `Vec` Casting @@ -1120,18 +1124,18 @@ See [here][0000-ext-byte-transmutation.md]. [ext-vec-casting]: #extension-slice-and-vec-casting [0000-ext-container-casting.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-container-casting.md -See [here][0000-ext-container-casting.md]. +Given `TransmuteFrom`, crates can define traits for "transmuting" slices and `Vec`s. For additional details, see [here][0000-ext-container-casting.md]. ## Extension: `include_data!` [future-possibility-include_data]: #Extension-include_data [0000-ext-include-data.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-include-data.md -See [here][0000-ext-include-data.md]. +Given `TransmuteFrom`, crates can define a more useful alternative to `include_bytes!`. For additional details, see [here][0000-ext-include-data.md]. ## Extension: Generic Atomics [future-possibility-generic-atomics]: #extension-generic-atomics [0000-ext-generic-atomic.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-generic-atomic.md -See [here][0000-ext-generic-atomic.md]. +Given `TransmuteFrom`, crates can define a generic `Atomic` alternative to the various `Atomic*` types. For additional details, see [here][0000-ext-generic-atomic.md]. From cff2ab1864fb05291416bb1d9d2962603745a184 Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Fri, 30 Oct 2020 14:26:20 -0400 Subject: [PATCH 17/18] Out with scope, in with `Muckable`! --- text/0000-safer-transmute.md | 448 ++++++++++++++++------------------- 1 file changed, 209 insertions(+), 239 deletions(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index f567bc2c7a3..0de7b7716c1 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -11,6 +11,7 @@ We propose traits, namely `TransmuteFrom`, that are implemented *automatically* for combinations of types that may be safely transmuted. In other words, this RFC makes safe transmutation *as easy as 1..., 2..., `repr(C)`!* ```rust +#[derive(Muckable)] #[repr(C)] pub struct Foo(pub u8, pub u16); // ^ there's a padding byte here, between these fields @@ -90,6 +91,9 @@ A transmutation is ***well-defined*** if the mere act of transmuting a value fro ### 📖 Safety A well-defined transmutation is ***safe*** if *using* the transmuted value cannot violate memory safety. +### 📖 Stability +A safe transmutation is ***stable*** if the authors of the source type and destination types have indicated that the layouts of those types is part of their libraries' stability guarantees. + ## Concepts in Depth ***Disclaimer:** While the high-level definitions of transmutation well-definedness and safety is a core component of this RFC, the detailed rules and examples in this section are **not**. We expect that the initial implementation of `TransmuteFrom` may initially be considerably less sophisticated than the examples in this section (and thus forbid valid transmutations). Nonetheless, this section explores nuanced cases of transmutation well-definedness and safety to demonstrate that the APIs we propose can grow to handle that nuance.* @@ -126,6 +130,7 @@ The bits of any valid instance of the source type must be a bit-valid instance o For example, we are permitted to transmute a `Bool` into a [`u8`]: ```rust +#[derive(Muckable)] #[repr(u8)] enum Bool { True = 1, @@ -146,11 +151,11 @@ let _ : Bool = transmute!(u8::default()); // Compile Error! Another example: While laying out certain types, Rust may insert padding bytes between the layouts of fields. In the below example `Padded` has two padding bytes, while `Packed` has none: ```rust #[repr(C)] -#[derive(Default)] +#[derive(Default, Muckable)] struct Padded(pub u8, pub u16, pub u8); #[repr(C)] -#[derive(Default)] +#[derive(Default, Muckable)] struct Packed(pub u16, pub u16, pub u16); assert_eq!(mem::size_of::(), mem::size_of::()); @@ -279,25 +284,13 @@ If this example did not produce a compile error, the value of `z` would not be a -### 📖 When is a transmutation safe? -A well-defined transmutation is ***safe*** if *using* the transmuted value safely cannot violate memory safety. Whereas well-definedness solely concerns the act of transmutation, *safety* is concerned with what might happen with a value *after* transmutation occurs. A well-defined transmutation must be safe if the involved types are *implicitly constructable*. - -#### Implicit Constructability -A struct or enum variant is *fully implicitly constructable* at a given location only if, at that location, that type can be instantiated via its *implicit constructor*, and its fields are also *implicitly constructable*. - -The *implicit constructor* of a struct or enum variant is the constructor Rust creates implicitly from its definition; e.g.: -```rust -struct Point { - x: T, - y: T, -} - -let p = Point { x: 4, y: 2 }; - // ^^^^^^^^^^^^^^^^^^^^ An instance of `Point` is created here, via its implicit constructor. -``` +### 📖 When is a transmutation safe and stable? +[safe-and-stable transmutation]: #-when-is-a-transmutation-safe-and-stable -Limiting implicit constructability is the fundamental mechanism with which type authors build safe abstractions for `unsafe` code, whose soundness is dependent on preserving invariants on fields. Usually, this takes the form of restricting the visibility of fields. For instance, consider the type `NonEmptySlice`, which enforces a validity constraint on its fields via its constructor: +A well-defined transmutation is ***safe*** if *using* the transmuted value safely cannot violate memory safety. Whereas well-definedness solely concerns the act of transmutation, *safety* is concerned with what might happen with a value *after* transmutation occurs. Since transmutation provides a mechanism for arbitrarily reading and modifying the bytes of a type, a well-defined transmutation is not necessarily safe, nor stable. +#### Well-Definedness Does Not Imply Safety +For instance, consider the type `NonEmptySlice`, which enforces a validity constraint on its fields via privacy and its constructor `from_array`: ```rust pub mod crate_a { @@ -326,63 +319,41 @@ pub mod crate_a { } ``` -It is sound for `first` to be a *safe* method is because the `from_array` constructor ensures that `data` is safe to dereference, and because `from_array` is the *only* way to safely initialize `NonEmptySlice` outside of `crate_a` (note that `NonEmptySlice`'s fields are *not* `pub`). As a rule: any field that is not marked `pub` should be assumed to be private *because* it is subject to safety invariants. +It is sound for `first` to be a *safe* method is because the `from_array` constructor ensures that `data` is safe to dereference, and because `from_array` is the *only* way to safely initialize `NonEmptySlice` outside of `crate_a` (note that `NonEmptySlice`'s fields are *not* `pub`). -Unfortunately, field visibility modifiers are not a surefire indicator of whether a type is *fully* implicitly constructable. A type author may restrict the implicit constructability of a type even in situations where all fields of that type (*and all fields of those fields*) are `pub`; consider: +However, transmutation supplies a mechanism for constructing instances of a type *without* invoking its implicit constructor, nor any constructors defined by the type's author. In the previous examples, it would be *unsafe* to transmute `[usize; 2]` into `NonEmptySlice` outside `crate_a`, because subsequent *safe* use of that value (namely, calling `first`) would violate memory safety: ```rust -pub mod crate_a { - - #[repr(C)] - pub struct NonEmptySlice<'a, T>(pub private::NonEmptySliceInner<'a, T>); +/* ⚠️ This example intentionally does not compile. */ +// [usize; 2] ⟶ NonEmptySlice +let _: NonEmptySlice<'static, u8> = transmute!([0usize; 2]); // Compile Error: `NonEmptySlice<_, _>` is not safely transmutable from `[usize; 2]`. +``` - impl<'a, T> NonEmptySlice<'a, T> { - pub fn from_array(arr: &'a [T; N], len: usize) -> Self { - assert!(len <= N && len > 0); - Self( - private::NonEmptySliceInner { - data: arr as *const T, - len, - lifetime: core::marker::PhantomData, - } - ) - } +#### Well-Definedness Does Not Imply Stability +Since the well-definedness of a transmutation is affected by the layouts of the source and destination types, internal changes to those types' layouts may cause code which previously compiled to produce errors. In other words, transmutation causes a type's layout to become part of that type's API for the purposes of SemVer stability. - pub fn first(&self) -> &'a T { - unsafe { &*self.0.data } - } - } - // introduce a private module to avoid `private_in_public` error (E0446): - pub(crate) mod private { - #[repr(C)] - pub struct NonEmptySliceInner<'a, T> { - pub data: *const T, - pub len: usize, - pub lifetime: core::marker::PhantomData<&'a ()>, - } - } +#### Signaling Safety and Stability with `Muckable` +To signal that your type may be safely and safely constructed via transmutation, implement the `Muckable` marker trait: +```rust +use mem::transmute::Muckable; -} +#[derive(Muckable)] +#[repr(C)] +pub struct Foo(pub u8, pub u16); ``` -In the above example, the definitions of both `NonEmptySlice` and its field `NonEmptySliceInner` are marked `pub`, and all fields of these types are marked `pub`. However, `NonEmptySlice` is *not* fully implicitly constructible outside of `crate_a`, because the module containing `NonEmptySliceInner` is not visibile outside of `crate_a`. -#### Constructability and Transmutation -Transmutation supplies a mechanism for constructing instances of a type *without* invoking its implicit constructor, nor any constructors defined by the type's author. In the previous examples, it would be *unsafe* to transmute `[usize; 2]` into `NonEmptySlice` outside `crate_a`, because subsequent *safe* use of that value (namely, calling `first`) would violate memory safety: -```rust -/* ⚠️ This example intentionally does not compile. */ -// [usize; 2] ⟶ NonEmptySlice -let _: NonEmptySlice<'static, u8> = transmute!([0usize; 2]); // Compile Error: `NonEmptySlice<_, _>` is not constructible here. -``` -If a field is private, then instantiating or modifying it via transmutation is not, generally speaking, safe. +The `Muckable` marker trait signals that your type's fields may be safely initialized and modified to *any* value. If you would not be comfortable making your type's fields `pub`, you probably should not implement `Muckable` for your type. By implementing `Muckable`, you promise to treat *any* observable modification to your type's layout as a breaking change. (Unobservable changes, such as renaming a private field, are fine.) + +As a rule, the destination type of a transmutation must be `Muckable`. -For transmutations where the destination type involves mutate-able references, the constructability of the *source* type is also relevant. Consider: +For transmutations where the destination type involves mutate-able references, the `Muckab`ility of the *source* type is also relevant. Consider: ```rust /* ⚠️ This example intentionally does not compile. */ let arr = [0u8, 1u8, 2u8]; let mut x = NonEmptySlice::from_array(&arr, 2); { // &mut NonEmptySlice ⟶ &mut [usize; 2] - let y : &mut u128 = transmute!(&mut x) // Compile Error! `&mut NonEmptySlice` is not constructible here. + let y : &mut u128 = transmute!(&mut x) // Compile Error! `&mut NonEmptySlice` is not safely transmutable from `&mut u128`. *y[0] = 0; *y[1] = 0; } @@ -391,101 +362,6 @@ let z : NonEmptySlice = x; ``` If this example did not produce a compile error, the value of `z` would not be a safe instance of its type, `NonEmptySlice`, because `z.first()` would dereference a null pointer. -#### Constructability and Scope -Whether a type is fully implicitly constructable will depends on the *scope* in which that question is asked. Consider: -```rust -pub mod a { - - #[repr(C)] pub struct Foo(private::Bar); - - mod private { - #[repr(C)] pub struct Bar; - } - - // `Foo` is fully implicitly constructible in this module. - const _: Foo = Foo { private::Bar }; - - // Thus, `Foo` is transmutable in this module! - const _: Foo = transmute!(()); -} - -pub mod b { - use super::a; - - // `Foo` is NOT fully implicitly constructible in this module. - const _: Foo = a::Foo { a::private::Bar }; // Compile Error: the module `a::private` is private. - - // Thus, `Foo` is NOT transmutable in this module: - const _: Foo = transmute!(()); // Compile Error: `Foo` is not constructible here. -} -``` - -The `transmute!` macro provides a shorthand for safely transmuting a value using its invocation scope as its reference frame: -```rust -pub macro transmute($expr: expr) { - TransmuteFrom::<_, Here!()>::transmute_from($expr) - // ┯ ━━━┯━━━ - // │ ┕ check constructability from `transmute!`'s invocation scope - // ┕ the destination type of the transmute (`_` used to infer the type from context) -} -``` -The `Here!()` macro produces a type that uniquely identifies its invocation scope. - -This explicit `Scope` parameter of `TransmuteFrom` makes possible the creation of generic abstractions over it. For instance, consider a hypothetical `FromZeros` trait that indicates whether `Self` is safely initializable from a a sufficiently large buffer of zero-initialized bytes: -```rust -pub mod zerocopy { - pub unsafe trait FromZeros { - /// Safely initialize `Self` from zeroed bytes. - fn zeroed() -> Self; - } - - #[derive(Copy, Clone)] - #[repr(u8)] - enum Zero { - Zero = 0u8 - } - - unsafe impl FromZeros for Dst - where - Dst: TransmuteFrom<[Zero; usize::MAX], ??? >, - { - fn zeroed() -> Self { - [Zero; size_of::].transmute_into() - } - } -} -``` -The above definition leaves ambiguous (`???`) the scope in which the constructability of `Dst` is checked: is it from the perspective of where this trait is defined, or where this trait is *used*? You probably do *not* intend for this trait to *only* be usable with `Dst` types that are defined in the same scope as the `FromZeros` trait! - -Adding an explicit `Scope` parameter to `FromZeros` makes this unambiguous; the transmutability of `Dst` should be assessed from where the trait is used, *not* where it is defined: -```rust -pub unsafe trait FromZeros { - /// Safely initialize `Self` from zeroed bytes. - fn zeroed() -> Self; -} - -unsafe impl FromZeros for Dst -where - Dst: TransmuteFrom<[Zero; usize::MAX], Scope> -{ - fn zeroed() -> Self { - [Zero; size_of::].transmute_into() - } -} -``` - -A thid-party could then use `FromZeros` like so: -```rust -use zerocopy::FromZeros; - -#[repr(C)] -struct Foo { - ... -} - -// Initialize `Foo` from zero-initialized bytes. -let _: Foo = FromZeros::<_, Here!()>::zeroed(); -``` ## Mechanisms of Transmutation @@ -494,7 +370,7 @@ The `TransmuteFrom` trait provides the fundamental mechanism checking the transm ```rust // this trait is implemented automagically by the compiler #[lang = "transmute_from"] -pub unsafe trait TransmuteFrom +pub unsafe trait TransmuteFrom where Neglect: TransmuteOptions, { @@ -532,15 +408,28 @@ where In the above definitions, `Src` represents the source type of the transmutation, `Dst` represents the destination type of the transmutation, and `Neglect` is a parameter that [encodes][options] which static checks the compiler ought to neglect when considering if a transmutation is valid. The default value of `Neglect` is `()`, which reflects that, by default, the compiler does not neglect *any* static checks. +The transmute! macro provides a shorthand for safely transmuting a value: +```rust +pub macro transmute($expr: expr) { + core::convert::transmute::TransmuteFrom::<_>::transmute_from($expr) + // ┯ + // ┕ the destination type of the transmute (`_` used to infer the type from context) +} +``` + +A `differing_sizes` lint warns when the source and destination types of a transmutation (conducted via `transmute!` or `transmute_from`) have different sizes. + + ### Neglecting Static Checks [options]: #Neglecting-Static-Checks -The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are ill-defined or unsafe. However, you may explicitly opt-out of some static checks; e.g.: +The default value of the `Neglect` parameter, `()`, statically forbids transmutes that are ill-defined or unsafe. However, you may explicitly opt-out of some static checks; namely: | Transmute Option | Usable With | |---------------------|---------------------------------------------------------| | `NeglectAlignment` | `unsafe_transmute_{from,into}` | | `NeglectValidity` | `unsafe_transmute_{from,into}` | +| `NeglectSafety` | `unsafe_transmute_{from,into}` | The selection of multiple options is encoded by grouping them as a tuple; e.g., `(NeglectAlignment, NeglectValidity)` is a selection of both the `NeglectAlignment` and `NeglectValidity` options. @@ -572,9 +461,9 @@ By using the `NeglectAlignment` option, you are committing to ensure that the tr /// /// This produces `None` if the referent isn't appropriately /// aligned, as required by the destination type. -pub fn try_cast_ref<'t, 'u, T, U, Scope>(src: &'t T) -> Option<&'u U> +pub fn try_cast_ref<'t, 'u, T, U>(src: &'t T) -> Option<&'u U> where - &'t T: TransmuteFrom<&'u U, Scope, NeglectAlignment>, + &'t T: TransmuteFrom<&'u U, NeglectAlignment>, { if (src as *const T as usize) % align_of::() != 0 { None @@ -613,15 +502,15 @@ enum Bool { False = 0, } -pub trait TryIntoBool +pub trait TryIntoBool { fn try_into_bool(self) -> Option; } -impl TryIntoBool for T +impl TryIntoBool for T where - u8: TransmuteFrom, - Bool: TransmuteFrom + u8: TransmuteFrom, + Bool: TransmuteFrom { fn try_into_bool(self) -> Option { let val: u8 = TransmuteFrom::transmute_from(self); @@ -643,93 +532,73 @@ Even with `NeglectValidity`, the compiler will statically reject transmutations #[repr(C)] enum Bar { Z = 42 } -let _ = ::unsafe_transmute_from(Foo::N) // Compile error! +let _ = ::unsafe_transmute_from(Foo::N) // Compile error! ``` -# Reference-level explanation -[reference-level-explanation]: #reference-level-explanation - -## Implementation Guidance -Three items in this RFC require special compiler support: - - `Here!()` - - `Constructible` (a private implementation detail of `TransmuteFrom`) - - `TransmuteFrom` - -### Implementing `Here!()` -The `Here!` macro produces a type that uniquely identifies its invocation scope. For instance: -```rust -use static_assertions::*; - -type A = Here!(); -type B = Here!(); +#### `NeglectSafety` +By default, `TransmuteFrom`'s methods require that all instantiations of the source type are `Muckable`. If the destination type is a mutate-able reference, the source type must *also* be `Muckable`. This precludes transmutations that are [well-defined][sound transmutation] but not [safe][safe-and-stable transmutation]. -/* Types A and !::B are equal */ -assert_type_eq_all!(A, B); +Whether the bound `Dst: TransmuteFrom` is implemented depends *solely* on the compiler's analysis of the layouts of `Src` and `Dst` (see [*When is a transmutation well-defined?*][sound transmutation])—and *not* the opt-in of the authors of `Src` and `Dst`. When using this option, the onus is on *you* to ensure that you are adhering to the documented layout and library validity guarantees of the involved types. -trait Foo { - type C; - const CONST: (); -} - -impl Foo for ! { - type C = Here!(); - - const CONST: () = { - type D = Here!(); +You might use this option if the involved types predate the `Muckable` trait (e.g., old versions of `libc`). For instance, checking `libc::in6_addr: TransmuteFrom` is better than nothing; it statically ensures that the transmutation from `Src` to `libc::in6_addr` is [well-defined][sound transmutation]. - /* A, B and !::C are equal */ - assert_type_eq_all!(A, B, !::C); - - /* C and D are NOT equal; C and D inhabit different scopes */ - assert_type_ne_all!(C, D); - }; +You might also use this option to signal that a *particular* transmutation is stable *without* implementing `Muckable` (which would signal that *all* transmutations are stable): +```rust +impl From for Bar +where + Bar: TransmuteFrom +{ + fn from(src: Foo) -> Self { + unsafe { Bar::unsafe_transmute_from(src) } + } } ``` -Scope types should (as much as possible) pretty print in compiler error messages as their definition path. -These scope types should generated with `pub(self)` visibility. We are not currently aware of any reason why publicly re-exporting a scope type via a type alias would be a good idea; restricting the visibility of these types will warn users against doing so. If compelling use-cases for re-exported scope types are discovered in the future, a broader visibility could be used instead without breaking backwards compatibility. - -### Implementing `Constructible` -The compiler implements `Constructible` for `T` if `T` is fully implicitly constructible in the scope uniquely identified by the type `Scope`. +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation -A type `T` is fully implicitly constructible in a particular scope if: - - `T`'s implicit constructor is reachable from the scope, and either: - - `T` has no fields, or - - `T`'s fields are fully implicitly constructible from the scope. +## Implementation Guidance +Two items in this RFC require special compiler support: + - `Muckable` + - `TransmuteFrom` + - `differing_sizes` lint -The `Constructible` trait does not ever need to be made stable, or even visible (à la the virtual `Freeze` trait). It is merely a useful device for implementing `TransmuteFrom`. +### Implementing `Muckable` +The `Muckable` marker trait is similar to `Copy` in that all fields of a `Muckable` type must, themselves, be `Muckable`. ### Implementing `TransmuteFrom` The implementation of `TransmuteFrom` is completely internal to the compiler (à la [`Sized`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_sized) and [`Freeze`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyS.html#method.is_freeze)). #### Constructability and Transmutation -Unless `NeglectConstructability` is used as `Neglect` option, a `Src` is *safely* transmutable into `Dst` in a given `Scope` if: - 1. `Dst: Constructible` - 2. `Dst: TransmuteFrom` +A `Src` is *safely* transmutable into `Dst` in a given if: + 1. `Dst: Muckable` + 2. `Dst: TransmuteFrom` + 3. `NeglectSafety` ∉ `Neglect` If `Src` is a mutatable reference, then additionally: - 1. `Src: Constructible` + 1. `Src: Muckable` -### Implementing `differing_sizes` -The `differing_sizes` lint reports a compiler warning when the source and destination types of a `transmute!()`, `transmute_into` or `transmute_from` invocation differ. This lint shall be warn-by-default. +### Implementing `differing_sizes` lint +The `differing_sizes` lint reports a compiler warning when the source and destination types of a `transmute!()`, `transmute_from` or `unsafe_transmute_from` invocation differ. This lint shall be warn-by-default. ### Minimal Useful Stabilization Surface Stabilizing *only* this subset of the Initial Smart Implementation will cover many use-cases: - `transmute!()` To define traits that generically abstract over `TransmuteFrom`, these items must be stabilized: - - `Here!()` - `TransmuteFrom` - `TransmuteOptions` and `SafeTransmuteOptions` ### Complete API Surface -[minimal-impl]: #Listing-for-Initial-Minimal-Implementation -This listing is the **canonical specification** of this RFC's API surface ([playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0a4dda2760244110bcf829c298c45c34)): +[minimal-impl]: #complete-api-surface +This listing is the **canonical specification** of this RFC's API surface ([playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=65eaff7ba0568fe281f9303c57d56ded)): ```rust #![feature(untagged_unions,const_fn,const_fn_union)] // for the impl of unsafe_transmute_from -#![feature(decl_macro)] // for stub implementations of derives -#![allow(warnings)] +#![feature(decl_macro)] // for `transmute!` and `#[derive(Muckable)]` macros +#![feature(const_generics)] // for stability declarations on `[T; N]` +#![feature(never_type)] // for stability declarations on `!` +#![allow(unused_unsafe, incomplete_features)] /// Transmutation conversions. // suggested location: `core::convert` @@ -739,13 +608,13 @@ pub mod transmute { /// Safely transmute $expr pub macro transmute($expr: expr) { - TransmuteFrom::<_, Here!()>::transmute_from($expr) + core::convert::transmute::TransmuteFrom::<_>::transmute_from($expr) } /// `Self: TransmuteFrom + pub unsafe trait TransmuteFrom where Neglect: TransmuteOptions, { @@ -777,44 +646,42 @@ pub mod transmute { src: ManuallyDrop, dst: ManuallyDrop, } - + unsafe { ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst) } } } - /// A type is always transmutable from itself. - // This impl will be replaced with a compiler-supported for arbitrary source and destination types. - unsafe impl TransmuteFrom for T - where - Neglect: TransmuteOptions - {} - /// Static checks that may be neglected when determining if a type is `TransmuteFrom` some other type. pub mod options { /// Options that may be used with safe transmutations. pub trait SafeTransmuteOptions: TransmuteOptions {} + /// `()` denotes that no static checks should be neglected. + impl SafeTransmuteOptions for () {} + /// Options that may be used with unsafe transmutations. pub trait TransmuteOptions: private::Sealed {} - impl SafeTransmuteOptions for () {} - impl TransmuteOptions for () {} - /// Neglect the alignment check of `TransmuteFrom`. pub struct NeglectAlignment; /// Neglect the validity check of `TransmuteFrom`. pub struct NeglectValidity; - /* additional options */ + /// Neglect the safety check of `TransmuteFrom`. + pub struct NeglectSafety; + impl TransmuteOptions for () {} impl TransmuteOptions for NeglectAlignment {} impl TransmuteOptions for NeglectValidity {} - + impl TransmuteOptions for NeglectSafety {} impl TransmuteOptions for (NeglectAlignment, NeglectValidity) {} + impl TransmuteOptions for (NeglectAlignment, NeglectSafety) {} + impl TransmuteOptions for (NeglectSafety, NeglectValidity) {} + impl TransmuteOptions for (NeglectAlignment, NeglectSafety, NeglectValidity) {} // prevent third-party implementations of `TransmuteOptions` mod private { @@ -825,9 +692,76 @@ pub mod transmute { impl Sealed for () {} impl Sealed for NeglectAlignment {} impl Sealed for NeglectValidity {} + impl Sealed for NeglectSafety {} impl Sealed for (NeglectAlignment, NeglectValidity) {} + impl Sealed for (NeglectAlignment, NeglectSafety) {} + impl Sealed for (NeglectSafety, NeglectValidity) {} + impl Sealed for (NeglectAlignment, NeglectSafety, NeglectValidity) {} } } + + /// Traits for declaring the SemVer stability of types. + pub mod stability { + + /// Denotes that `Self`'s fields may be arbitarily initialized or + /// modified, regardless of their visibility. Implementing this trait + /// additionally denotes that you will treat any observable changes to + /// `Self`'s layout as breaking changes. (Unobservable changes, such as + /// renaming a private field, are fine.) + pub trait Muckable {} + + /// `#[derive(Muckable)]` + pub macro Muckable($expr: expr) { + /* stub */ + } + + impl Muckable for ! {} + impl Muckable for () {} + + impl Muckable for f32 {} + impl Muckable for f64 {} + + impl Muckable for i8 {} + impl Muckable for i16 {} + impl Muckable for i32 {} + impl Muckable for i64 {} + impl Muckable for i128 {} + impl Muckable for isize {} + + impl Muckable for u8 {} + impl Muckable for u16 {} + impl Muckable for u32 {} + impl Muckable for u64 {} + impl Muckable for u128 {} + impl Muckable for usize {} + + impl Muckable for core::marker::PhantomData {} + + impl Muckable for [T; N] + where + T: Muckable, + {} + + impl Muckable for *const T + where + T: Muckable, /* discuss this bound */ + {} + + impl Muckable for *mut T + where + T: Muckable, /* discuss this bound */ + {} + + impl<'a, T: ?Sized> Muckable for &'a T + where + T: Muckable, + {} + + impl<'a, T: ?Sized> Muckable for &'a mut T + where + T: Muckable, + {} + } } ``` @@ -920,9 +854,9 @@ While this RFC does not provide a grand, all-encompassing mechanism for fallible /// /// This produces `None` if the referent isn't appropriately /// aligned, as required by the destination type. -pub fn try_cast_ref<'t, 'u, T, U, Scope>(src: &'t T) -> Option<&'u U> +pub fn try_cast_ref<'t, 'u, T, U>(src: &'t T) -> Option<&'u U> where - &'t T: TransmuteFrom<&'u U, Scope, NeglectAlignment>, + &'t T: TransmuteFrom<&'u U, NeglectAlignment>, { if (src as *const T as usize) % align_of::() != 0 { None @@ -1061,7 +995,7 @@ Our RFC, [typic][crate-typic], and Haskell exploit the related concept of *const Haskell's [`Coercible`](https://hackage.haskell.org/package/base-4.14.0.0/docs/Data-Coerce.html#t:Coercible) typeclass is implemented for all types `A` and `B` when the compiler can infer that they have the same representation. As with our proposal's `TransmuteFrom` trait, instances of this typeclass are created "on-the-fly" by the compiler. `Coercible` primarily provides a safe means to convert to-and-from newtypes, and does not seek to answer, for instance, if two `u8`s are interchangeable with a `u16`. -Haskell takes an algebraic approach to this problem, reasoning at the level of type definitions, not type layouts. However, not all type parameters have an impact on1 a type's layout; for instance: +Haskell takes an algebraic approach to this problem, reasoning at the level of type definitions, not type layouts. However, not all type parameters have an impact on a type's layout; for instance: ```rust #[repr(C)] struct Bar(PhantomData); @@ -1097,6 +1031,30 @@ In [*Future Possibilities*][future-possibilities], we propose a number of additi ### Questions To Be Resolved Before Feature Stabilization The following unresolved questions should be resolved before feature stabilization: +#### When should `Muckable` be automatically implemented? + +There is considerable overlap between the effect of `Muckable` and making fields `pub`. A type that is implicitly constructible *already* permits the arbitrary initialization and modification of its fields. While there may be use-cases for implementing `Muckable` on a type with private fields, it is an odd thing to do, as it sends a confusing, mixed-message about visibility. Downstream, forgetting to implement `Muckable` for an implicitly constructible type forces users to needlessly resort to unsafe transmutation. + +`Muckable` may be automatically derived for types that are publicly implicitly constructible, without posing a stability or safety hazard. The type `Foo` is effectively `Muckable` here: +``` +#[repr(C)] +pub struct Foo(pub u8, pub u16); +``` +...and here: +``` +#[repr(C)] +pub struct Foo(pub Bar, pub u16); + +#[repr(C)] +pub struct Bar; +``` +...and here: +``` +#[repr(C)] +pub struct Foo(pub T, pub U); +``` +A type is *not* effectively `Muckable` if its fields are not all `pub`, or if it is marked with `#[non_exhaustive]`, or if the fields themselves are not effectively `Muckable`. + ### Questions Out of Scope We consider the following unresolved questions to be out-of-scope of *this* RFC process: @@ -1105,7 +1063,19 @@ We consider the following unresolved questions to be out-of-scope of *this* RFC ## Safe Union Access -Given `TransmuteFrom`, the compiler can determine whether an access of a union variant of type `V` from a union `U` is safe by checking `V: TransmuteFrom`. In accesses where that bound is satisfied, the compiler can omit the requirement that the access occur in an `unsafe` block. +Given `TransmuteFrom`, the compiler can determine whether an access of a union variant of type `V` from a union `U` is safe by checking `V: TransmuteFrom`. In accesses where that bound is satisfied, the compiler can omit the requirement that the access occur in an `unsafe` block. + +## Limited Stability Guarantees +Implementing `Muckable` for a type allows for safe and stable transmutations *without* requiring the type's author to enumerate all useful transmutations (à la `From`), but at the cost of requiring full layout stability. For some use-cases, the reverse might be preferable: explicitly enumerate the set of stable transmutations *without* promising full layout stability. + +To accommodate this use-case, we could permit users to write implementations of `TransmuteFrom` in the form: +``` +unsafe impl TransmuteFrom for Bar +where + Bar: TransmuteFrom +{} +``` +Such implementations would conform to the usual orphan rules and would not permit users to override `TransmuteFrom`'s methods. ## Extension: Layout Property Traits [0000-ext-layout-traits.md]: https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-layout-traits.md From ee1204c23320c33437107ff222e09afd9d5fef1d Mon Sep 17 00:00:00 2001 From: Jack Wrenn Date: Mon, 2 Nov 2020 09:50:43 -0500 Subject: [PATCH 18/18] "safely and safely" -> "safely and stably" Co-authored-by: Ryan Levick --- text/0000-safer-transmute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-safer-transmute.md b/text/0000-safer-transmute.md index 0de7b7716c1..cf239c090b8 100644 --- a/text/0000-safer-transmute.md +++ b/text/0000-safer-transmute.md @@ -333,7 +333,7 @@ Since the well-definedness of a transmutation is affected by the layouts of the #### Signaling Safety and Stability with `Muckable` -To signal that your type may be safely and safely constructed via transmutation, implement the `Muckable` marker trait: +To signal that your type may be safely and stably constructed via transmutation, implement the `Muckable` marker trait: ```rust use mem::transmute::Muckable;