mem::transmute is too restrictive

[Diggsey]

Because mem::transmute checks its constraints before monomorphization, it fails for types containing fields whose type is determined by a generic parameter:

cannot transmute to or from a type that contains unsubstituted type parameters

It's great that it attempts to catch errors as early as possible, but if it's unable to prove that the 'to' and 'from' types have the same or different representations before monomorphization, it should fall back to checking them after, when it can get a definitive answer.

There's a slightly convoluted work-around which involves using ptr::read followed by mem::forget on the original to prevent it being dropped twice, but there's no guarantee that will compile into the same zero-cost operation as transmute, and it won't perform any checks that the two types have the same representation.

An alternative would be to leave the restrictions on mem::transmute unchanged and add a new intrinsic instead.

[bluss]

Rust doesn't want a solution that involves some monomorphizations working and others not. If the function definition type checks, all instantiations that pass the required type parameter bounds should compile.

[Diggsey]

A possible compromise would be for invalid transmutes which are detected after monomorphization to compile to an unconditional panic, and also emit a warning at compile time.

[huonw]

As you say, one can go the unconditional panic route via a transmute_copy wrapper:

use std::mem;
unsafe fn ugh_transmute<T, U>(x: T) -> U {
    assert_eq!(mem::size_of::<T>(), mem::size_of::<U>());
    let y = mem::transmute_copy(&x);
    mem::forget(x);
    y
}

In any case, the compile-time checks of transmute could be generalised in a more principled way with a trait like

/// `Self` has the same size as `T`
trait SizeOf<T> { /* compiler magic */ }

fn transmute<T, U>(x: T) -> U
    where U: SizeOf<T>

I believe the monomorphic-only rule allows to do this backwards compatibly (if we do it at all), which wouldn't necessarily be true with a panic! substitution.

[Diggsey]

The trait route you describe would require adding a SizeOf bound in many places throughout the code, possibly even in 3rd party code, which will often not be feasible.

[Aatch]

@Diggsey if you could provide a reasonable use case, I'd be much more open to considering this. I can't think of a case where this doesn't just scream "I'm doing something wrong" at me.

[Diggsey]

For a specific example, my PR #27573 would have been better implemented via transmute, rather than the workaround I had to use.

Basically for any current use of transmute, it can't be generalised across type parameters, which is unfortunate.

[oli-obk]

if there are no other use cases, the specific case of Option<NonZero<T>> having the same size as T could be added to transmute. This should obviously not be done for Option but for any type that hits this optimization.

The trait route you describe would require adding a SizeOf bound in many places throughout the code, possibly even in 3rd party code, which will often not be feasible.

Not in your use case ;), as everything is known inside the function. There's no reason for the outside to know about this.

there could additionally be an impl<T> SizeOf<T> for T {} and an impl<T> SizeOf<T> for Option<NonZero<T>> {}

see a crippled non-compiler-magic transmute at work in the playpen: http://is.gd/oiJQt3

[nikomatsakis]

@Diggsey

The trait route you describe would require adding a SizeOf bound in many places throughout the code, possibly even in 3rd party code, which will often not be feasible.

This is not necessarily true, because the current check is strictly more conservative than a bound would be (i.e., the bound would be met in all the cases we currently accept). At least that was the intention. I personally still favor some sort of bound as the way to generalize transmute.

[Diggsey]

@oli-obk

see a crippled non-compiler-magic transmute at work in the playpen:

Assuming you intend SizeOf<T> to be an unsafe trait which can be implemented for new types, that's just moving the problem elsewhere, eg:

unsafe impl<T> SizeOf<A<T>> for B<T> { }

The compiler still has to prove from that implementation that A<T> has the same size as B<T> for all possible T, which in general it cannot do before monomorphization.

@nikomatsakis

the bound would be met in all the cases we currently accept

So it would be possible to "materialize" the bound from nothing whenever the compiler could prove that it held?

Here's my take on the situation: with safe rust, the compiler lets you do anything it can prove is valid. With unsafe rust, it's reversed: the compiler should let you do anything it can't prove is invalid.

Adding bounds to transmute allows the compiler to prove that more transmutes are valid, but there will always be some which it can't prove either way. In that case, as this is an unsafe operation, it should give the programmer the benefit of the doubt, and assume they know what they're doing.

[nikomatsakis]

@Diggsey

So it would be possible to "materialize" the bound from nothing whenever the compiler could prove that it held?

To be clear, I was envisioning a "lang item" sort of trait, though if we make a SIZE_OF an associated constant, it may not need a special trait at all.

Adding bounds to transmute allows the compiler to prove that more transmutes are valid, but there will always be some which it can't prove either way. In that case, as this is an unsafe operation, it should give the programmer the benefit of the doubt, and assume they know what they're doing.

This isn't quite right. The compiler is not trying to show that your transmutes are valid. It's trying to check that it is even possible for us to generate code for them, without either synthesizing bits or dropping bits on the floor (in the former, what value should those bits have? in the latter, what bits should we drop?). So another plausible answer would be to say "transmute should just always compile to SOMETHING".

[arielb1]

@nikomatsakis

It could generate an unreachable

[steveklabnik]

So, what's the conclusion of this discussion?

[steveklabnik]

Triage: it's been a year since I asked my question, and it seems nobody has an answer. I'm going to give this one a close. If there's something still valuable here, please let me know!