Do not const prop unions

[gurry]

Unions can produce values whose types don't match their underlying layout types which can lead to ICEs on eval.

Fixes #121534

[rustbot]

r? @cjgillot

rustbot has assigned @cjgillot.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

[oli-obk]

could add the bail out logic here

[gurry]

That was my first approach, but it looked ugly especially with that longish comment

[oli-obk]

r? oli-obk

@bors r+ rollup

[bors]

📌 Commit 633c92c has been approved by oli-obk

It is now in the queue for this repository.

[cjgillot]

I suspect that this PR does not fully fix #121534. From a glance, that ICE is caused by projecting a union field. This PR prohibits creating constants of union type in const-prop. We could still have constants of union type due to inline consts for instance.

[gurry]

I suspect that this PR does not fully fix #121534. From a glance, that ICE is caused by projecting a union field. This PR prohibits creating constants of union type in const-prop. We could still have constants of union type due to inline consts for instance.

That makes sense, but I tested it and the eval of inline consts does not ICE 😑.

This snippet, which is effectively the one reported in #121534, does ICE:

union Union {
    u32_field: u32,
    i32_field: i32,
}

pub fn main() {
    let u32_variant = Union { u32_field: 2 };
    let i32_variant = Union { i32_field: 10 };
    let a = unsafe { u32_variant.u32_field };
    let b = unsafe { i32_variant.u32_field };

    let _diff = b - a;
}

(Plaground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=9aa52ce49ce5c1a37bde946ab6e74631)

but this one, which evaluates an inline const with the exact same code in the body, does not:

union Union {
    u32_field: u32,
    i32_field: i32,
}

const C: u32 = {
    let u32_variant = Union { u32_field: 2 };
    let i32_variant = Union { i32_field: 10 };
    let a = unsafe { u32_variant.u32_field };
    let b = unsafe { i32_variant.u32_field };

    b - a
};

pub fn main() {
    let a = [3; C as usize];
    println!("{}", a.len())
}

(Playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=4bcf0f59f14964a9c090aa8bdc2d043e)

In fact, not only does it not ICE, it prints the correct value of 8.

I've looked into the location where the ICE originates in bin_int_op():

rust/compiler/rustc_const_eval/src/interpret/operator.rs

Line 209 in 548e14b

if left_layout.ty != right_layout.ty {

and found that in the first case the layout tys of the operands passed to this method are different (left_layout.ty is i32 and right_layout.ty is u32), but in the second case they are the same (both u32). I have not investigated further as to why this is. Perhaps some accidental twist of type inference.

@cjgillot If you can share a different reproducer, I can try again.