Compiler segfaults when casting 0 as fat pointer

[apoelstra]

This is really nasty code, but simias on IRC was trying to get a null fat-pointer constant and this was one idea I tried.

type WillRobinson = u8;
trait Danger {}
impl Danger for WillRobinson {}

static mut danger: *mut Danger = (0 as *mut WillRobinson) as *mut Danger;

fn main() {}

This causes a segfault in rustc during compilation. The "unused variable" warning displays. I don't have a rustc with debug symbols, so not attaching backtrace, sorry.

Possibly related to #13520

[nagisa]

We’re hitting an assertion in LLVM:

rustc: /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/llvm/include/llvm/Support/Casting.h:237: typename llvm::cast_retty<X, Y*>::ret_type llvm::cast(Y*) [with X = llvm::GlobalVariable; Y = llvm::Value; typename llvm::cast_retty<X, Y*>::ret_type = llvm::GlobalVariable*]: Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.

BT:

#0  0x00007ffff70ce5f8 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff70cfa7a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff70c7417 in __assert_fail_base () from /usr/lib/libc.so.6
#3  0x00007ffff70c74c2 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00007ffff1af3670 in llvm::cast_retty<llvm::GlobalVariable, llvm::Value*>::ret_type llvm::cast<llvm::GlobalVariable, llvm::Value>(llvm::Value*) [clone .part.315] () from /usr/lib/librustc_llvm-17a8ccbd.so
#5  0x00007ffff1af7759 in LLVMGetInitializer () from /usr/lib/librustc_llvm-17a8ccbd.so
#6  0x00007ffff5f216bc in trans::consts::const_deref::h4c3a8b09e8457c33vrt () from /usr/lib/librustc_trans-17a8ccbd.so
#7  0x00007ffff5f20204 in trans::consts::const_expr::ha3c4de4b4bca8e8bFFt () from /usr/lib/librustc_trans-17a8ccbd.so
#8  0x00007ffff5f24500 in trans::consts::const_expr_unadjusted::h44f98122ca3bd1a5BXt () from /usr/lib/librustc_trans-17a8ccbd.so
#9  0x00007ffff5f1ff93 in trans::consts::const_expr::ha3c4de4b4bca8e8bFFt () from /usr/lib/librustc_trans-17a8ccbd.so
#10 0x00007ffff5eca923 in trans::base::trans_item::h9c19e9e4d1e730e5Xnj () from /usr/lib/librustc_trans-17a8ccbd.so
#11 0x00007ffff5ed6c55 in trans::base::trans_crate::h0270125f4db782cbK7j () from /usr/lib/librustc_trans-17a8ccbd.so
#12 0x00007ffff7a631d2 in driver::phase_4_translate_to_llvm::hbcd26352c79b2357uTa () from /usr/lib/librustc_driver-17a8ccbd.so
#13 0x00007ffff7a5d077 in driver::phase_3_run_analysis_passes::_$LT$closure$GT$::closure.26225 () from /usr/lib/librustc_driver-17a8ccbd.so
#14 0x00007ffff7a3ada6 in middle::ty::context::ctxt$LT$$u27$tcx$GT$::create_and_enter::create_and_enter::h6909965444811752025 () from /usr/lib/librustc_driver-17a8ccbd.so
#15 0x00007ffff7a366c2 in driver::phase_3_run_analysis_passes::h1635207307752584168 () from /usr/lib/librustc_driver-17a8ccbd.so
#16 0x00007ffff7a0b20a in driver::compile_input::h2a35f55d38a51b9ejca () from /usr/lib/librustc_driver-17a8ccbd.so
#17 0x00007ffff79fd41c in run_compiler::ha112fd42f27cc2125wc () from /usr/lib/librustc_driver-17a8ccbd.so
#18 0x00007ffff79fa1d7 in sys_common::unwind::try::try_fn::try_fn::h16052710914380154563 () from /usr/lib/librustc_driver-17a8ccbd.so
#19 0x00007ffff74f5b09 in __rust_try () from /usr/lib/libstd-17a8ccbd.so
#20 0x00007ffff74ed30c in sys_common::unwind::try::inner_try::hf02767e744fc6763dZs () from /usr/lib/libstd-17a8ccbd.so
#21 0x00007ffff79fa531 in boxed::F.FnBox$LT$A$GT$::call_box::call_box::h9288839064392637460 () from /usr/lib/librustc_driver-17a8ccbd.so
#22 0x00007ffff74fcec4 in sys::thread::Thread::new::thread_start::h1da3018ef918f37crjx () from /usr/lib/libstd-17a8ccbd.so
#23 0x00007ffff02944a4 in start_thread () from /usr/lib/libpthread.so.0
#24 0x00007ffff718413d in clone () from /usr/lib/libc.so.6

[apoelstra]

Maybe this is a separate bug?:

trait Danger {}
struct DangerDummy;
impl Danger for DangerDummy {}

static mut danger: *mut Danger = (&mut DangerDummy as *mut DangerDummy) as *mut Danger;

fn main() {
}

{ i8*, void (i8*)** } { i8* null, void (i8*)** getelementptr inbounds ({ void (i8*)*, i64, i64 }, { void (i8*)*, i64, i64 }* @vtable2631, i32 0, i32 0) }
{} undef
error: internal compiler error: const expr(18: &mut DangerDummy as *mut DangerDummy) of type DangerDummy has size 16 instead of 0
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: run with `RUST_BACKTRACE=1` for a backtrace
thread 'rustc' panicked at 'Box<Any>', ../src/libsyntax/diagnostic.rs:253

playpen: application terminated with error code 101

[nagisa]

Its the same thing. LLVM will report the same error in different ways depending on optimisation and debug settings it was built with – and also for different IRs that essentially have the same issue.

[arielb1]

This is the coercion code being silly.

DEBUG:rustc_typeck::check::cast: check_cast(25, *const u8 as *const core::cmp::PartialEq<u8>)                                                                                 
DEBUG:rustc_typeck::check::cast:  -> CoercionCast                                      

cc @eddyb

[eddyb]

@arielb1 It's a valid coercion, and the reason smart pointers have DST support.
I don't understand how this ends up in const_deref though.

[arielb1]

So its the coercion trans code rather than the coercion typeck code being silly.

[arielb1]

I think a cast gets skipped or something.

[arielb1]

The problem is that translating 1 as *const u8 somehow skips the cast and ends up returning a i64

[steveklabnik]

I'm not sure if this is the same bug or not, but steveklabnik@b5b2602#diff-da456bd3af6d94a9693e625ff7303113R787 SEEMED to be causing a segfault as well, without the extra casts. I'm trying to investigate a bit more now.

[sanxiyn]

This was fixed by #32430.