miri function argument passing should use FnAbi

[RalfJung]

In the miri engine, when evaluating a function call, it can happen that caller and callee do not agree on the type of an argument. In this case, the argument effectively gets transmuted from the caller type to the callee type. However, this is not legal for all types, and whether it is legal depends on horrible details of the ABI. This logic is implemented for codegen, where it is called FnType. miri should probably use that same infrastructure.

The relevant code in miri that would need changing is here. Currently, we only allow argument type punning for the Rust ABI, and we only allow the valid range of a Scalar/ScalarPair to change -- effectively, we allow one side to have &T while the other side has *const T.

For the FnType side, I do not know much, but @eddyb offered to mentor someone trying to do this cleanup. :) He also left the following hints:

move some code from rustc_codegen_ssa and then compare ArgType pairwise.

The main issue to moving that stuff around is pointee_info_at (it would have to be moved to rustc::ty::layout, as an additional method in TyLayoutMethods):

rust/src/librustc_codegen_llvm/abi.rs

Line 487 in dae6c93

if let Some(pointee) = layout.pointee_info_at(cx, offset) {

Everything else is mostly relying on rustc_target doing the heavy lifting, already.

Cc @oli-obk

[wildarch]

It seems this issue has been open for a while, is it still relevant? If so, I would like to take a shot at it 😄.

[RalfJung]

Awesome! Nobody has been working on this so far, so you are free to grab it. :)

@eddyb your mentoring is needed :D

[wildarch]

I have started with the refactoring by moving the pointee_info_at function to TyLayoutMethods. For now I still need to figure out how to translate some of the calls that were previously using the CodegenCx reference to work with the different context type available in TyLayoutMethods.
I'm quite confident I can get that to work, the only thing I'm struggling with at the moment are the MaybeResult<TyLayout>s I get when calling layout_of on the context. It's the first time I encounter this trait, and I am not quite sure how to work with it, given that I eventually want to end up with an Option<PointeeInfo> instead of a MaybeResult<PointeeInfo>.

@eddyb do you have some pointers on this? I might also ask around on IRC. Perhaps I should change the signature to be MaybeResult<PointeeInfo>?

[wildarch]

Figured out the situation with the MaybeResult, will post another update when pointee_info_at has been moved

[wildarch]

I have created a duplicate implementation of pointee_info_at in TyLayoutMethods as suggested by @eddyb (I will remove the original in a later commit). I think the method by itself is okay now, except that I want to move the type_is_freeze method in src/librustc_codegen_ssa/common.rs to somewhere in the rustc crate, right now I have simply copied to implementation into pointee_info_at, which is far from ideal. Any ideas on where to put that function @eddyb or @RalfJung?

Current changes are in my fork, should I already turn it into a WIP pull request?

[oli-obk]

should I already turn it into a WIP pull request?

That's a good idea, because it makes commenting on the impl much easier

[RalfJung]

I want to move the type_is_freeze method in src/librustc_codegen_ssa/common.rs

I wonder why it even has such a method, can't it use is_freeze?

But anyway, @eddyb is much more qualified to answer these questions. ;)

[wildarch]

@RalfJung type_is_freeze actually calls out to is_freeze, providing the param_env and span arguments. The following two are equivalent:

cx.type_is_freeze(ty)
ty.is_freeze(cx, ParamEnv::reveal_all(), DUMMY_SPAN)

Instead of moving type_is_freeze I could also change the occurrences to use is_freeze, how do you feel about that?

[wildarch]

should I already turn it into a WIP pull request?

That's a good idea, because it makes commenting on the impl much easier

Done!

[RalfJung]

Instead of moving type_is_freeze I could also change the occurrences to use is_freeze, how do you feel about that?

Depends on how often it gets called. But really I have very little experience in rustc outside miri, so I'll leave this to @eddyb.

[saleemjaffer]

@eddyb Regarding moving FnType stuff from librustc_codegen_llvm/abi.rs to librustc_target/abi/call/mod.rs:

This is defined in librustc_codegen_llvm/abi.rs

pub trait FnTypeExt<'tcx> {
    fn of_instance(cx: &CodegenCx<'ll, 'tcx>, instance: &ty::Instance<'tcx>) -> Self;
    fn new(cx: &CodegenCx<'ll, 'tcx>,
           sig: ty::FnSig<'tcx>,
           extra_args: &[Ty<'tcx>]) -> Self;
    fn new_vtable(cx: &CodegenCx<'ll, 'tcx>,
                  sig: ty::FnSig<'tcx>,
                  extra_args: &[Ty<'tcx>]) -> Self;
    fn new_internal(
        cx: &CodegenCx<'ll, 'tcx>,
        sig: ty::FnSig<'tcx>,
        extra_args: &[Ty<'tcx>],
        mk_arg_type: impl Fn(Ty<'tcx>, Option<usize>) -> ArgType<'tcx, Ty<'tcx>>,
    ) -> Self;
    fn adjust_for_abi(&mut self,
                      cx: &CodegenCx<'ll, 'tcx>,
                      abi: Abi);
    fn llvm_type(&self, cx: &CodegenCx<'ll, 'tcx>) -> &'ll Type;
    fn ptr_to_llvm_type(&self, cx: &CodegenCx<'ll, 'tcx>) -> &'ll Type;
    fn llvm_cconv(&self) -> llvm::CallConv;
    fn apply_attrs_llfn(&self, llfn: &'ll Value);
    fn apply_attrs_callsite(&self, bx: &mut Builder<'a, 'll, 'tcx>, callsite: &'ll Value);
}

Looks like we need to move everything except

fn llvm_type(&self, cx: &CodegenCx<'ll, 'tcx>) -> &'ll Type;
fn ptr_to_llvm_type(&self, cx: &CodegenCx<'ll, 'tcx>) -> &'ll Type;
fn llvm_cconv(&self) -> llvm::CallConv;
fn apply_attrs_llfn(&self, llfn: &'ll Value);

These methods seem to so some conversion from FnType into what LLVM needs.

[RalfJung]

Also see what @eddyb wrote at rust-lang/miri#1038 (comment)

[RalfJung]

So... what would it take to use this in Miri? @eddyb @oli-obk and maybe @bjorn3
I assume I have to somehow get hold of an FnAbi around here

rust/compiler/rustc_mir/src/interpret/terminator.rs

Line 72 in 057050a

let (fn_val, abi, caller_can_unwind) = match *func.layout.ty.kind() {

and adjust the eval_fn_call function to take such a FnAbi and use it -- though I am not entirely clear on how to use it either. Would this replace one of the existing arguments, or would this be a new argument? But if it's a new argument then nothing forces eval_fn_call to even use it.^^

Is there code in our codegen backends that uses FnAbi that Miri should basically follow?