[Windows] Wrong ABI when interfacing with C code

[tomaka]

Here is a gist containing the entire source code if you want to reproduce.

When calling the function foo defined like this in C:

typedef struct {
    size_t a;
    size_t b;
    size_t c;
    size_t d;
} Foo;

size_t foo(void* a, void* b, void* c, void* d, Foo f) {
    return f.c;
}

And like this in Rust:

extern {
    fn foo(_: *const libc::c_void, _: *const libc::c_void, _: *const libc::c_void,
           _: *const libc::c_void, _: Foo) -> libc::size_t;
}

You get a crash with the MSVC version of Rust, but the MinGW version works.

This is because apparently the C code expects Foo to actually be a pointer.

If you change the Rust declaration to *const Foo, then it works with the MSVC version and crashes with the MinGW version.

[tomaka]

Someone pointed out that this is maybe not a MSVC-related issue.
I only tested 32 MinGW vs 64 bits MSVC, so maybe it's an issue of 32bits vs 64bits instead of MinGW vs MSVC.

[retep998]

Possibly related to #24427

[eefriedman]

rust/src/librustc_trans/trans/cabi_x86_win64.rs

Line 49 in cb5cd54

_ => ArgType::indirect(t, Some(Attribute::ByVal))

should probably be ArgType::indirect(t, None). The equivalent clang code: https://github.com/llvm-mirror/clang/blob/69b5694b76d36fde04c1f3511459a575e48f201d/lib/CodeGen/TargetInfo.cpp#L3338 .