Add support for the win64 calling convention

[eholk]

This was needed to access UEFI boot services in my new Boot2Rust experiment.

I also realized that Rust functions declared as extern always use the C calling convention regardless of how they were declared, so this pull request fixes that as well.

[alexcrichton]

It's unfortunate to not see any tests for this. From what it looks like, if I declare a rust function as extern "something not C" fn foo() and call it, it should segfault today, right? If so, can we add some tests to make sure that this doesn't regress?

[thestinger]

@alexcrichton: I expect this could expose some functions with a weird calling convention from one crate and call them from another, but you'd actually need to find a difference between the calling conventions and it would be very unlikely that it would segfault (as it's not going to break any memory protection rules in 99.9% of cases). You could write one where it would read the wrong registers or wrong bit of the stack I guess.

[alexcrichton]

I'm not sure that you can do this, but if you can write code which segfaults before this patch, then you definitely have a test case right there. I would like to at least explore the idea of adding a test before having no tests at all.

[thestinger]

It probably causes a segmentation fault after reading an incorrect value as the parameter (a null pointer from a register that happens to be zero, maybe).

[thestinger]

(temporarily removed r+, waiting for a test)

[emberian]

An easier test would be finding things which are passed differently, and asserting that a parameter has a specific value.

[eholk]

I'm working on a test case now. I'm making a function foo that takes 4 integers and makes sure they come in the same order. The win64 calling convention and the normal rust calling convention use different registers for argument passing, so the numbers come in the wrong order if the caller and callee don't agree on the calling convention. The only tricky part is you have to split the caller and the callee into separate crates because otherwise the LLVM optimizer is too smart. I'll add this change once I get a chance to make sure the test actually works.

[eholk]

Alright, I added the test case. I confirmed on that the test fails with the change that just adds support for the win64 convention, and it passes with the second change.

[alexcrichton]

Hm, I think that the tests may not pass the make check target, could you add the license block to the top of the files?

Otherwise, thanks! Looks good to me!

[eholk]

I added the copyright header. Surprisingly, I don't think make check checked for this.

I did get a rustpkg test failure about rebuilding a C file, but I think that is unrelated to this change.

[alexcrichton]

Could you also squash these commits into the first one being the win64 ABI/test case, and then the second is the ABI fix in general?

[brson]

Make tidy indeed doesn't currently check test cases: #4534.

[eholk]

Ok, I squashed these down, rebased onto the latest master and made sure make check passes.

[eholk]

@alexcrichton - would you might reviewing my latest addition? The tests failed on the 32-bit machines, so I've just substituted the normal calling convention there.