Add x86_64-unknown-uefi target

[dvdhrm]

This adds a new rustc target-configuration called 'x86_64-unknown_uefi'.
Furthermore, it adds a UEFI base-configuration to be used with other
targets supported by UEFI (e.g., i386, armv7hl, aarch64, itanium, ...).

UEFI systems provide a very basic operating-system environment, meant
to unify how systems are booted. It is tailored for simplicity and fast
setup, as it is only meant to bootstrap other systems. For instance, it
copies most of the ABI from Microsoft Windows, rather than inventing
anything on its own. Furthermore, any complex CPU features are
disabled. Only one CPU is allowed to be up, no interrupts other than
the timer-interrupt are allowed, no process-separation is performed,
page-tables are identity-mapped, ...

Nevertheless, UEFI has an application model. Its main purpose is to
allow operating-system vendors to write small UEFI applications that
load their kernel and terminate the UEFI system. However, many other
UEFI applications have emerged in the past, including network-boot,
debug-consoles, and more.

This UEFI target allows to compile rust code natively as UEFI
applications. No standard library support is added, but libcore can be
used out-of-the-box if a panic-handler is provided. Furthermore,
liballoc works as well, if a GlobalAlloc handler is provided. Both
have been tested with this target-configuration.

Note that full libstd support is unlikely to happen. While UEFI does
have standardized interfaces for networking and alike, none of these
are mandatory and they are unlikely to be shipped in common consumer
firmwares. Furthermore, several features like process-separation are
not available (or only in very limited fashion). Those parts of libstd
would have to be masked.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @cramertj (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[cramertj]

r? @alexcrichton

[alexcrichton]

Thanks for this! My only hesitation would be that this defaults to link.exe for a linker, but that means this target would largely only work on Windows I think? That seems like a plausible place to start, but I imagine that we'd want to compile for this target on systems like Linux as well, right?

[IsaacWoods]

@alexcrichton I believe this uses lld-link as the linker if I'm not mistaken, which is available from Linux as well and is able to compile UEFI compatible PEs

[dvdhrm]

Thanks for this! My only hesitation would be that this defaults to link.exe for a linker, but that means this target would largely only work on Windows I think? That seems like a plausible place to start, but I imagine that we'd want to compile for this target on systems like Linux as well, right?

Ah, no, it doesn't default to link.exe. It defaults to the link backend of lld, which on linux, defaults to lld-link, not link.exe. I tested this on my linux machine and it works fine. This is a bit confusing, because lld has 2 backends, one for ELF (called lld-ld) and one for COFF (called lld-link), named after their respective native counterparts.

Thing is, you need a COFF/PE32+ linker for UEFI targets. So if you run this on linux, it is as if you cross-compile for Windows (UEFI and windows targets are almost the same). Since lld-link is a built-in of lld, it is a perfect default linker.
An alternative to lld would be to use GNU ld. But ld has the problem that each binary only supports one target triple. So to link COFF/PE32+ binaries, you need an ld cross-toolchain for windows (usually known as mingw-for-linux), which is only partially available in most mainstream distributions.
I personally admire the LLVM-lld approach of combining all target compilers/linkers in a single binary. And since LLVM is already available for rustc, I thought lld-link as default linker sounds good.

Long story short: The lld-link link-flavor just selects the COFF backend of lld as default linker.

[alexcrichton]

@bors: r+

Oh wow sorry about that, I completely missed that aspect! Glad LLD can save the day again for us here :)

[bors]

📌 Commit 88cf2a2 has been approved by alexcrichton

[alexcrichton]

@bors: rollup

[nagisa]

How does this work for application writers?

Are they responsible for defining an efi_main symbol? Perhaps not the best approach then, as we already have #[start] and expect function annotated with that attribute to get invoked at start-up even in #[no_std] contexts.

[dvdhrm]

Some crate in your application must provide the efi_main symbol. If we port std, we could make the #[start] logic do this. However, I don't think core should provide wrappers around the entry-point. We explicitly want a way to link applications with full control over the entry-point. With this target, you need:

pub extern fn efi_main(_h: efi::Handle, st: *mut efi::SystemTable) -> efi::Status

..to exist somewhere in your application.

I expect applications to pull in helper crates that wrap the UEFI API with a safe rust interface. Those crates can then provide the entry-point and retain the system-table somehow. However, note that due to ExitBootServices(), it is non-trivial to safely wrap all of the UEFI API (at least it requires some thought). By blocking ExitBootServices() and the VM-remapping, it is at least easier to imagine a safe wrapper.

We already have a public crate that simply provides the symbols from the UEFI spec as rust symbols (really just copying the definitions, not wrapper code in there) at https://github.com/r-util/r-efi
Our safe wrappers are still being worked on.

[nagisa]

Seems debatable to me. Are you sure this is correct (as opposed to just "x86_64"?)

[dvdhrm]

You have to trick llvm into creating COFF objects, rather than ELF objects. In theory this shouldn't matter. You should be able to use any available target compiler to get your object files, and then in the end tell your linker about the target format. However, this sadly does not work. The different object files have different capabilities, or specific annotations are only supported in one, not the other, etc. So I never succeeded in linking coff-objects into ELF-executables, or vice versa.

Preferably, I would use x86_64-unknown-coff, but that does not exit. If anyone has better suggestions, I would gladly switch over.

Selecting the windows target might cause LLVM to optimize for something that will not apply to LLVM. However, I don't think anyone can apply any assumptions on a generic target like the one I picked, because it has to be backwards-compatible to all the existing windows versions. So I believe this will be safe.

I contacted colleagues of mine here at Red Hat, maybe they have a suggestion. Sadly, it is all gcc, not LLVM ;)

[nagisa]

Makes sense then. It appears that llc has no way to specify format of the produced object file either, which most likely makes the target specification like this the best approach.

[nagisa]

Nit: x64 UEFI uses the exact same msabi (aka. x64) calling convention that windows uses. There is not a single difference (and therefore “mostly” is not applicable).

[dvdhrm]

Are you sure about that? UEFI explicitly disallows passing values bigger than 64bit by value (both as argument or return value; see UEFI-2.7 2.3.4.2). Isn't this allowed in msabi?

[nagisa]

UEFI explicitly disallows passing values bigger than 64bit by value (both as argument or return value; see UEFI-2.7 2.3.4.2). Isn't this allowed in msabi?

Other than XMM/floating point registers, nothing larger than 64-bit is passed by-register in x64 msabi.

[dvdhrm]

Other than XMM/floating point registers, nothing larger than 64-bit is passed by-register in x64 msabi.

Well, in registers, yes, but is that true for the stack as well? Does msabi require you to put objects bigger than 64bit in caller allocated memory and pass it by reference? Because that is how I read the UEFI spec, even for arguments passed on the stack.

The caller passes arrays and strings via a pointer to memory allocated by the
caller. The caller passes structures and unions of size 8, 16, 32, or 64 bits as if
they were integers of the same size. The caller is not allowed to pass structures
and unions of other than these sizes and must pass these unions and structures
via a pointer.

[nagisa]

Yes. Either way, this is proved by the fact that it wasn’t necessary for you to do anything special to implement UEFI’s ABI: it was already implemented!

[dvdhrm]

Right, the MSDN docs are almost identical. Thanks a lot for the link!

I certainly expecteded the ABIs to be compatible. I just wasn't sure whether the UEFI ABI is a subset (e.g., their coding-convention forbids returning anything but ints, which I expected to be rooted in a restricted ABI). I am confident using the msabi will just work, given that's what tianocore does as well.

[nagisa]

Cool trick with lld though!

[kkebo]

Why isn’t this using rust-lld as a linker? It is able to create an UEFI application and bundled with the default toolchain, so an external linker like lld-link is not needed.

[dvdhrm]

rust-lld is only available for tier-1 platforms. It is not shipped with the others. I am not sure what the reasoning is, and whether that is still true. Furthermore, I considered the system linker to be preferable to a bundled one. I am not sure why rust-lld is bundled at all.
To be quite honest, I thought using the system linker is the expected behavior. If you have more insight on this topic, please lemme know what the recommended way here is (or, of course, feel free to open a PR).

[Rua]

When trying to compile a project with this target, lld-link can't be found. lld-link-6.0 exists, though. I'm using Linux Mint 19.1.

[joshtriplett]

@dvdhrm

While UEFI does have standardized interfaces for networking and alike, none of these are mandatory and they are unlikely to be shipped in common consumer firmwares.

I've used those interfaces quite a bit, and they're available on many common firmware images. The filesystem interfaces are always available. And in general, almost all of what std needs should be regularly available, other than things that depend on fork or similar.

[dvdhrm]

I've used those interfaces quite a bit, and they're available on many common firmware images.

I know implementations exist, and I know that people use them. Can you clarify what "many common firmware images" means? Does that include low-budget devices? Does it mean 50% of devices? 10%? Or 99.9%?

I know that we had issues with the most simple of UEFI protocols being implemented wrongly in consumer hardware (speaking of efi_simple_input_protocol and efi_graphics_output_protocol), and we had to work around peculiarities in very simple projects like gummiboot, sd-boot, efifb, etc. (see their implementations for details). So I am a bit skeptical if you claim that the more elaborate protocols like the network stack exists on many devices. I hope you are right ;)

The filesystem interfaces are always available. And in general, almost all of what std needs should be regularly available, other than things that depend on fork or similar.

Just because a protocol is available does not mean that you can use it. For instance, to safely use any of the UEFI protocols that support device hotplugging, you must register a EFI_DRIVER_BINDING_PROTOCOL instance yourself. Otherwise, there is no way that you are notified of devices being removed. For instance, UEFI Applications (unlike UEFI Drivers) usually only make use of singleton protocols (the ones that multiplex input/output of multiple devices on a single interface). While technically nobody stops you from using other protocols, they can be dropped at any point in time without you being notified, thus getting lots of stale pointers.

I still think porting std to UEFI is more complex than it initially looks. Furthermore, I think it requires a lot of background tasks to be performed, which you preferably don't want your standard library to do. Lastly, it requires global state which works against what the UEFI spec tries to build.

[joshtriplett]

I've certainly encountered devices that don't have the EFI network protocols, I'm just saying I've run into many devices that do (various random PC hardware, both laptops and desktops). There'd be value in being able to write DXE applications using Rust, and also in writing PEI or similar code in Rust without std.

The stock Tiano core includes the network protocols, so it's more a matter of whether the device vendor shipped a working driver for network hardware.