aaarch64: there is no way to use the FPU on a softfloat target without creating an ABI mess

[RalfJung]

On aarch64, if one is building for a softfloat target (in Rust that's e.g. the target aarch64-unknown-none-softfloat, which in particular sets -neon,-fp-armv8 target features by default), there seems to be no way to build some code that does make use of the FPU while remaining ABI-compatible with the rest of the binary: to enable use of the FPU, we have to set +fp-armv8, but this will inevitably also change which registers are being used to pass float arguments around.

On other targets, one can have the target do something like set +soft-float, so even if someone now enables e.g. SSE features on x86 or fpregs on ARM, code will be built with the softfloat ABI and thus be compatible with this target. But aarch64 doesn't seem to have something like +soft-float, meaning it is impossible to disentangle the float ABI from whether FPU instructions can be used.

Is there a specific reason this is currently not possible on aarch64, or is it just that nobody implemented this yet? Or am I missing some other way to generate aarch64 code that uses the FPU but uses a softfloat ABI?

[pinskia]

Turning off simd and/or fp disallows all usage of float types (and vector types).

[RalfJung]

What does "disallow" mean? LLVM still happily compiles such code. Rust also generally supports f32/f64 on all targets, and LLVM is perfectly capable of building such code even on aarch64 with the FPU turned off.

(This is an LLVM issue, not a clang issue.)

[pinskia]

(This is an LLVM issue, not a clang issue.)

Oh right and GCC does not disconnect the target options part from the backend ...
Anyways there is no soft-float ABI defined by Arm. And why would there be?

[pinskia]

softfloat is not a target that makes sense why LLVM has it I have no idea.

[pinskia]

Also clang rejects that target:
clang++: error: version 'softfloat' in target triple 'aarch64-unknown-none-softfloat' is invalid

[pinskia]

aarch64-unknown-none-softfloat triple does not exist outside of rust. Why was it added in the first place?

[RalfJung]

Also clang rejects that target:

You could try emulating this on clang by setting the -neon,-fp-armv8 target features. I don't know how to set those flags in clang.

Anyways there is no soft-float ABI defined by Arm. And why would there be?

LLVM defines one, I can't tell you why. 🤷

[pinskia]

With -march=armv8+nofp, float types are rejected with clang:

<source>:1:7: error: 'f' requires 'float' type support, but ABI 'aapcs' does not support it

[RalfJung]

aarch64-unknown-none-softfloat triple does not exist outside of rust. Why was it added in the first place?

Seems like that happened in rust-lang/rust#64589. Cc @andre-richter @Amanieu
Also Cc some other Rust folks that may know more about the how and why here, @jacobbramley @Darksonn @thejpster

[RalfJung]

With -march=armv8+nofp, float types are rejected with clang:

Okay, maybe this is not possible to reproduce with clang then.

However, LLVM seems to offer support for a softfloat ABI on aarch64 targets -- without providing sufficient control over whether and when that ABI is used. That's fundamentally what this issue is about.

[llvmbot]

@llvm/issue-subscribers-backend-aarch64

Author: Ralf Jung (RalfJung)

On aarch64, if one is building for a softfloat target (in Rust that's e.g. the target aarch64-unknown-none-softfloat, which in particular sets `-neon,-fp-armv8` target features by default), there seems to be no way to build some code that does make use of the FPU while remaining ABI-compatible with the rest of the binary: to enable use of the FPU, we have to set `+fp-armv8`, but this will inevitably also change which registers are being used to pass float arguments around.

On other targets, one can have the target do something like set +soft-float, so even if someone now enables e.g. SSE features on x86 or fpregs on ARM, code will be built with the softfloat ABI and thus be compatible with this target. But aarch64 doesn't seem to have something like +soft-float, meaning it is impossible to disentangle the float ABI from whether FPU instructions can be used.

Is there a specific reason this is currently not possible on aarch64, or is it just that nobody implemented this yet? Or am I missing some other way to generate aarch64 code that uses the FPU but uses a softfloat ABI?

[ostannard]

When we added the AArch64 soft-float ABI in clang (#74460, #84146), we deliberately didn't allow the combination of the soft-float ABI with floating-point hardware, to avoid there being multiple, incompatible ABIs for any one target.

However, the ABI used by clang is controlled by the -mabi= option, so you might be able to check how that's implemented and enable that combination in your compiler (is this for rust?). I'd be opposed to adding that option to clang without a very good use-case though, to avoid creating more compatibility issues.

[thejpster]

I don't understand the use case. AIUI Linux needs to avoid persisting FPU registers. This then implies a soft float ABI - but that's not the goal, merely an outcome.

Using the FPU means you need to persist the registers so ... why would you still want the soft-float ABI?

This is distinct from Arm 32-bit where the FPU is optional and two ABIs are defined and so compatibility with soft-float libraries is required.

[efriedma-quic]

clang currently has two "no-fp" configurations: -mgeneral-regs-only, and -mabi=aapcs-soft. Internally, they're actually the same thing, but -mgeneral-regs-only forbids any call that would pass a floating-point value in general registers. -mgeneral-regs-only is intended for places like the Linux kernel, which don't use any floating-point. -mabi=aapcs-soft is intended for microcontrollers that don't have an FPU.

It's hard for me to imagine a situation where you'd actually want a "soft-float" ABI and access to FP registers, given that all general-purpose AArch64 processors have an FPU. But if there is some use-case, we could look at adding it.

[RalfJung]

@efriedma-quic that's a fair question. The ability to enable neon and fp-armv8 on our aarch64-softfloat target has been added to Rust a few years ago, and I also don't quite know what the motivation for that is (I wasn't involved in the discussion back then)... but it's a feature we have now, it's hard to take back, so we'll likely have to support it.

We did, however, in the mean time come up with a plan that solves our problem and does not require LLVM changes. It boils down to avoiding the implicit LLVM-defined ABI for float types on the aarch64-softfloat target, and instead always passing them as i32/i64, with type conversions on the caller/callee side to convert from/to float types. This is architecturally a bit harder for us than if we could use the LLVM ABI but it should work.

[efriedma-quic]

It boils down to avoiding the implicit LLVM-defined ABI for float types on the aarch64-softfloat target, and instead always passing them as i32/i64, with type conversions on the caller/callee side to convert from/to float types.

This is going to break down in some edge cases... if you do have users for this, you probably want a real LLVM target feature.

[RalfJung]

Which kind of edge cases are you thinking about? We're using these kind of ABI overrides for a bunch of stuff already so I don't foresee any problems here -- in particular given that we don't have to be ABI-compatible with anything else since there is no standard softfloat ABI.

[efriedma-quic]

varargs (ABI rules require storing parts of floating-point registers when you va_start), intrinsics/libcalls where the backend generates a call.

[RalfJung]

It's hard for me to imagine a situation where you'd actually want a "soft-float" ABI and access to FP registers, given that all general-purpose AArch64 processors have an FPU. But if there is some use-case, we could look at adding it.

It seems like one motivation is for a kernel that is generally "nofloat" to temporarily enable use of the FPU in a kernel thread (after saving all the registers that clobbers), e.g. to execute some cryptographic code more efficiently.