-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
s390x vector facilities support #130869
Comments
cc @uweigand |
Thanks for putting together this list! I can certainly look into providing proper vector register ABI support. |
I've looked into this a bit, but ran into a problem I'm not sure how to address. The main issue is that on s390x, we use a different ABI depending on whether or not the vector feature is active (i.e. the processor has vector registers). The difference affects only vector types (which I guess would correspond to
The calling convention should be implemented in the In addition, I'm not sure where to handle the vector alignment. This currently seems to be derived solely from the datalayout string, which likely isn't correct even now - the datalayout string gives the alignment for the LLVM vector types, which is always bounded to 8 bytes, but Clang explicitly uses a different alignment for C/C++ level vector types if necessary. |
@uweigand Thanks for the investigation!
Hmm, I thought it is always 8-byte aligned since LLVM 16 (2e7a964).
I think we can implement this by referring to wasm code (1, 2) that determines the ABI depending on the command line option. |
I haven't done much testing yet, but this approach appears to be working: master...taiki-e:rust:s390x-vector-abi // no_vector: mvc 8(8,%r2), 8(%r3)
// no_vector-NEXT: mvc 0(8,%r2), 0(%r3)
// no_vector-NEXT: br %r14
// vector: vl %v24, 0(%r2), 3
// vector-NEXT: br %r14
#[no_mangle]
extern "C" fn vector(x: &i8x16) -> i8x16 {
*x
} |
The point is that there's a difference between the LLVM IR vector types and the C/C++ language vector types. The LLVM IR vector types are indeed always 8-byte aligned now. But the C/C++ language vector types still use different ABI alignment rules depending on the vector feature. Clang handles this by mapping the C/C++ type not directly to an LLVM IR type (leaving LLVM to use its own alignment rules), but rather to an LLVM IR type with an explicit alignment override (which implements the appropriate ABI alignment rule).
Ah, interesting! It seems this
calls back into the LLVM back-end, which does already know which features are available by default depending on the target-cpu setting. I wasn't aware of that ... I'm wondering whether/how this works when using another codegen backend like GCC or Cranelift? Otherwise, your patch looks good to me, except for
|
Ah, thanks for the clarification.
EDIT: I think #130869 (comment) 's approach will work.
rustc_codegen_gcc seems to have code to handle IIRC rustc_codegen_cranelift does not currently support the target feature at all (rust-lang/rustc_codegen_cranelift#1400), so I suspect that vector ABI cannot be enabled from any way.
It looks like the vector_small case is processed in the branch before processing the 128-bit vector. rust/compiler/rustc_target/src/abi/call/s390x.rs Lines 35 to 38 in a805e3e
However, I think we should handle it explicitly in the
Thanks for pointing that out. I tried to fix this, but it appears that the ABI of Wrapper<Vector> is considered Vector, even without |
Rust plans to disallow passing vector types to non-Rust ABI if the required target feature is disabled (#127731), so I think the issue of ABI differences with C/C++ when |
Hmm. Your test case uses the default (Rust) representation for the Wrapper type - my understanding is that this does not actually guarantee interoperability with the native platform ABI. Only
That looks like a reasonable approach to me as well. |
Oh, you are right. Changing Wrapper to |
…nieu Stabilize s390x inline assembly This stabilizes inline assembly for s390x (SystemZ). Corresponding reference PR: rust-lang/reference#1643 --- From the requirements of stabilization mentioned in rust-lang#93335 > Each architecture needs to be reviewed before stabilization: > - It must have clobber_abi. Done in rust-lang#130630. > - It must be possible to clobber every register that is normally clobbered by a function call. Done in the PR that added support for clobber_abi. > - Generally review that the exposed register classes make sense. The followings can be used as input/output: - `reg` (`r[0-10]`, `r[12-14]`): General-purpose register - `reg_addr` (`r[1-10]`, `r[12-14]`): General-purpose register except `r0` which is evaluated as zero in an address context This class is needed because `r0`, which may be allocated when using the `reg` class, cannot be used as a register in certain contexts. This is identical to the `a` constraint in LLVM and GCC. See rust-lang#119431 for details. - `freg` (`f[0-15]`): Floating-point register The followings are clobber-only: - `vreg` (`v[0-31]`): Vector register Technically `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `vector` target feature added is enabled, but `core::arch` has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable. Everything related is unstable, so the fact that this is currently a clobber-only should not be considered a stabilization blocker. (rust-lang#130869 tracks unstable stuff here) - `areg` (`a[2-15]`): Access register All of the above register classes except `reg_addr` are needed for `clobber_abi`. The followings cannot be used as operands for inline asm (see also [getReservedRegs](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp#L258-L282) and [SystemZELFRegisters](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.h#L107-L128) in LLVM): - `r11`: frame pointer - `r15`: stack pointer - `a0`, `a1`: Reserved for system use - `c[0-15]` (control register) Reserved by the kernel Although not listed in the above requirements, `preserves_flags` is implemented in rust-lang#111331. --- cc `@uweigand` r? `@Amanieu` `@rustbot` label +O-SystemZ +A-inline-assembly
…nieu Stabilize s390x inline assembly This stabilizes inline assembly for s390x (SystemZ). Corresponding reference PR: rust-lang/reference#1643 --- From the requirements of stabilization mentioned in rust-lang#93335 > Each architecture needs to be reviewed before stabilization: > - It must have clobber_abi. Done in rust-lang#130630. > - It must be possible to clobber every register that is normally clobbered by a function call. Done in the PR that added support for clobber_abi. > - Generally review that the exposed register classes make sense. The followings can be used as input/output: - `reg` (`r[0-10]`, `r[12-14]`): General-purpose register - `reg_addr` (`r[1-10]`, `r[12-14]`): General-purpose register except `r0` which is evaluated as zero in an address context This class is needed because `r0`, which may be allocated when using the `reg` class, cannot be used as a register in certain contexts. This is identical to the `a` constraint in LLVM and GCC. See rust-lang#119431 for details. - `freg` (`f[0-15]`): Floating-point register The followings are clobber-only: - `vreg` (`v[0-31]`): Vector register Technically `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `vector` target feature added is enabled, but `core::arch` has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable. Everything related is unstable, so the fact that this is currently a clobber-only should not be considered a stabilization blocker. (rust-lang#130869 tracks unstable stuff here) - `areg` (`a[2-15]`): Access register All of the above register classes except `reg_addr` are needed for `clobber_abi`. The followings cannot be used as operands for inline asm (see also [getReservedRegs](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp#L258-L282) and [SystemZELFRegisters](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.h#L107-L128) in LLVM): - `r11`: frame pointer - `r15`: stack pointer - `a0`, `a1`: Reserved for system use - `c[0-15]` (control register) Reserved by the kernel Although not listed in the above requirements, `preserves_flags` is implemented in rust-lang#111331. --- cc ``@uweigand`` r? ``@Amanieu`` ``@rustbot`` label +O-SystemZ +A-inline-assembly
Rollup merge of rust-lang#131258 - taiki-e:s390x-stabilize-asm, r=Amanieu Stabilize s390x inline assembly This stabilizes inline assembly for s390x (SystemZ). Corresponding reference PR: rust-lang/reference#1643 --- From the requirements of stabilization mentioned in rust-lang#93335 > Each architecture needs to be reviewed before stabilization: > - It must have clobber_abi. Done in rust-lang#130630. > - It must be possible to clobber every register that is normally clobbered by a function call. Done in the PR that added support for clobber_abi. > - Generally review that the exposed register classes make sense. The followings can be used as input/output: - `reg` (`r[0-10]`, `r[12-14]`): General-purpose register - `reg_addr` (`r[1-10]`, `r[12-14]`): General-purpose register except `r0` which is evaluated as zero in an address context This class is needed because `r0`, which may be allocated when using the `reg` class, cannot be used as a register in certain contexts. This is identical to the `a` constraint in LLVM and GCC. See rust-lang#119431 for details. - `freg` (`f[0-15]`): Floating-point register The followings are clobber-only: - `vreg` (`v[0-31]`): Vector register Technically `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `vector` target feature added is enabled, but `core::arch` has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable. Everything related is unstable, so the fact that this is currently a clobber-only should not be considered a stabilization blocker. (rust-lang#130869 tracks unstable stuff here) - `areg` (`a[2-15]`): Access register All of the above register classes except `reg_addr` are needed for `clobber_abi`. The followings cannot be used as operands for inline asm (see also [getReservedRegs](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp#L258-L282) and [SystemZELFRegisters](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.h#L107-L128) in LLVM): - `r11`: frame pointer - `r15`: stack pointer - `a0`, `a1`: Reserved for system use - `c[0-15]` (control register) Reserved by the kernel Although not listed in the above requirements, `preserves_flags` is implemented in rust-lang#111331. --- cc ``@uweigand`` r? ``@Amanieu`` ``@rustbot`` label +O-SystemZ +A-inline-assembly
Stabilize s390x inline assembly This stabilizes inline assembly for s390x (SystemZ). Corresponding reference PR: rust-lang/reference#1643 --- From the requirements of stabilization mentioned in rust-lang/rust#93335 > Each architecture needs to be reviewed before stabilization: > - It must have clobber_abi. Done in rust-lang/rust#130630. > - It must be possible to clobber every register that is normally clobbered by a function call. Done in the PR that added support for clobber_abi. > - Generally review that the exposed register classes make sense. The followings can be used as input/output: - `reg` (`r[0-10]`, `r[12-14]`): General-purpose register - `reg_addr` (`r[1-10]`, `r[12-14]`): General-purpose register except `r0` which is evaluated as zero in an address context This class is needed because `r0`, which may be allocated when using the `reg` class, cannot be used as a register in certain contexts. This is identical to the `a` constraint in LLVM and GCC. See rust-lang/rust#119431 for details. - `freg` (`f[0-15]`): Floating-point register The followings are clobber-only: - `vreg` (`v[0-31]`): Vector register Technically `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `vector` target feature added is enabled, but `core::arch` has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable. Everything related is unstable, so the fact that this is currently a clobber-only should not be considered a stabilization blocker. (rust-lang/rust#130869 tracks unstable stuff here) - `areg` (`a[2-15]`): Access register All of the above register classes except `reg_addr` are needed for `clobber_abi`. The followings cannot be used as operands for inline asm (see also [getReservedRegs](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp#L258-L282) and [SystemZELFRegisters](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.h#L107-L128) in LLVM): - `r11`: frame pointer - `r15`: stack pointer - `a0`, `a1`: Reserved for system use - `c[0-15]` (control register) Reserved by the kernel Although not listed in the above requirements, `preserves_flags` is implemented in rust-lang/rust#111331. --- cc ``@uweigand`` r? ``@Amanieu`` ``@rustbot`` label +O-SystemZ +A-inline-assembly
…nieu Stabilize s390x inline assembly This stabilizes inline assembly for s390x (SystemZ). Corresponding reference PR: rust-lang/reference#1643 --- From the requirements of stabilization mentioned in rust-lang#93335 > Each architecture needs to be reviewed before stabilization: > - It must have clobber_abi. Done in rust-lang#130630. > - It must be possible to clobber every register that is normally clobbered by a function call. Done in the PR that added support for clobber_abi. > - Generally review that the exposed register classes make sense. The followings can be used as input/output: - `reg` (`r[0-10]`, `r[12-14]`): General-purpose register - `reg_addr` (`r[1-10]`, `r[12-14]`): General-purpose register except `r0` which is evaluated as zero in an address context This class is needed because `r0`, which may be allocated when using the `reg` class, cannot be used as a register in certain contexts. This is identical to the `a` constraint in LLVM and GCC. See rust-lang#119431 for details. - `freg` (`f[0-15]`): Floating-point register The followings are clobber-only: - `vreg` (`v[0-31]`): Vector register Technically `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `vector` target feature added is enabled, but `core::arch` has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable. Everything related is unstable, so the fact that this is currently a clobber-only should not be considered a stabilization blocker. (rust-lang#130869 tracks unstable stuff here) - `areg` (`a[2-15]`): Access register All of the above register classes except `reg_addr` are needed for `clobber_abi`. The followings cannot be used as operands for inline asm (see also [getReservedRegs](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp#L258-L282) and [SystemZELFRegisters](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.h#L107-L128) in LLVM): - `r11`: frame pointer - `r15`: stack pointer - `a0`, `a1`: Reserved for system use - `c[0-15]` (control register) Reserved by the kernel Although not listed in the above requirements, `preserves_flags` is implemented in rust-lang#111331. --- cc ``@uweigand`` r? ``@Amanieu`` ``@rustbot`` label +O-SystemZ +A-inline-assembly
…er-errors,uweigand Support s390x z13 vector ABI cc rust-lang#130869 This resolves the following fixmes: - https://github.com/rust-lang/rust/blob/58420a065b68ecb3eec03b942740c761cdadd5c4/compiler/rustc_target/src/abi/call/s390x.rs#L1-L2 - https://github.com/rust-lang/rust/blob/58420a065b68ecb3eec03b942740c761cdadd5c4/compiler/rustc_target/src/spec/targets/s390x_unknown_linux_gnu.rs#L9-L11 Refs: Section 1.2.3 "Parameter Passing" and section 1.2.5 "Return Values" in ELF Application Binary Interface s390x Supplement, Version 1.6.1 (lzsabi_s390x.pdf in https://github.com/IBM/s390x-abi/releases/tag/v1.6.1) This PR extends ~~rust-lang#127731 rust-lang#132173 (merged) 's ABI check to handle cases where `vector` target feature is disabled. If we do not do ABI check, we run into the ABI problems as described in rust-lang#116558 and rust-lang#130869 (comment), and the problem of the compiler generating strange code (rust-lang#131586 (comment)). cc `@uweigand` `@rustbot` label +O-SystemZ +A-ABI
Rollup merge of rust-lang#131586 - taiki-e:s390x-vector-abi, r=compiler-errors,uweigand Support s390x z13 vector ABI cc rust-lang#130869 This resolves the following fixmes: - https://github.com/rust-lang/rust/blob/58420a065b68ecb3eec03b942740c761cdadd5c4/compiler/rustc_target/src/abi/call/s390x.rs#L1-L2 - https://github.com/rust-lang/rust/blob/58420a065b68ecb3eec03b942740c761cdadd5c4/compiler/rustc_target/src/spec/targets/s390x_unknown_linux_gnu.rs#L9-L11 Refs: Section 1.2.3 "Parameter Passing" and section 1.2.5 "Return Values" in ELF Application Binary Interface s390x Supplement, Version 1.6.1 (lzsabi_s390x.pdf in https://github.com/IBM/s390x-abi/releases/tag/v1.6.1) This PR extends ~~rust-lang#127731 rust-lang#132173 (merged) 's ABI check to handle cases where `vector` target feature is disabled. If we do not do ABI check, we run into the ABI problems as described in rust-lang#116558 and rust-lang#130869 (comment), and the problem of the compiler generating strange code (rust-lang#131586 (comment)). cc `@uweigand` `@rustbot` label +O-SystemZ +A-ABI
…anieu Support input/output in vector registers of s390x inline assembly (under asm_experimental_reg feature) This extends currently clobber-only vector registers (`vreg`) support to allow passing `#[repr(simd)]` types, floats (f32/f64/f128), and integers (i32/i64/i128) as input/output. This is unstable and gated under new `#![feature(asm_experimental_reg)]` (tracking issue: rust-lang#133416). If the feature is not enabled, only clober is supported as before. | Architecture | Register class | Target feature | Allowed types | | ------------ | -------------- | -------------- | -------------- | | s390x | `vreg` | `vector` | `i32`, `f32`, `i64`, `f64`, `i128`, `f128`, `i8x16`, `i16x8`, `i32x4`, `i64x2`, `f32x4`, `f64x2` | This matches the list of types that are supported by the vector registers in LLVM: https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.td#L301-L313 In addition to `core::simd` types and floats listed above, custom `#[repr(simd)]` types of the same size and type are also allowed. All allowed types other than i32/f32/i64/f64/i128, and relevant target features are currently unstable. Currently there is no SIMD type for s390x in `core::arch`, but this is tracked in rust-lang#130869. cc rust-lang#130869 about vector facility support in s390x cc rust-lang#125398 & rust-lang#116909 about f128 support in asm `@rustbot` label +O-SystemZ +A-inline-assembly
Rollup merge of rust-lang#131664 - taiki-e:s390x-asm-vreg-inout, r=Amanieu Support input/output in vector registers of s390x inline assembly (under asm_experimental_reg feature) This extends currently clobber-only vector registers (`vreg`) support to allow passing `#[repr(simd)]` types, floats (f32/f64/f128), and integers (i32/i64/i128) as input/output. This is unstable and gated under new `#![feature(asm_experimental_reg)]` (tracking issue: rust-lang#133416). If the feature is not enabled, only clober is supported as before. | Architecture | Register class | Target feature | Allowed types | | ------------ | -------------- | -------------- | -------------- | | s390x | `vreg` | `vector` | `i32`, `f32`, `i64`, `f64`, `i128`, `f128`, `i8x16`, `i16x8`, `i32x4`, `i64x2`, `f32x4`, `f64x2` | This matches the list of types that are supported by the vector registers in LLVM: https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.td#L301-L313 In addition to `core::simd` types and floats listed above, custom `#[repr(simd)]` types of the same size and type are also allowed. All allowed types other than i32/f32/i64/f64/i128, and relevant target features are currently unstable. Currently there is no SIMD type for s390x in `core::arch`, but this is tracked in rust-lang#130869. cc rust-lang#130869 about vector facility support in s390x cc rust-lang#125398 & rust-lang#116909 about f128 support in asm `@rustbot` label +O-SystemZ +A-inline-assembly
Support input/output in vector registers of s390x inline assembly (under asm_experimental_reg feature) This extends currently clobber-only vector registers (`vreg`) support to allow passing `#[repr(simd)]` types, floats (f32/f64/f128), and integers (i32/i64/i128) as input/output. This is unstable and gated under new `#![feature(asm_experimental_reg)]` (tracking issue: rust-lang/rust#133416). If the feature is not enabled, only clober is supported as before. | Architecture | Register class | Target feature | Allowed types | | ------------ | -------------- | -------------- | -------------- | | s390x | `vreg` | `vector` | `i32`, `f32`, `i64`, `f64`, `i128`, `f128`, `i8x16`, `i16x8`, `i32x4`, `i64x2`, `f32x4`, `f64x2` | This matches the list of types that are supported by the vector registers in LLVM: https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.td#L301-L313 In addition to `core::simd` types and floats listed above, custom `#[repr(simd)]` types of the same size and type are also allowed. All allowed types other than i32/f32/i64/f64/i128, and relevant target features are currently unstable. Currently there is no SIMD type for s390x in `core::arch`, but this is tracked in rust-lang/rust#130869. cc rust-lang/rust#130869 about vector facility support in s390x cc rust-lang/rust#125398 & rust-lang/rust#116909 about f128 support in asm `@rustbot` label +O-SystemZ +A-inline-assembly
This tracks the status of s390x vector facilities support in rustc and standard libraries.
rust/compiler/rustc_target/src/abi/call/s390x.rs
Lines 1 to 2 in 58420a0
rust/compiler/rustc_target/src/spec/targets/s390x_unknown_linux_gnu.rs
Lines 9 to 11 in 58420a0
cfg(target_feature = "vector")
and unstabletarget_feature(enable = "vector")
(underfeature(s390x_target_feature)
): done in rustc_target: add known safe s390x target features #127506target_feature = "vector"
(and other vector-related target features if available)At least blocked until ABI support doneUPDATE: ABI support doneAnd may be more things are needed given the precedent of postponed stabilization of vector features in RISC-V.
feature(asm_experimental_arch)
on s390x: done in Stabilize s390x inline assembly #131258#[repr(simd)]
types in input/output of asm (i.e., fully support vector registers) underfeature(asm_experimental_reg)
: pending in Support input/output in vector registers of s390x inline assembly (under asm_experimental_reg feature) #131664Both LLVM and GCC do not document
v
constraint, but actually seem to be supported.Blocked until ABI support done.UPDATE: ABI support donefeature(asm_experimental_reg)
for s390x vector registers (Tracking Issue for asm_experimental_reg #133416)Blocked until ABI support done.UPDATE: ABI support doneis_s390x_feature_detected!("vector")
:@rustbot label +O-SystemZ
The text was updated successfully, but these errors were encountered: