Add OpenMP builder for s390x

[uweigand]

No description provided.

[tuliom]

Heads up: I'm reproducing issue #40769 when building libomp using clang:

[40/881] : && /usr/bin/clang -fPIC -O2 -flto=thin -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Werror=implicit-function-declaration -Werror=implicit-int -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS --config /usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m64 -march=z13 -mtune=z14 -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Wcast-qual -Wformat-pedantic -Wimplicit-fallthrough -Wsign-compare -Wno-enum-constexpr-conversion -Wno-extra -Wno-pedantic -DNDEBUG  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now   -flto=thin -Wl,--build-id=sha1  -Wl,--as-needed -Wl,--version-script=/builddir/build/BUILD/openmp-18.0.0.src/runtime/src/exports_so.txt -static-libgcc -Wl,-z,noexecstack -shared -Wl,-soname,libomp.so -o runtime/src/libomp.so runtime/src/CMakeFiles/omp.dir/kmp_alloc.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_atomic.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_debug.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_itt.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_environment.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_error.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_global.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_i18n.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_io.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_runtime.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_settings.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_str.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_tasking.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_threadprivate.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_utility.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_barrier.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_wait_release.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_affinity.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_dispatch.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_lock.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_sched.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_collapse.cpp.o runtime/src/CMakeFiles/omp.dir/z_Linux_util.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_gsupport.cpp.o runtime/src/CMakeFiles/omp.dir/thirdparty/ittnotify/ittnotify_static.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_taskdeps.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_cancel.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_ftn_cdecl.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_ftn_extra.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_version.cpp.o runtime/src/CMakeFiles/omp.dir/ompt-general.cpp.o runtime/src/CMakeFiles/omp.dir/ompd-specific.cpp.o runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o  -lm  -ldl && cd /builddir/build/BUILD/openmp-18.0.0.src/redhat-linux-build/runtime/src && /usr/bin/cmake -E create_symlink libomp.so libgomp.so && /usr/bin/cmake -E create_symlink libomp.so libiomp5.so
FAILED: runtime/src/libomp.so 
: && /usr/bin/clang -fPIC -O2 -flto=thin -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Werror=implicit-function-declaration -Werror=implicit-int -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS --config /usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m64 -march=z13 -mtune=z14 -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Wcast-qual -Wformat-pedantic -Wimplicit-fallthrough -Wsign-compare -Wno-enum-constexpr-conversion -Wno-extra -Wno-pedantic -DNDEBUG  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now   -flto=thin -Wl,--build-id=sha1  -Wl,--as-needed -Wl,--version-script=/builddir/build/BUILD/openmp-18.0.0.src/runtime/src/exports_so.txt -static-libgcc -Wl,-z,noexecstack -shared -Wl,-soname,libomp.so -o runtime/src/libomp.so runtime/src/CMakeFiles/omp.dir/kmp_alloc.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_atomic.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_debug.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_itt.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_environment.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_error.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_global.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_i18n.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_io.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_runtime.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_settings.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_str.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_tasking.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_threadprivate.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_utility.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_barrier.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_wait_release.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_affinity.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_dispatch.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_lock.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_sched.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_collapse.cpp.o runtime/src/CMakeFiles/omp.dir/z_Linux_util.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_gsupport.cpp.o runtime/src/CMakeFiles/omp.dir/thirdparty/ittnotify/ittnotify_static.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_taskdeps.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_cancel.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_ftn_cdecl.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_ftn_extra.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_version.cpp.o runtime/src/CMakeFiles/omp.dir/ompt-general.cpp.o runtime/src/CMakeFiles/omp.dir/ompd-specific.cpp.o runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o  -lm  -ldl && cd /builddir/build/BUILD/openmp-18.0.0.src/redhat-linux-build/runtime/src && /usr/bin/cmake -E create_symlink libomp.so libgomp.so && /usr/bin/cmake -E create_symlink libomp.so libiomp5.so
'+backchain' is not a recognized feature for this target (ignoring feature)
'+backchain' is not a recognized feature for this target (ignoring feature)
LLVM ERROR: Unsupported stack frame traversal count
clang: error: unable to execute command: Aborted (core dumped)
clang: error: linker command failed due to signal (use -v to see invocation)

Am I missing anything?

[iii-i]

This should be fixed by llvm/llvm-project#69405. What version of clang do you have?

[tuliom]

This should be fixed by llvm/llvm-project#69405. What version of clang do you have?

@iii-i Interesting... I checked out git commit cf7d4f543c73c2707e0c53bae1e7b8419e12b871, which has 8e810dc7d93bebe5e2d3980d4db084f58248b37f.

For the record, more details is available at https://copr.fedorainfracloud.org/coprs/g/fedora-llvm-team/llvm-snapshots-incubator-20231106/build/6601238/

Build log is available at: https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-incubator-20231106/fedora-rawhide-s390x/06601238-libomp/builder-live.log.gz

[iii-i]

Thanks for the logs! Apparently this has to do with LTO, since I can now reproduce this using a trivial example:

$ cat 1.c
__attribute__((target("backchain")))
void *f(void) { return __builtin_return_address(42); }

$ clang-18 -flto=thin -c 1.c
$ clang-18 -flto=thin 1.o -shared -o lib1.so
'+backchain' is not a recognized feature for this target (ignoring feature)
'+backchain' is not a recognized feature for this target (ignoring feature)
LLVM ERROR: Unsupported stack frame traversal count
clang-18: error: unable to execute command: Aborted (core dumped)
clang-18: error: linker command failed due to signal (use -v to see invocation)

[uweigand]

I don't think this is necessarily related to LTO, I'm seeing the failure also without LTO. The problem seems to be that the -mbackchain command line option in clang causes each function to be annotated with a backchain attribute:

attributes #0 = { noinline nounwind optnone "backchain" "frame-pointer"="all"
"no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="z10" }

while the target("backchain") attribute causes the function to be annotated with a backchain target feature:

attributes #0 = { noinline nounwind optnone "frame-pointer"="all" "no-trapping-math"="true"
"stack-protector-buffer-size"="8" "target-cpu"="z10" "target-features"="+backchain" }

but the latter isn't actually supported by the SystemZ back-end.

Not sure exactly what the best fix here is, but this seems to require at least some back-end changes.

[iii-i]

Not sure exactly what the best fix here is, but this seems to require at least some back-end changes.

Is there a downside to adding a SystemZFeature for that, just like for soft-float?

[uweigand]

Not sure exactly what the best fix here is, but this seems to require at least some back-end changes.

Is there a downside to adding a SystemZFeature for that, just like for soft-float?

I agree, that seems to be the best solution.

[iii-i]

@tuliom your issue should be now fixed by llvm/llvm-project#71668.

[tuliom]

@iii-i Thank you! But the build is still failing.

[46/881] : && /usr/bin/clang -fPIC -O2 -flto -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS --config /usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m64 -march=zEC12 -mtune=z13 -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Wcast-qual -Wformat-pedantic -Wimplicit-fallthrough -Wsign-compare -Wno-enum-constexpr-conversion -Wno-extra -Wno-pedantic -DNDEBUG  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now   -flto -fno-openmp-implicit-rpath -Wl,--build-id=sha1  -Wl,--as-needed -Wl,--version-script=/builddir/build/BUILD/openmp-18.0.0.src/runtime/src/exports_so.txt -static-libgcc -Wl,-z,noexecstack -shared -Wl,-soname,libomp.so -o runtime/src/libomp.so runtime/src/CMakeFiles/omp.dir/kmp_alloc.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_atomic.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_debug.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_itt.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_environment.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_error.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_global.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_i18n.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_io.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_runtime.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_settings.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_str.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_tasking.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_threadprivate.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_utility.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_barrier.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_wait_release.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_affinity.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_dispatch.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_lock.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_sched.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_collapse.cpp.o runtime/src/CMakeFiles/omp.dir/z_Linux_util.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_gsupport.cpp.o runtime/src/CMakeFiles/omp.dir/thirdparty/ittnotify/ittnotify_static.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_taskdeps.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_cancel.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_ftn_cdecl.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_ftn_extra.cpp.o runtime/src/CMakeFiles/omp.dir/kmp_version.cpp.o runtime/src/CMakeFiles/omp.dir/ompt-general.cpp.o runtime/src/CMakeFiles/omp.dir/ompd-specific.cpp.o runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o  -lm  -ldl && cd /builddir/build/BUILD/openmp-18.0.0.src/redhat-linux-build/runtime/src && /usr/bin/cmake -E create_symlink libomp.so libgomp.so && /usr/bin/cmake -E create_symlink libomp.so libiomp5.so
FAILED: runtime/src/libomp.so
...
LLVM ERROR: Unsupported stack frame traversal count
clang: error: unable to execute command: Aborted (core dumped)
clang: error: linker command failed due to signal (use -v to see invocation)

The complete log is available at: https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-incubator-20231109/fedora-37-s390x/06614614-libomp/builder-live.log.gz

I'm wondering if another openmp function is missing the backchain attribute...

Please let me know if you'd like to move this discussion to its own issue.

[iii-i]

Thanks for the analysis, I think you are correct that we need to compile another function with backchain. I posted llvm/llvm-project#71834.

[tuliom]

@iii-i It worked! Thank you!

[gkistanova]

LGTM.