[LLVM][CMake] Build examples for llvm-test-depends #171998

aengelke · 2025-12-12T12:29:41Z

Build examples and example plug-ins by default when running tests. If
examples are unwanted, they can still be disabled completely using
LLVM_INCLUDE_EXAMPLES=OFF. Plugin tests depend on examples and it is
beneficial to test them by default. By default, Examples will still not
be included in the default target or be installed, this remains
controlled by LLVM_BUILD_EXAMPLES (which defaults to OFF).

The additional cost for building examples for tests is 17 compilation
units (12 C++, 5 C), which should be tolerable.

I don't know how broken the examples currently are in the various build
configurations, but if we find breakage, it would be good to fix it.

Created using spr 1.3.5-bogner

nikic

To check my understanding of what this is going to do, does this sound correct?

LLVM_INCLUDE_EXAMPLES=OFF: Examples not built (and targets not generated).
LLVM_INCLUDE_EXAMPLES=ON, LLVM_BUILD_EXAMPLES=OFF: Examples not built by default target, but built by check-llvm. Examples not installed.
LLVM_INCLUDE_EXAMPLES=ON, LLVM_BUILD_EXAMPLES=ON: Examples built by default target and installed by install taret.

aengelke · 2025-12-12T16:10:35Z

Yes.

Edit: worth noting that check-llvm will not cause all examples to be built, only those that are used in tests.

nikic

Okay, I think that sounds reasonable to me. It would be good to update llvm/docs/CMake.rst.

Created using spr 1.3.5-bogner

nikic

LGTM, but please wait for a second opinion.

boomanaiden154

Seems reasonable enough to me, especially given the relative compile time/test cost.

llvm/test/CMakeLists.txt

Build examples and example plug-ins by default when running tests. If examples are unwanted, they can still be disabled completely using LLVM_INCLUDE_EXAMPLES=OFF. Plugin tests depend on examples and it is beneficial to test them by default. By default, Examples will still not be included in the default target or be installed, this remains controlled by LLVM_BUILD_EXAMPLES (which defaults to OFF). The additional cost for building examples for tests is 17 compilation units (12 C++, 5 C), which should be tolerable. I don't know how broken the examples currently are in the various build configurations, but if we find breakage, it would be good to fix it. Pull Request: llvm/llvm-project#171998

review benchmark format fix comma fix comma lwg4482 rebase error [LLVM][CMake] Build examples for llvm-test-depends Build examples and example plug-ins by default when running tests. If examples are unwanted, they can still be disabled completely using LLVM_INCLUDE_EXAMPLES=OFF. Plugin tests depend on examples and it is beneficial to test them by default. By default, Examples will still not be included in the default target or be installed, this remains controlled by LLVM_BUILD_EXAMPLES (which defaults to OFF). The additional cost for building examples for tests is 17 compilation units (12 C++, 5 C), which should be tolerable. I don't know how broken the examples currently are in the various build configurations, but if we find breakage, it would be good to fix it. Pull Request: llvm#171998 Update libcxx/include/__ranges/adjacent_view.h Co-authored-by: Louis Dionne <ldionne.2@gmail.com> ?

[libc++] implement adjacent_view ci ci ci ci Update libcxx/include/__ranges/zip_utils.h Co-authored-by: Louis Dionne <ldionne.2@gmail.com> review review benchmark format fix comma fix comma lwg4482 rebase error [LLVM][CMake] Build examples for llvm-test-depends Build examples and example plug-ins by default when running tests. If examples are unwanted, they can still be disabled completely using LLVM_INCLUDE_EXAMPLES=OFF. Plugin tests depend on examples and it is beneficial to test them by default. By default, Examples will still not be included in the default target or be installed, this remains controlled by LLVM_BUILD_EXAMPLES (which defaults to OFF). The additional cost for building examples for tests is 17 compilation units (12 C++, 5 C), which should be tolerable. I don't know how broken the examples currently are in the various build configurations, but if we find breakage, it would be good to fix it. Pull Request: llvm#171998 Update libcxx/include/__ranges/adjacent_view.h Co-authored-by: Louis Dionne <ldionne.2@gmail.com> ? undo rebase ci

aengelke · 2025-12-13T09:28:48Z

There's less breakage than expected. From the buildbot failures:

llvm-clang-ubuntu-x-aarch64-pauth: Kaleidoscope fails with an assertion failure, orcv2 tests look for the x86-64 target (which is not built). Not sure what's going on here, cc @lhames. Workaround: temporarily require x86-registered-target
clang-armv8-quick, clang-armv7-global-isel: orcv2-cbindings-lazy.test is unsupported on armv8l/s390x; supported targets should be listed explicitly.
sanitizer-aarch64-linux-bootstrap-hwasan: test/Examples/IRTransforms/SimplifyCFG/* Fix: mark as unsupported if plugins are unavailable
sanitizer-x86_64-linux-android, reverse-iteration seem spurious/unrelated to this patch.

I fixed these with #172140, as the changes seemed simple enough.

aengelke · 2025-12-13T10:29:21Z

More failures, fixed with #172147:

clang-s390x-linux, clang-s390x-linux-lnt: test/Examples/OrcV2Examples/lljit-with-remote-debugging.test requires JITLink, fix by adding target=s390{{.*}} to unsupported.
clang-riscv-rva23-evl-vec-2stage, clang-riscv-rva20-2stage, clang-riscv-rva23-zvl512b-2stage, clang-riscv-rva23-zvl1024b-2stage: Kaleidoscope tests fail with errors like JIT session error: Symbols not found: [ __adddf3 ] (or __gedf2 etc.), fix by marking riscv as unsupported.
ppc64le-mlir-rhel-clang: timeout, unrelated
ppc64le-flang-rhel-clang: seems unrelated, Flang test failure
sanitizer-x86_64-linux-fast seems spurious, CodeGen/X86/basic-block-sections-clusters-bb-hash.ll failed.
clang-arm64-windows-msvc: libomp test failure seems unrelated.

aengelke · 2025-12-13T14:44:08Z

aix-ppc64 is also broken, there's neither JIT nor plugin support. Excluded AIX in 249acb6.

aengelke · 2025-12-14T11:06:23Z

Filed #172203 for AIX plugins.

clang-solaris11-sparcv9 has no JIT support, disabled ORC tests there with 66d92d4.

Build examples and example plug-ins by default when running tests. If examples are unwanted, they can still be disabled completely using LLVM_INCLUDE_EXAMPLES=OFF. Plugin tests depend on examples and it is beneficial to test them by default. By default, Examples will still not be included in the default target or be installed, this remains controlled by LLVM_BUILD_EXAMPLES (which defaults to OFF). The additional cost for building examples for tests is 17 compilation units (12 C++, 5 C), which should be tolerable. I don't know how broken the examples currently are in the various build configurations, but if we find breakage, it would be good to fix it. Pull Request: llvm#171998

mgorny · 2025-12-15T05:50:32Z

FAIL: LLVM :: Examples/OrcV2Examples/lljit-with-remote-debugging.test (32940 of 62631)
******************** TEST 'LLVM :: Examples/OrcV2Examples/lljit-with-remote-debugging.test' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 9
/var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/bin/LLJITWithRemoteDebugging /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.ll 2>&1 | /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/bin/FileCheck --check-prefix=CHECK0 /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
# executed command: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/bin/LLJITWithRemoteDebugging /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.ll
# note: command had no output on stdout or stderr
# error: command failed with exit status: 1
# executed command: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/bin/FileCheck --check-prefix=CHECK0 /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
# .---command stderr------------
# | /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test:13:11: error: CHECK0: expected string not found in input
# | # CHECK0: Running: main()
# |           ^
# | <stdin>:5:130: note: scanning from here
# | Parsing input IR code from: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.ll
# |                                                                                                                                  ^
# | <stdin>:6:5: note: possible intended match here
# | Initializing LLJIT for remote executor
# |     ^
# | 
# | Input file: <stdin>
# | Check file: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/lljit-with-remote-debugging.test
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |             1: __jit_debug_descriptor.last_entry = 0x0000000000000000 
# |             2: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/bin/LLJITWithRemoteDebugging: Cannot enable LLJIT debugger support: Debugger support requires JITLink 
# |             3: Found out-of-process executor: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/bin/llvm-jitlink-executor 
# |             4: Launched executor in subprocess: 44610 
# |             5: Parsing input IR code from: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm/test/Examples/OrcV2Examples/Inputs/argc_sub1.ll 
# | check:13'0                                                                                                                                      X error: no match found
# |             6: Initializing LLJIT for remote executor 
# | check:13'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | check:13'1         ?                                   possible intended match
# |             7: __jit_debug_descriptor.last_entry = 0x0000000000000000 
# | check:13'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

--

********************

aengelke · 2025-12-15T07:07:45Z

@mgorny Thanks for reporting. What architecture is this? Is this x32? Or is there anything else unusual about the build config? (Full target triple would be good to mark the test as unsupported.)

weliveindetail · 2025-12-15T08:26:29Z

FYI: I never tested the ELFDebugObject JITLink plugin for 32bit targets. You can mark it unsupported.

mgorny · 2025-12-15T09:55:48Z

32-bit multilib on amd64, i.e. i686-pc-linux-gnu.

mgorny · 2025-12-15T11:35:20Z

FYI: I never tested the ELFDebugObject JITLink plugin for 32bit targets. You can mark it unsupported.

Sorry, do you mean the test or some larger piece of code?

weliveindetail · 2025-12-15T12:18:35Z

Sorry, do you mean the test or some larger piece of code?

@mgorny This is unrelated from the PR. Let me ping you in a review that is in flight.

mgorny · 2025-12-15T15:52:06Z

BTW this is weird:

# |             2: /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/bin/LLJITWithRemoteDebugging: Cannot enable LLJIT debugger support: Debugger support requires JITLink

but it definitely does build /var/tmp/portage/llvm-core/llvm-22.0.0.9999/work/llvm_build-abi_x86_32.x86/lib/libLLVMJITLink.a.

Build examples and example plug-ins by default when running tests. If examples are unwanted, they can still be disabled completely using LLVM_INCLUDE_EXAMPLES=OFF. Plugin tests depend on examples and it is beneficial to test them by default. By default, Examples will still not be included in the default target or be installed, this remains controlled by LLVM_BUILD_EXAMPLES (which defaults to OFF). The additional cost for building examples for tests is 17 compilation units (12 C++, 5 C), which should be tolerable. I don't know how broken the examples currently are in the various build configurations, but if we find breakage, it would be good to fix it. Pull Request: llvm#171998

[𝘀𝗽𝗿] initial version

26ec186

Created using spr 1.3.5-bogner

aengelke changed the title ~~[WIP] [LLVM][CMake] Always build examples for tests~~ [LLVM][CMake] Build examples for llvm-test-depends Dec 12, 2025

aengelke marked this pull request as ready for review December 12, 2025 13:02

aengelke requested review from boomanaiden154, nikic and vgvassilev December 12, 2025 13:02

aengelke added the cmake Build system in general and CMake in particular label Dec 12, 2025

nikic reviewed Dec 12, 2025

View reviewed changes

update docs for LLVM_BUILD_EXAMPLES

7ef4f1a

Created using spr 1.3.5-bogner

nikic approved these changes Dec 12, 2025

View reviewed changes

boomanaiden154 approved these changes Dec 12, 2025

View reviewed changes

vgvassilev reviewed Dec 12, 2025

View reviewed changes

llvm/test/CMakeLists.txt Show resolved Hide resolved

aengelke merged commit c81d449 into main Dec 13, 2025
11 checks passed

aengelke deleted the users/aengelke/spr/wip-llvmcmake-always-build-examples-for-tests branch December 13, 2025 08:05

kikairoya mentioned this pull request Dec 13, 2025

[LLVM][Examples][Cygwin] Exclude examples that are not built from test dependencies #172145

Merged

aengelke mentioned this pull request Dec 16, 2025

[ORC] Broken example tests in unusual configurations #172482

Open

[LLVM][CMake] Build examples for llvm-test-depends #171998

[LLVM][CMake] Build examples for llvm-test-depends #171998

Uh oh!

Conversation

aengelke commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nikic left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aengelke commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nikic left a comment

Choose a reason for hiding this comment

Uh oh!

nikic left a comment

Choose a reason for hiding this comment

Uh oh!

boomanaiden154 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aengelke commented Dec 13, 2025

Uh oh!

aengelke commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aengelke commented Dec 13, 2025

Uh oh!

aengelke commented Dec 14, 2025

Uh oh!

mgorny commented Dec 15, 2025

Uh oh!

aengelke commented Dec 15, 2025

Uh oh!

weliveindetail commented Dec 15, 2025

Uh oh!

mgorny commented Dec 15, 2025

Uh oh!

mgorny commented Dec 15, 2025

Uh oh!

weliveindetail commented Dec 15, 2025

Uh oh!

mgorny commented Dec 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

aengelke commented Dec 12, 2025 •

edited

Loading

nikic left a comment •

edited

Loading

aengelke commented Dec 12, 2025 •

edited

Loading

aengelke commented Dec 13, 2025 •

edited

Loading