Skip to content

Add support for --split-compile and --jobserver in build.sh #626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aliceb-nv wants to merge 10 commits into
base: main
Choose a base branch
from
+31 −2
Open

Add support for --split-compile and --jobserver in build.sh #626

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
1f4a0e0
build script changes
aliceb-nv Nov 25, 2025
970ccc4
fix CI build
aliceb-nv Nov 25, 2025
a111bf6
address rewview comment
aliceb-nv Nov 25, 2025
a7ed7a7
review comment
aliceb-nv Nov 25, 2025
e0d89ac
Merge branch 'main' into build-script-splitcompile
aliceb-nv Dec 17, 2025
bce1494
address review comments
aliceb-nv Dec 17, 2025
da9e76f
Merge branch 'main' into build-script-splitcompile
aliceb-nv Jan 8, 2026
3159ab1
update copyrights
aliceb-nv Jan 9, 2026
14d9f4e
version check for jobserver option
aliceb-nv Jan 13, 2026
0aeb417
Merge branch 'main' into build-script-splitcompile
aliceb-nv Jan 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ CACHE_ARGS=()
PYTHON_ARGS_FOR_INSTALL=("-m" "pip" "install" "--no-build-isolation" "--no-deps")
LOGGING_ACTIVE_LEVEL="INFO"
FETCH_RAPIDS=ON
PARALLEL_LEVEL=${PARALLEL_LEVEL:=$(nproc)}

# Set defaults for vars that may not have been defined externally
# FIXME: if PREFIX is not set, check CONDA_PREFIX, but there is no fallback
Expand Down Expand Up @@ -379,14 +380,17 @@ if buildAll || hasArg libcuopt; then
-DSKIP_ROUTING_BUILD=${SKIP_ROUTING_BUILD} \
-DWRITE_FATBIN=${WRITE_FATBIN} \
-DHOST_LINEINFO=${HOST_LINEINFO} \
-DPARALLEL_LEVEL="${PARALLEL_LEVEL}" \
-DINSTALL_TARGET="${INSTALL_TARGET}" \
"${CACHE_ARGS[@]}" \
"${EXTRA_CMAKE_ARGS[@]}" \
"${REPODIR}"/cpp
JFLAG="${PARALLEL_LEVEL:+-j${PARALLEL_LEVEL}}"
if hasArg -n; then
cmake --build "${LIBCUOPT_BUILD_DIR}" ${VERBOSE_FLAG}
# Manual make invocation to start its jobserver
make ${JFLAG} -C "${REPODIR}/cpp" LIBCUOPT_BUILD_DIR="${LIBCUOPT_BUILD_DIR}" VERBOSE_FLAG="${VERBOSE_FLAG}" PARALLEL_LEVEL="${PARALLEL_LEVEL}" ninja-build
Copy link
Member

@jameslamb jameslamb Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of needing to directly invoke make, you could explicitly select that in the cmake configure step above if it's important, like this:

cmake -G 'Unix Makefiles' \
    -DDEFINE_BENCHMARK="${DEFINE_BENCHMARK}" \
    # ... etc., etc.

That'd allow you to continue to just use cmake --build instead of invoking the preferred build tool yourself.

Would you consider that?

Copy link
Contributor Author

@aliceb-nv aliceb-nv Dec 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use Ninja as the default generator; this step is only necessary in order to create a make jobserver that ninja can hook into (as a jobserver client). This workaround is (to the best of my knowledge) necessary to get nvcc's jobserver support to play nice with Ninja, which is a jobsever client itself

else
cmake --build "${LIBCUOPT_BUILD_DIR}" --target ${INSTALL_TARGET} ${VERBOSE_FLAG} -j"${PARALLEL_LEVEL}"
cmake --build "${LIBCUOPT_BUILD_DIR}" --target ${INSTALL_TARGET} ${VERBOSE_FLAG} ${JFLAG}
fi
fi

Expand Down
1 change: 1 addition & 0 deletions conda/recipes/libcuopt/recipe.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ cache:
- ${{ stdlib("c") }}
- cuda-version =${{ cuda_version }}
- cmake ${{ cmake_version }}
- make
- ninja
- tbb-devel
- zlib
Expand Down
9 changes: 9 additions & 0 deletions cpp/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,15 @@ if(CMAKE_CUDA_COMPILER_VERSION VERSION_GREATER_EQUAL 12.9 AND CMAKE_CUDA_COMPILE
endif()
list(APPEND CUOPT_CUDA_FLAGS -fopenmp)

# Add jobserver flags for parallel compilation if PARALLEL_LEVEL is set
if(PARALLEL_LEVEL AND NOT "${PARALLEL_LEVEL}" STREQUAL "")
message(STATUS "Enabling nvcc parallel compilation support")
list(APPEND CUOPT_CUDA_FLAGS --threads=0 --split-compile=0)
if(CMAKE_CUDA_COMPILER_VERSION VERSION_GREATER_EQUAL 13.0)
message(STATUS "Enabling nvcc jobserver support (NVCC >= 13.0)")
list(APPEND CUOPT_CUDA_FLAGS --jobserver)
endif()
endif()

if(NOT DISABLE_OPENMP)
find_package(OpenMP)
Expand Down
15 changes: 15 additions & 0 deletions cpp/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Makefile

LIBCUOPT_BUILD_DIR ?= cpp/build
Copy link

@coderabbitai coderabbitai bot Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Default LIBCUOPT_BUILD_DIR is likely wrong when invoked from cpp/.

If users run make -C cpp / cd cpp && make, LIBCUOPT_BUILD_DIR=cpp/build resolves to cpp/cpp/build and breaks. Consider anchoring the build dir to the Makefile location.

Proposed fix 
-LIBCUOPT_BUILD_DIR ?= cpp/build
+MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+LIBCUOPT_BUILD_DIR ?= $(MAKEFILE_DIR)/build
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
LIBCUOPT_BUILD_DIR ?= cpp/build
MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
LIBCUOPT_BUILD_DIR ?= $(MAKEFILE_DIR)/build
🤖 Prompt for AI Agents 
In @cpp/Makefile at line 6, LIBCUOPT_BUILD_DIR currently defaults to a relative
path that becomes incorrect when running make from the repo root with make -C
cpp; update the Makefile to compute an absolute build dir anchored to the
Makefile's location instead of a literal "cpp/build". Replace the simple
assignment for LIBCUOPT_BUILD_DIR with one that uses $(lastword
$(MAKEFILE_LIST)) combined with $(abspath) and $(dir) (e.g., set
LIBCUOPT_BUILD_DIR ?= $(dir $(abspath $(lastword $(MAKEFILE_LIST))))build) so
the build directory resolves correctly regardless of where make is invoked from.

VERBOSE_FLAG ?=
PARALLEL_LEVEL ?=

.PHONY: all ninja-build

all: ninja-build

ninja-build:
cmake --build $(LIBCUOPT_BUILD_DIR) $(VERBOSE_FLAG) $(if $(PARALLEL_LEVEL),-j$(PARALLEL_LEVEL),)
Loading