Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use nvcomp wheel instead of bundling nvcomp #16946

Open
wants to merge 13 commits into
base: branch-24.12
Choose a base branch
from
+47 −13
Open

Use nvcomp wheel instead of bundling nvcomp #16946

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
73797ab
Use nvcomp wheel instead of bundling nvcomp
KyleFromNVIDIA Sep 27, 2024
9651827
Don't install downloaded nvcomp, depend on wheel
KyleFromNVIDIA Sep 27, 2024
dcf0b50
Style
KyleFromNVIDIA Sep 30, 2024
661d834
Exclude nvcomp from auditwheel
KyleFromNVIDIA Sep 30, 2024
723cdd0
Set nvcomp rpath
KyleFromNVIDIA Sep 30, 2024
418344c
Style
KyleFromNVIDIA Sep 30, 2024
b334dc5
Fix rpath
KyleFromNVIDIA Sep 30, 2024
08f1369
Make rpath usage configurable
KyleFromNVIDIA Sep 30, 2024
1d875c3
Review feedback
KyleFromNVIDIA Oct 1, 2024
69ee11e
Update variable name
KyleFromNVIDIA Oct 1, 2024
ba7e736
Merge branch 'branch-24.12' into nvcomp-wheel
KyleFromNVIDIA Oct 1, 2024
b72b059
Update description
KyleFromNVIDIA Oct 1, 2024
319c95e
Update variable name
KyleFromNVIDIA Oct 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/build_wheel_libcudf.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ package_dir="python/libcudf"
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

mkdir -p ${package_dir}/final_dist
python -m auditwheel repair -w ${package_dir}/final_dist ${package_dir}/dist/*
python -m auditwheel repair -w ${package_dir}/final_dist --exclude libnvcomp.so.4 ${package_dir}/dist/*
KyleFromNVIDIA marked this conversation as resolved.
Show resolved Hide resolved

RAPIDS_PY_WHEEL_NAME="libcudf_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 cpp ${package_dir}/final_dist
8 changes: 2 additions & 6 deletions cpp/cmake/thirdparty/get_nvcomp.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# =============================================================================
# Copyright (c) 2021-2022, NVIDIA CORPORATION.
# Copyright (c) 2021-2024, NVIDIA CORPORATION.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
# in compliance with the License. You may obtain a copy of the License at
Expand All @@ -16,11 +16,7 @@
function(find_and_configure_nvcomp)

include(${rapids-cmake-dir}/cpm/nvcomp.cmake)
rapids_cpm_nvcomp(
BUILD_EXPORT_SET cudf-exports
INSTALL_EXPORT_SET cudf-exports
USE_PROPRIETARY_BINARY ${CUDF_USE_PROPRIETARY_NVCOMP}
)
rapids_cpm_nvcomp(USE_PROPRIETARY_BINARY ${CUDF_USE_PROPRIETARY_NVCOMP})
KyleFromNVIDIA marked this conversation as resolved.
Show resolved Hide resolved

# Per-thread default stream
if(TARGET nvcomp AND CUDF_USE_PER_THREAD_DEFAULT_STREAM)
Expand Down
22 changes: 22 additions & 0 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,13 @@ files:
- build_cpp
- depends_on_libkvikio
- depends_on_librmm
py_run_libcudf:
output: pyproject
pyproject_dir: python/libcudf
extras:
table: project
includes:
- depends_on_nvcomp
py_build_pylibcudf:
output: pyproject
pyproject_dir: python/pylibcudf
Expand Down Expand Up @@ -370,6 +377,21 @@ dependencies:
# Align nvcomp version with rapids-cmake
- nvcomp==4.0.1
KyleFromNVIDIA marked this conversation as resolved.
Show resolved Hide resolved
- spdlog>=1.14.1,<1.15
depends_on_nvcomp:
specific:
KyleFromNVIDIA marked this conversation as resolved.
Show resolved Hide resolved
- output_types: [requirements, pyproject]
matrices:
- matrix:
cuda: "12.*"
packages:
- nvidia-nvcomp-cu12==4.0.1
- matrix:
cuda: "11.*"
packages:
- nvidia-nvcomp-cu11==4.0.1
- matrix:
packages:
- nvidia-nvcomp==4.0.1
rapids_build_skbuild:
common:
- output_types: [conda, requirements, pyproject]
Expand Down
3 changes: 3 additions & 0 deletions python/libcudf/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ classifiers = [
"Programming Language :: C++",
"Environment :: GPU :: NVIDIA CUDA",
]
dependencies = [
"nvidia-nvcomp==4.0.1",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

[project.urls]
Homepage = "https://github.com/rapidsai/cudf"
Expand Down
Loading