Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] raft v23.08 #1692

Merged
merged 76 commits into from
Aug 9, 2023
Merged

[RELEASE] raft v23.08 #1692

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
254b540
DOC
raydouglass May 19, 2023
99655ea
Merge pull request #1545 from rapidsai/branch-23.06
GPUtester May 23, 2023
a9ea1e5
Merge pull request #1561 from rapidsai/branch-23.06
GPUtester May 31, 2023
9d0f59c
Fix update version and pinnings for 23.08. (#1556)
bdice Jun 1, 2023
e804156
Remove documentation build scripts for Jenkins (#1570)
ajschmidt8 Jun 1, 2023
cfe27ec
Update CAGRA knn_graph_sort to use Raft::bitonic_sort (#1550)
enp1s0 Jun 2, 2023
6bc237f
Add support to __half and nv_bfloat16 to most math functions (#1554)
Kh4ster Jun 2, 2023
9bf7b4b
Add sample filtering for ivf_flat. Filtering code refactoring and cle…
alexanderguzhva Jun 5, 2023
651402d
Merge pull request #1576 from rapidsai/branch-23.06
GPUtester Jun 6, 2023
64f15b6
Unpin `dask` and `distributed` for development and fix `merge_labels`…
galipremsagar Jun 6, 2023
c1a7b7c
Merge pull request #1581 from rapidsai/branch-23.06
GPUtester Jun 7, 2023
4657954
Add identity matrix function (#1548)
lowener Jun 7, 2023
a995c5b
use rapids-upload-docs script (#1578)
AyodeAwe Jun 7, 2023
567dfd7
preprocessor syntax fix (#1582)
AyodeAwe Jun 8, 2023
6ec78e9
CAGRA pad dataset for 128bit vectorized load (#1505)
tfeher Jun 9, 2023
671bdad
Unpin scikit-build upper bound (#1547)
vyasr Jun 9, 2023
681008e
Fix raft::bitonic_sort small usage example (#1580)
enp1s0 Jun 10, 2023
ad0c1c1
This PR adds support to __half and nb_bfloat16 to myAtomicReduce (#1585)
Kh4ster Jun 10, 2023
5dd7017
[IMP] move core CUDA RT macros to cuda_rt_essentials.hpp (#1584)
MatthiasKohl Jun 10, 2023
2da93a1
Merge pull request #1590 from rapidsai/branch-23.06
GPUtester Jun 12, 2023
c69231e
Use exceptions instead of `exit(-1)` (#1594)
benfred Jun 21, 2023
bca4d93
pass correct stream to cutlass kernel launch of L2/cosine pairwise di…
mdoijade Jun 22, 2023
afa5963
Fix the documentation about changing the logging level (#1596)
enp1s0 Jun 22, 2023
cb77979
Fix launchconfig y-gridsize too large in epilogue kernel (#1586)
mfoerste4 Jun 22, 2023
6f0abae
[REVIEW] Add scheduler_file argument to support MNMG setup (#1593)
VibhuJawa Jun 22, 2023
28b61c4
Rename the CAGRA prune function to optimize (#1588)
enp1s0 Jun 22, 2023
0db8f15
Add support for row-major slice (#1591)
lowener Jun 23, 2023
185e933
Add RAFT ANN benchmark for CAGRA (#1552)
enp1s0 Jun 23, 2023
1034a41
Fix for function exposing KNN merge (#1418)
viclafargue Jun 26, 2023
503fc43
Merge pull request #1624 from rapidsai/branch-23.06
GPUtester Jun 29, 2023
58631df
Set pool memory resource for raft IVF ANN benchmarks (#1625)
tfeher Jun 30, 2023
7358763
Fixing ann bench cmake (and docs) (#1630)
cjnolet Jul 1, 2023
e9d86f1
Sparse Pairwise Distances API Updates (#1502)
divyegala Jul 3, 2023
104f3ba
CAGRA max_queries auto configuration (#1613)
enp1s0 Jul 3, 2023
fda9cd1
[WIP] Test CI issues (#1626)
VibhuJawa Jul 3, 2023
744881e
Adding sort option to matrix::select_k api (#1615)
cjnolet Jul 3, 2023
cc1d1ed
Fix cagra multi CTA bug (#1628)
enp1s0 Jul 4, 2023
32b3dba
raft: Build CUDA 12 packages (#1388)
vyasr Jul 5, 2023
c9052f8
Drop blank line in `raft-dask/meta.yaml` (#1637)
jakirkham Jul 5, 2023
82fbac3
Enable conservative memory allocations for RAFT IVF-Flat benchmarks. …
tfeher Jul 6, 2023
e49cc0c
Fix uploading of RAFT nightly packages (#1638)
dantegd Jul 6, 2023
f34cdbd
Further removal of uses-setup-env-vars (#1639)
dantegd Jul 6, 2023
88bf324
Stop using setup.py in build.sh (#1645)
vyasr Jul 13, 2023
cb7d01a
IVF-PQ: Add a (faster) direct conversion fp8->half (#1644)
achirkin Jul 14, 2023
8173112
Revert CUDA 12.0 CI workflows to branch-23.08. (#1652)
bdice Jul 18, 2023
1824e42
Add CAGRA gbench (#1496)
tfeher Jul 18, 2023
2ee9a57
Cagra template instantiations (#1650)
tfeher Jul 19, 2023
6b62ab4
Hotfix: wrong constant in IVF-PQ fp_8bit2half (#1654)
achirkin Jul 19, 2023
f0e75f2
ANN: Optimize host-side refine (#1651)
achirkin Jul 19, 2023
0abedc6
Add rmm to pylibraft run dependencies, since it is used by Cython. (#…
bdice Jul 19, 2023
db4797f
Cagra index construction without copying device mdarrays (#1494)
tfeher Jul 20, 2023
06b3aa0
ANN-benchmarks: avoid using the dataset during search when possible (…
achirkin Jul 20, 2023
dad78de
Fix CMake CUDA support for pylibraft when raft is found. (#1659)
bdice Jul 20, 2023
61f0d94
Cagra ANN benchmark improvements (#1658)
tfeher Jul 21, 2023
11b0dd1
Migrate wheel workflow scripts locally (#1546)
divyegala Jul 21, 2023
9884f69
Use rapids-cmake new parallel testing feature (#1623)
robertmaynard Jul 21, 2023
422e937
Modify comm_split to avoid ucp (#1649)
ChuckHastings Jul 24, 2023
202385b
Renaming namespaces to promote CAGRA from experimental (#1666)
cjnolet Jul 24, 2023
a66c3a3
[FEA] Masked NN for connect_components (#1445)
tarang-jain Jul 25, 2023
f99a418
Simplify `bench/ann` scripts to Python based module (#1642)
divyegala Jul 26, 2023
f25907b
Fix sparse KNN for large batches (#1640)
viclafargue Jul 26, 2023
617d33a
Limiting workspace memory resource (#1356)
achirkin Jul 26, 2023
acd40f5
ANN benchmarks python wrapper for splitting billion-scale dataset gro…
divyegala Jul 26, 2023
a20f497
Separate CAGRA index type from internal idx type (#1664)
tfeher Jul 27, 2023
4574e9a
IVF-PQ: Fix illegal memory access with large max_samples (#1685)
achirkin Jul 27, 2023
d8b9187
Rename CAGRA parameter num_parents to search_width (#1676)
tfeher Jul 27, 2023
bcb7404
Add algo parameter for CAGRA ANN bench (#1687)
tfeher Jul 27, 2023
47a4385
Fix missing parameter for select_k (#1682)
ucassjy Jul 28, 2023
8d9d682
Remove push condition on docs-build (#1693)
raydouglass Jul 29, 2023
c87af2d
CAGRA Python wrappers (#1665)
dantegd Jul 31, 2023
c957037
IVF-PQ tutorial notebook (#1544)
achirkin Jul 31, 2023
25c20f5
Add notebook for Vector Search - Question Retrieval (#1662)
lowener Aug 1, 2023
f42fa40
[FEA] Codepacking for IVF-flat (#1632)
tarang-jain Aug 1, 2023
31f89ca
Critical doc fixes and updates for 23.08 (#1705)
cjnolet Aug 1, 2023
7a6fe79
Pin dask and distributed for 23.08 release (#1711)
galipremsagar Aug 2, 2023
4862546
Update Changelog [skip ci]
raydouglass Aug 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
24 changes: 10 additions & 14 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ concurrency:
jobs:
cpp-build:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.08
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -37,7 +37,7 @@ jobs:
python-build:
needs: [cpp-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@branch-23.08
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -46,7 +46,7 @@ jobs:
upload-conda:
needs: [cpp-build, python-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@branch-23.08
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -57,7 +57,7 @@ jobs:
if: github.ref_type == 'branch'
needs: python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@branch-23.08
with:
arch: "amd64"
branch: ${{ inputs.branch }}
Expand All @@ -69,19 +69,17 @@ jobs:
sha: ${{ inputs.sha }}
wheel-build-pylibraft:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-build.yaml@branch-23.08
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
sha: ${{ inputs.sha }}
date: ${{ inputs.date }}
package-name: pylibraft
package-dir: python/pylibraft
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
script: ci/build_wheel_pylibraft.sh
wheel-publish-pylibraft:
needs: wheel-build-pylibraft
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-publish.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-publish.yaml@branch-23.08
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -91,19 +89,17 @@ jobs:
wheel-build-raft-dask:
needs: wheel-publish-pylibraft
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-build.yaml@branch-23.08
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
sha: ${{ inputs.sha }}
date: ${{ inputs.date }}
package-name: raft_dask
package-dir: python/raft-dask
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
script: ci/build_wheel_raft_dask.sh
wheel-publish-raft-dask:
needs: wheel-build-raft-dask
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-publish.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-publish.yaml@branch-23.08
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand Down
42 changes: 15 additions & 27 deletions .github/workflows/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,41 +23,41 @@ jobs:
- wheel-build-raft-dask
- wheel-tests-raft-dask
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@branch-23.08
checks:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@branch-23.08
with:
enable_check_generated_files: false
conda-cpp-build:
needs: checks
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.08
with:
build_type: pull-request
node_type: cpu16
conda-cpp-tests:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@branch-23.08
with:
build_type: pull-request
conda-python-build:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@branch-23.08
with:
build_type: pull-request
conda-python-tests:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.08
with:
build_type: pull-request
docs-build:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@branch-23.08
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -67,40 +67,28 @@ jobs:
wheel-build-pylibraft:
needs: checks
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-build.yaml@branch-23.08
with:
build_type: pull-request
package-name: pylibraft
package-dir: python/pylibraft
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
script: ci/build_wheel_pylibraft.sh
wheel-tests-pylibraft:
needs: wheel-build-pylibraft
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-test.yaml@branch-23.08
with:
build_type: pull-request
package-name: pylibraft
test-unittest: "python -m pytest ./python/pylibraft/pylibraft/test"
test-smoketest: "python ./ci/wheel_smoke_test_pylibraft.py"
script: ci/test_wheel_pylibraft.sh
wheel-build-raft-dask:
needs: wheel-tests-pylibraft
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-build.yaml@branch-23.08
with:
build_type: pull-request
package-name: raft_dask
package-dir: python/raft-dask
before-wheel: "RAPIDS_PY_WHEEL_NAME=pylibraft_${{ '${PIP_CU_VERSION}' }} rapids-download-wheels-from-s3 ./local-pylibraft && python -m pip install --no-deps ./local-pylibraft/pylibraft*.whl"
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
script: "ci/build_wheel_raft_dask.sh"
wheel-tests-raft-dask:
needs: wheel-build-raft-dask
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-test.yaml@branch-23.08
with:
build_type: pull-request
package-name: raft_dask
# Always want to test against latest dask/distributed.
test-before-amd64: "RAPIDS_PY_WHEEL_NAME=pylibraft_${{ '${PIP_CU_VERSION}' }} rapids-download-wheels-from-s3 ./local-pylibraft-dep && pip install --no-deps ./local-pylibraft-dep/pylibraft*.whl && pip install git+https://github.com/dask/dask.git@2023.3.2 git+https://github.com/dask/distributed.git@2023.3.2.1 git+https://github.com/rapidsai/dask-cuda.git@branch-23.06"
test-before-arm64: "RAPIDS_PY_WHEEL_NAME=pylibraft_${{ '${PIP_CU_VERSION}' }} rapids-download-wheels-from-s3 ./local-pylibraft-dep && pip install --no-deps ./local-pylibraft-dep/pylibraft*.whl && pip install git+https://github.com/dask/dask.git@2023.3.2 git+https://github.com/dask/distributed.git@2023.3.2.1 git+https://github.com/rapidsai/dask-cuda.git@branch-23.06"
test-unittest: "python -m pytest ./python/raft-dask/raft_dask/test"
test-smoketest: "python ./ci/wheel_smoke_test_raft_dask.py"
script: ci/test_wheel_raft_dask.sh
16 changes: 6 additions & 10 deletions .github/workflows/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,39 +16,35 @@ on:
jobs:
conda-cpp-tests:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@branch-23.08
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
conda-python-tests:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.08
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
wheel-tests-pylibraft:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-test.yaml@branch-23.08
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
package-name: pylibraft
test-unittest: "python -m pytest ./python/pylibraft/pylibraft/test"
script: ci/test_wheel_pylibraft.sh
wheel-tests-raft-dask:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@branch-23.06
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-test.yaml@branch-23.08
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
package-name: raft_dask
test-before-amd64: "pip install git+https://github.com/dask/dask.git@2023.3.2 git+https://github.com/dask/distributed.git@2023.3.2.1 git+https://github.com/rapidsai/dask-cuda.git@branch-23.06"
test-before-arm64: "pip install git+https://github.com/dask/dask.git@2023.3.2 git+https://github.com/dask/distributed.git@2023.3.2.1 git+https://github.com/rapidsai/dask-cuda.git@branch-23.06"
test-unittest: "python -m pytest ./python/raft-dask/raft_dask/test"
script: ci/test_wheel_raft_dask.sh
Loading