Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Migrate nn-descent updates from RAFT #364

Open
cjnolet opened this issue Sep 27, 2024 · 0 comments
Open

[FEA] Migrate nn-descent updates from RAFT #364

cjnolet opened this issue Sep 27, 2024 · 0 comments
Assignees
@divyegala
Labels
feature request New feature or request

Comments

@cjnolet
Copy link
Member

cjnolet commented Sep 27, 2024

Recently, there were several updates made to the nn-descent algorithm in RAFT which need to be migrated over to cuVS. This includes:

  1. Ability to compute mutual reachability distance by applying an epilogue during the distance computation.
  2. Batching API for scalability
  3. Ability to specify whether distances should be returned with search results (this option should ultimately be made available to all index types)

There might be more updates. @divyegala please keep me honest here.
@cjnolet cjnolet added the feature request New feature or request label Sep 27, 2024
@benfred benfred mentioned this issue Sep 27, 2024
Merged
rapids-bot bot pushed a commit to rapidsai/cuml that referenced this issue Oct 4, 2024
@benfred
Migrate to use cuVS for vector search (#6085)
70fe526 
This PR updates to use cuVS instead of RAFT for vector search, pairwise distances and clustering. This is required for us to deprecate the vector search functionality in RAFT, in favour of the code in cuVS.

Because some code hasn't been migrated over to cuvs yet, we will continue to use the version in RAFT - but with RAFT in header only mode. In particular this functionality will be used in RAFT header only mode:

* Random Ball Cover (see rapidsai/cuvs#218)
* Sparse KNN
* nn-descent rapidsai/cuvs#364
* [MetricProcessor](c7d1b0e)
* knn_merge_parts
* build_dendrogram_host
* build_sorted_mst
*  raft DistanceType

Because sparse KNN in RAFT uses the DistanceType in RAFT, we can't fully move over to use the DistanceType code in cuVS with this PR. (Also the DistanceType code in RAFT has a `Precomputed` option that isn't available in cuvs - but is needed by cuml for dbscan.)  This means that we have both the raft and cuvs DistanceType enum's in use with this change, with conversions between them.

Authors:
  - Ben Frederickson (https://github.com/benfred)
  - Bradley Dice (https://github.com/bdice)
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)
  - Corey J. Nolet (https://github.com/cjnolet)

Approvers:
  - Corey J. Nolet (https://github.com/cjnolet)
  - Dante Gama Dessavre (https://github.com/dantegd)
  - Bradley Dice (https://github.com/bdice)

URL: #6085
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@divyegala divyegala

Labels
feature request New feature or request
Projects
VS/ML/DM Primitives Release Board
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cjnolet @divyegala