Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generate group offsets from element labels #11017

Merged
merged 67 commits into from
Jun 3, 2022
Merged

Generate group offsets from element labels #11017

merged 67 commits into from
Jun 3, 2022

Conversation

ttnghia
Copy link
Contributor

@ttnghia ttnghia commented Jun 1, 2022 •
edited
Loading

Given an array of integer values which may be the labels of some list elements, we want to generate an array of offsets so we can create a lists column from these offsets.

For example:

input_labels = [ 0, 0, 0, 0, 1, 1, 4, 4, 4, 4 ]
output = [ 0, 4, 6, 6, 6, 10 ]

This is basically the reverse operation of #10945.

Closes #10955.

ttnghia added 30 commits May 23, 2022 14:54
@ttnghia
Switch to use generate_list_labels
41a7baa
@ttnghia
Remove comments
1204e9a
@ttnghia
Switch to use 0-based list labels
165f752
@ttnghia
Implement fill_segmented_labels
3f385eb
@ttnghia
Move file and change file name
2684625
@ttnghia
Use fill_segmented_labels in groupby
0cfe856
@ttnghia
Add comment
1007176
@ttnghia
Add example
9b5a88d
@ttnghia
Rename and move file
9903007
@ttnghia
Rename variable
7bd714d
@ttnghia
Add a benchmark
031302b
@ttnghia
Rewrite label_segments
28e1463
@ttnghia
Fix compile error
d3708a5
@ttnghia
Hack to test
60259c9
@ttnghia
Revert "Add a benchmark"
97b9a57 
This reverts commit 031302b.

# Conflicts:
#	cpp/benchmarks/groupby/group_struct.cu
@ttnghia
Revert "Hack to test"
a094dab 
This reverts commit 60259c9.

# Conflicts:
#	cpp/benchmarks/groupby/group_struct.cu
@ttnghia
Merge branch 'branch-22.08' into list_label
8315764
@ttnghia
Add comment
f5a5520
@ttnghia
Merge branch 'list_label' into offsets_from_labels
3be6b16
@ttnghia
Add comment clarifying bound check
a060e3d
@ttnghia
Implement labels_to_offsets
6ac3f49
@ttnghia
Rewrite example
b8cb363
@ttnghia
Merge branch 'list_label' into offsets_from_labels
2218efa
@ttnghia
Fix initialization
9bdfa6c
@ttnghia
Use label_to_offsets
baa303f
@ttnghia
Change variable name
039b92f
@ttnghia
Reverse comments. They will be removed completely later on so don't c…
ab1e25a 
…hange them now.
@ttnghia
Remove unused headers
f9e0819
@ttnghia
Rewrite example
52f6c30
@ttnghia
Merge branch 'branch-22.08' into list_label
8e1f01a
ttnghia added 6 commits May 31, 2022 16:43
@ttnghia
Fix comment
77002ef 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Rename out_ iterators into label_
136511d 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Merge branch 'list_label' into offsets_from_labels
fea7d6d
@ttnghia
Rewrite comments
6578a3e
@ttnghia
Merge branch 'branch-22.08' into offsets_from_labels
0e24124 
# Conflicts:
#	cpp/include/cudf/detail/labeling/label_segments.cuh
#	cpp/src/lists/drop_list_duplicates.cu
@ttnghia
Update comments
88d5554
@ttnghia ttnghia added feature request New feature or request 3 - Ready for Review Ready for review by team libcudf blocker libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS non-breaking Non-breaking change labels Jun 1, 2022
@ttnghia ttnghia requested review from devavret and jrhemstad June 1, 2022 18:05
@ttnghia ttnghia requested a review from a team as a code owner June 1, 2022 18:05
@ttnghia ttnghia self-assigned this Jun 1, 2022
@codecov

This comment was marked as off-topic.

Sign in to view
@ttnghia ttnghia mentioned this pull request Jun 3, 2022
Merged
@ttnghia
Copy link
Contributor Author

ttnghia commented Jun 3, 2022

@gpucibot merge

@rapids-bot rapids-bot bot merged commit a042be6 into rapidsai:branch-22.08 Jun 3, 2022
@ttnghia ttnghia deleted the offsets_from_labels branch June 3, 2022 21:43
rapids-bot bot pushed a commit that referenced this pull request Jul 26, 2022
@ttnghia
Support set operations (#11043)
b5a2efb 
This PR adds the following APIs for set operations:
 * `lists::have_overlap`
 * `lists::intersect_distinct`
 * `lists::union_distinct`
 * `lists::difference_distinct`

### Name Convention
Except for the first API (`lists::have_overlap`) that returns a boolean column, the suffix `_distinct` of the rest APIs denotes that their results will be lists columns in which all list rows have been post-processed to remove duplicates. As such, their results are actually "set" columns in which each row is a "set" of distinct elements.

---

Depends on:
 * #10945
 * #11017
 * NVIDIA/cuCollections#175
 * #11052
 * #11118
 * #11100
 * #11149

Closes #10409.

Authors:
  - Nghia Truong (https://github.com/ttnghia)
  - Yunsong Wang (https://github.com/PointKernel)

Approvers:
  - Michael Wang (https://github.com/isVoid)
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Bradley Dice (https://github.com/bdice)
  - Yunsong Wang (https://github.com/PointKernel)

URL: #11043
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@devavret devavret devavret approved these changes

@jrhemstad jrhemstad jrhemstad approved these changes

Assignees

@ttnghia ttnghia

Labels
3 - Ready for Review Ready for review by team feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change Spark Functionality that helps Spark RAPIDS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] Generate offsets from element labels
3 participants
@ttnghia @devavret @jrhemstad