Skip to content

Introduce Add and IsEmpty operations to Roaring UDF #17650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vitstn merged 4 commits into from
Apr 24, 2025
Merged

Introduce Add and IsEmpty operations to Roaring UDF #17650

vitstn merged 4 commits into from
Apr 24, 2025

Conversation

@jsjant
Copy link
Contributor

@jsjant jsjant commented Apr 24, 2025 •
edited
Loading

Changelog entry

Add operation allows building aggregation queries. For instance, constucting Roaring bitmap from intermediate AGG_LIST may be replaced with custom UDAF using Add, which may be more space efficient

IsEmpty just nice operation to have, allows skip costly check for cardinality equals to zero

Changelog category

  • Improvement

Description for reviewers

@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 06:32:23 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9604daf has started.
⚪ 2025-04-24 06:32:40 UTC Artifacts will be uploaded here
⚪ 2025-04-24 06:35:04 UTC ya make is running...
🟡 2025-04-24 07:20:29 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
4976 4838 0 2 51 85

⚪ 2025-04-24 07:20:59 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-04-24 07:32:26 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
108 (only retried tests) 31 0 0 0 77

🟢 2025-04-24 07:32:32 UTC Build successful.
🟢 2025-04-24 07:32:45 UTC ydbd size 2.2 GiB changed* by +10.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0ab1114 merge: 9604daf diff diff %
ydbd size 2 354 023 296 Bytes 2 354 034 096 Bytes +10.5 KiB +0.000%
ydbd stripped size 494 131 680 Bytes 494 136 992 Bytes +5.2 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 06:32:24 UTC Pre-commit check linux-x86_64-release-asan for 9604daf has started.
⚪ 2025-04-24 06:32:39 UTC Artifacts will be uploaded here
⚪ 2025-04-24 06:35:03 UTC ya make is running...
🟡 2025-04-24 07:12:54 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1826 1773 0 8 33 12

⚪ 2025-04-24 07:13:11 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-24 07:25:08 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Details

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
53 (only retried tests) 33 0 5 5 10

⚪ 2025-04-24 07:25:16 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-04-24 07:37:24 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
23 (only retried tests) 10 0 4 0 9

🟢 2025-04-24 07:37:31 UTC Build successful.
🟢 2025-04-24 07:37:50 UTC ydbd size 3.9 GiB changed* by +13.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0ab1114 merge: 9604daf diff diff %
ydbd size 4 135 280 696 Bytes 4 135 294 096 Bytes +13.1 KiB +0.000%
ydbd stripped size 1 431 718 008 Bytes 1 431 723 576 Bytes +5.4 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

🟢 2025-04-24 06:35:02 UTC The validation of the Pull Request description is successful.

@jsjant jsjant marked this pull request as ready for review April 24, 2025 08:04
@jsjant jsjant requested a review from a team as a code owner April 24, 2025 08:04
@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 09:11:44 UTC Pre-commit check linux-x86_64-relwithdebinfo for e908781 has started.
⚪ 2025-04-24 09:12:02 UTC Artifacts will be uploaded here
⚪ 2025-04-24 09:14:32 UTC ya make is running...
⚫ 2025-04-24 09:33:04 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 09:24:27 UTC Pre-commit check linux-x86_64-release-asan for e908781 has started.
⚪ 2025-04-24 09:24:43 UTC Artifacts will be uploaded here
⚪ 2025-04-24 09:27:07 UTC ya make is running...
⚫ 2025-04-24 09:33:03 UTC Check cancelled

vitstn
vitstn previously approved these changes Apr 24, 2025
View reviewed changes
@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 09:39:33 UTC Pre-commit check linux-x86_64-release-asan for 0cd20d7 has started.
⚪ 2025-04-24 09:39:49 UTC Artifacts will be uploaded here
⚪ 2025-04-24 09:42:11 UTC ya make is running...
🟡 2025-04-24 10:17:34 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1827 1761 0 9 46 11

⚪ 2025-04-24 10:17:52 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-24 10:29:38 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Details

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
65 (only retried tests) 34 0 7 14 10

⚪ 2025-04-24 10:29:49 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-04-24 10:41:12 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
35 (only retried tests) 9 0 5 12 9

🟢 2025-04-24 10:41:19 UTC Build successful.
🟡 2025-04-24 10:41:37 UTC ydbd size 3.9 GiB changed* by +269.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 0ddb5fd merge: 0cd20d7 diff diff %
ydbd size 4 135 474 216 Bytes 4 135 749 872 Bytes +269.2 KiB +0.007%
ydbd stripped size 1 431 777 912 Bytes 1 431 872 312 Bytes +92.2 KiB +0.007%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@vitstn vitstn enabled auto-merge (squash) April 24, 2025 09:40
@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 10:48:06 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0cd20d7 has started.
⚪ 2025-04-24 10:48:32 UTC Artifacts will be uploaded here
⚪ 2025-04-24 10:51:24 UTC ya make is running...
🟡 2025-04-24 11:27:07 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
4977 4833 0 4 55 85

⚪ 2025-04-24 11:27:39 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-24 11:37:10 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
116 (only retried tests) 40 0 1 0 75

⚪ 2025-04-24 11:37:19 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-04-24 11:44:38 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
89 (only retried tests) 14 0 1 0 74

🟢 2025-04-24 11:44:44 UTC Build successful.
🟡 2025-04-24 11:44:57 UTC ydbd size 2.2 GiB changed* by +198.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 305b5cd merge: 0cd20d7 diff diff %
ydbd size 2 354 063 736 Bytes 2 354 267 280 Bytes +198.8 KiB +0.009%
ydbd stripped size 494 146 784 Bytes 494 183 488 Bytes +35.8 KiB +0.007%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

auto-merge was automatically disabled April 24, 2025 12:05

Head branch was pushed to by a user without write access

@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 12:09:22 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3ea3189 has started.
⚪ 2025-04-24 12:10:05 UTC Artifacts will be uploaded here
⚪ 2025-04-24 12:13:00 UTC ya make is running...
🟡 2025-04-24 13:20:15 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
4977 4842 0 1 47 87

⚪ 2025-04-24 13:20:45 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-04-24 13:32:14 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
105 (only retried tests) 27 0 0 0 78

🟢 2025-04-24 13:32:21 UTC Build successful.
🟡 2025-04-24 13:32:35 UTC ydbd size 2.2 GiB changed* by +360.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 305b5cd merge: 3ea3189 diff diff %
ydbd size 2 354 063 736 Bytes 2 354 432 776 Bytes +360.4 KiB +0.016%
ydbd stripped size 494 146 784 Bytes 494 200 640 Bytes +52.6 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

⚪ 2025-04-24 12:12:15 UTC Pre-commit check linux-x86_64-release-asan for 3ea3189 has started.
⚪ 2025-04-24 12:12:30 UTC Artifacts will be uploaded here
⚪ 2025-04-24 12:14:51 UTC ya make is running...
🟡 2025-04-24 13:12:06 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Details

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1827 1759 0 12 46 10

⚪ 2025-04-24 13:12:26 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-24 13:24:01 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Details

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
70 (only retried tests) 50 0 6 6 8

⚪ 2025-04-24 13:24:10 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-04-24 13:37:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
22 (only retried tests) 8 0 3 2 9

🟢 2025-04-24 13:37:50 UTC Build successful.
🟡 2025-04-24 13:38:07 UTC ydbd size 3.9 GiB changed* by +543.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 0ddb5fd merge: 3ea3189 diff diff %
ydbd size 4 135 474 216 Bytes 4 136 030 368 Bytes +543.1 KiB +0.013%
ydbd stripped size 1 431 777 912 Bytes 1 431 937 912 Bytes +156.2 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@vitstn vitstn merged commit ed72099 into ydb-platform:main Apr 24, 2025
17 of 19 checks passed
jsjant added a commit to jsjant/ydb that referenced this pull request Apr 25, 2025
@jsjant
Introduce Add and IsEmpty operations to Roaring UDF (ydb-platform#17650)
f18218e
@liruoko liruoko added the changelog/f25-3 PR участвует в списке изменений label Nov 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vitstn vitstn vitstn approved these changes

Assignees

No one assigned

Labels

changelog/f25-3 PR участвует в списке изменений improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jsjant @vitstn @liruoko