-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[C++] A way to assert that arrays of a certain type have the validity of the values defined by the validity bitmap #41114
Comments
felipecrv
added a commit
that referenced
this issue
Apr 23, 2024
### Rationale for this change To make it easier to find bugs that are very likely to be silent in the codebas because users rarely use unions and REE types. ### What changes are included in this PR? Adding the type predicate and two usages in `builder_nested.h`. ### Are these changes tested? By the compilation process, since they are both `static_asserts`. * GitHub Issue: #41114 Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com> Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Issue resolved by pull request 41115 |
zanmato1984
pushed a commit
to zanmato1984/arrow
that referenced
this issue
Apr 24, 2024
…pache#41115) ### Rationale for this change To make it easier to find bugs that are very likely to be silent in the codebas because users rarely use unions and REE types. ### What changes are included in this PR? Adding the type predicate and two usages in `builder_nested.h`. ### Are these changes tested? By the compilation process, since they are both `static_asserts`. * GitHub Issue: apache#41114 Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com> Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
tolleybot
pushed a commit
to tmct/arrow
that referenced
this issue
May 2, 2024
…pache#41115) ### Rationale for this change To make it easier to find bugs that are very likely to be silent in the codebas because users rarely use unions and REE types. ### What changes are included in this PR? Adding the type predicate and two usages in `builder_nested.h`. ### Are these changes tested? By the compilation process, since they are both `static_asserts`. * GitHub Issue: apache#41114 Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com> Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
vibhatha
pushed a commit
to vibhatha/arrow
that referenced
this issue
May 25, 2024
…pache#41115) ### Rationale for this change To make it easier to find bugs that are very likely to be silent in the codebas because users rarely use unions and REE types. ### What changes are included in this PR? Adding the type predicate and two usages in `builder_nested.h`. ### Are these changes tested? By the compilation process, since they are both `static_asserts`. * GitHub Issue: apache#41114 Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com> Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the enhancement requested
Certain types don't have the validity defined by the bitmap buffer's presence and bit values:
NA
SPARSE_UNION
DENSE_UNION
RUN_END_ENCODED
This means that recently
inline bool ArrayData::IsValid()
had to grow:arrow/cpp/src/arrow/array/data.h
Lines 201 to 216 in cd607d0
Many type-specific loops in Arrow don't use
IsValid
and check the bitmap directly instead (good to reduce the branching in hot loops or to do lower-level operations with the bitmaps).To be future-proof, these loops would have to migrate to using
IsValid
, but since doing that would be overkill for most cases (unions and REEs are rare), I propose the creation of a function close to the special validity checking functions that checks if a type allows for validity checks to be performed solely through the bitmap:This means that if a template or function that only looks at the validity bitmap starts being used for these special types we can get
static_assert
,assert
andDCHECK
violations.Component(s)
C++
The text was updated successfully, but these errors were encountered: