-
Notifications
You must be signed in to change notification settings - Fork 738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Different numeric type may be able to compare #6357
Comments
The major difficulty here is combinatorial explosion of code generation that would result, something that is already a major issue. This not only bloats binaries but contributes to long compilation times. Instead what most query engines, such as DataFusion do, is use the cast kernels to coerce the inputs to a common type before performing the comparison. This may not be as optimal as doing the comparison on the heterogenous types, but is likely good enough given such heterogeneous comparisons are comparatively rare. |
It is also fairly straightforward to implement your own kernels for specific comparisons if you have special cases that are not handled by arrow-rs (the challenge is mananging the code complexity for all the possible type combinations, as alluded to above by @tustvold ) |
arrow-rs/arrow-ord/src/cmp.rs
Lines 239 to 243 in 6bf2bda
Crates like num-cmp shows that different numeric type (e.g., u64 and i64) is comparable.
Is it possible to extend arrow array comparison to support compare these two kind of datum?
The text was updated successfully, but these errors were encountered: