-
Notifications
You must be signed in to change notification settings - Fork 253
feat: Support for StringSplit #2772
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: Support for StringSplit #2772
Conversation
comphead
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Shekharrajak for you contribution, please add a function to the fuzztesting kit, similar to #2755
|
In the past I think we've encountered differences in Java and Rust's regex engines wrt graphemes. Could we get some larger UTF-8 characters in the tests? |
We probably need to fall back to Spark unless this config is enabled: |
Thanks! Added in commit 8eddd29 |
Added tests 987b646 |
How can we check if it is not falling back to Spark's JVM execution? @andygrove |
spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
Outdated
Show resolved
Hide resolved
dbb34d5 to
1f8f2b2
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2772 +/- ##
=============================================
- Coverage 56.12% 45.46% -10.66%
- Complexity 976 1206 +230
=============================================
Files 119 157 +38
Lines 11743 14124 +2381
Branches 2251 2365 +114
=============================================
- Hits 6591 6422 -169
- Misses 4012 6683 +2671
+ Partials 1140 1019 -121 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Fixes #2707
We have scalar function:
https://datafusion.apache.org/user-guide/sql/scalar_functions.html#string-to-array