We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When I try to concat two dictionary encoded columns it doesn't work
Concat constants
> select arrow_cast('foo', 'Dictionary(Int32, Utf8)') || arrow_cast('bar', 'Dictionary(Int32, Utf8)'); Error during planning: Cannot infer common string type for string concat operation Dictionary(Int32, Utf8) || Dictionary(Int32, Utf8)
Concat columns
> create table t as values (arrow_cast('foo', 'Dictionary(Int32, Utf8)'), arrow_cast('bar', 'Dictionary(Int32, Utf8)')); 0 row(s) fetched. Elapsed 0.010 seconds. > select column1 || column2 from t; Error during planning: Cannot infer common string type for string concat operation Dictionary(Int32, Utf8) || Dictionary(Int32, Utf8)
Both queries should result in the single string foobar
foobar
Found this working on #12063 with @dharanad
I believe the fix will be to extend string_concat_coercion (also changed in #12063) to also handle dictionaries.
string_concat_coercion
We can likely follow the model of dictionary comparison coercion:
datafusion/datafusion/expr-common/src/type_coercion/binary.rs
Line 886 in 121f330
The text was updated successfully, but these errors were encountered:
I think this is a good first issue as there is a clear and small reproducer and the place in the code is known
It should also include some new sqllogictests
Here are the instructions: https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest
Ideally you should be able to extend one of the existing test files in https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest/test_files
(you could uncomment the test in string_view.slt that was added in #12063)
string_view.slt
Sorry, something went wrong.
||
take
thinh2
Successfully merging a pull request may close this issue.
Describe the bug
When I try to concat two dictionary encoded columns it doesn't work
To Reproduce
Concat constants
Concat columns
Expected behavior
Both queries should result in the single string
foobar
Additional context
Found this working on #12063 with @dharanad
I believe the fix will be to extend
string_concat_coercion
(also changed in #12063) to also handle dictionaries.We can likely follow the model of dictionary comparison coercion:
datafusion/datafusion/expr-common/src/type_coercion/binary.rs
Line 886 in 121f330
The text was updated successfully, but these errors were encountered: