-
Notifications
You must be signed in to change notification settings - Fork 807
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement FromStr
for DataType / Parse DataType description
#3821
Comments
I think this is a good first issue because the code and tests already exists, so this ticket would be a matter of porting the code, and adjusting the interface |
This is nice extention. I want to take this ticket. |
Should we also keep the part like, |
I'm not sure I follow what you mean, but it should be able to parse the output of |
Sorry for the ambiguity. I mean, should we migrate parts that have been completed in |
that would be my suggestion |
Maybe we can port the implementation in arrow-datafusion first and then add support for |
If this issue is still open, can I take a stab at it please? |
Yes please @opensourcegeek -- I think this would be a fairly straightforward addition (port code, add some docs and tests) |
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As part of implementing apache/datafusion#5016 in DataFusion I needed some way to convert from a string passed in by the user to a
DataType
.Since we already had a function
arrow_typeof
that provides a useful human readable type name by callingDataType::to_string()
I wanted the opposite: A way to take the output ofDataType::to_string()
and make aDataType
Describe the solution you'd like
I think having
FromStr
implementation that matches the https://docs.rs/arrow/34.0.0/arrow/datatypes/enum.DataType.html#impl-Display-for-DataType implementation for DataType would be very nice.Example usage:
Describe alternatives you've considered
@tustvold pointed out that there is already a way to encode data types in String for the IPC format https://arrow.apache.org/docs/format/CDataInterface.html#data-type-description-format-strings
While this format is (designed to be) easy to parse by computers, I don't think it is easy to parse by Humans (quick quiz, what type does
tdD
represent?)Additional context
See apache/datafusion#5166 (comment)
We can probably lift the implementation from apache/datafusion#5166 into Arrow-rs
The implementation in apache/datafusion#5166 currently does not cover:
Dictionary
)The text was updated successfully, but these errors were encountered: