Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement from_unixtime function #2871

Closed
andygrove opened this issue Jul 11, 2022 · 7 comments · Fixed by #2924
Closed

Implement from_unixtime function #2871

andygrove opened this issue Jul 11, 2022 · 7 comments · Fixed by #2924
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@andygrove
Copy link
Member

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In the SO post https://stackoverflow.com/questions/72888852/extract-year-month-day-from-unix-timestamp-column-in-rust-datafusion-dataframe/72941102#72941102 the user needed help with translating a unixtime to a timestamp. The solution now is to use a cast and this is verbose and not obvious. It would be nice to have a specific from_unixtime function.

Describe the solution you'd like
See https://spark.apache.org/docs/2.3.0/api/sql/index.html#from_unixtime for Spark's version

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@andygrove andygrove added enhancement New feature or request good first issue Good for newcomers labels Jul 11, 2022
@waitingkuo
Copy link
Contributor

@andygrove
would you mind giving me some hint? I'm still new to this project. thanks~

Is this the right place to begin with https://github.com/apache/arrow-datafusion/blob/master/datafusion/expr/src/built_in_function.rs#L150 ?

@comphead
Copy link
Contributor

@waitingkuo https://github.com/apache/arrow-datafusion/blob/master/CONTRIBUTING.md#how-to-add-a-new-scalar-function this can be some start for you

@comphead
Copy link
Contributor

#2324 please check this PR also

@waitingkuo
Copy link
Contributor

thank you @comphead !

@waitingkuo
Copy link
Contributor

@comphead i finally figured out how to add a new function, thank you~

@andygrove
the current implementation only support Int64 as input value, should we support floating number and utf-8 as well?
the pyspark version has the second argument to render user preferred time format, should we support that in this pr?

@MrDataPsycho
Copy link
Contributor

@waitingkuo Can you please refer me where you have added that code? I only found 2 references of the function:

  1. expr_fn.rs
  2. timestamp.rs

But could not find the actual function. I want to add some example usage doc in the function. As it looks empty without any example: crates doc . Or if you can tell how I should add some doc. The contribution guide says the function should be added into datetime expression. But could not find that function there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants