Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(core): register timestamptz type to DataFusion timestamp with time zone type #908

Merged
merged 8 commits into from
Nov 14, 2024
Merged

feat(core): register timestamptz type to DataFusion timestamp with time zone type #908

merged 8 commits into from
Nov 14, 2024

Conversation

goldmedal
Copy link
Contributor

@goldmedal goldmedal commented Nov 14, 2024 •
edited
Loading

Description

Now we can distinguish TIMESTAMP and TIMESTAMP WITH TIME ZONE in the DataFusion layer. We also know the sqlglot will transpile SQL for Bigquery

  • TIMESTAMP -> DATETIME ( without timezone)
  • TIMESTAMP WITH TIME ZONE -> TIMESTAMP ( with timezone)

We can apply DataFusion's type coercion rule for them. If needed, DataFusion will cast the timestamp side to the timestamp with the time zone.

Additional change

  • Enhance the validation script. Now we can set the log level for the detailed debug information.
LOGURU_LEVEL=DEBUG python tools/mdl_validation.py mdl.json function.csv
  • add boolean alias for bool type
  • add required function current_datetime for BigQuery

@github-actions github-actions bot added core ibis rust Pull requests that update Rust code python Pull requests that update Python code labels Nov 14, 2024
@goldmedal goldmedal marked this pull request as ready for review November 14, 2024 06:53
@goldmedal goldmedal marked this pull request as draft November 14, 2024 08:24
@goldmedal goldmedal marked this pull request as ready for review November 14, 2024 08:56
grieve54706
grieve54706 approved these changes Nov 14, 2024
View reviewed changes
Copy link
Contributor

@grieve54706 grieve54706 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @goldmedal.

@grieve54706 grieve54706 added this to the 0.11.3 milestone Nov 14, 2024
@goldmedal goldmedal merged commit bc4a6da into Canner:main Nov 14, 2024
15 checks passed
@goldmedal goldmedal deleted the feature/define-timezone-type branch November 14, 2024 09:30
@goldmedal
Copy link
Contributor Author

Thanks @grieve54706 for reviewing

@goldmedal goldmedal restored the feature/define-timezone-type branch November 14, 2024 09:35
@goldmedal goldmedal deleted the feature/define-timezone-type branch November 15, 2024 02:19
grieve54706 pushed a commit that referenced this pull request Dec 13, 2024
@goldmedal
feat(core): register timestamptz type to DataFusion timestamp with ti…
935677e 
…me zone type (#908)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@grieve54706 grieve54706 grieve54706 approved these changes

Assignees
No one assigned
Labels
bigquery core ibis python Pull requests that update Python code rust Pull requests that update Rust code
Projects
None yet
Milestone
0.12.0
Development

Successfully merging this pull request may close these issues.
2 participants
@goldmedal @grieve54706