Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EPIC] Normalization should properly handle airbyte_type_format #12907

Closed
3 tasks
alexandr-shegeda opened this issue May 17, 2022 · 2 comments
Closed
3 tasks
Assignees
Labels
Epic team/destinations Destinations team's backlog

Comments

@alexandr-shegeda
Copy link
Contributor

Tell us about the problem you're trying to solve

As part of the [EPIC] Expand date-time schema declarations to describe timezone awareness we would like to add the ability for all destinations to handle additional information from the JSON schema about
timezone aware/unaware.

There are next date-related types that should be covered:

  • timestamp with/without timezone
  • date
  • time with/without timezone

Describe the solution you’d like

  • Add possibility to handle airbyte_type_format information from JSON Schema types. For example, it can provide information about TZairbyte_type_format: timestamp_with_timezone or without_timezone.
  • destinations/normalization to take this into account and create columns with the appropriate type
  • destination acceptance tests should verify that destinations can appropriately handle these types

Additional context

It is a common practice for OLTP databases that for all time-related data types except timestamp, it doesn't care about TZ and returns a constant time value irrespective of the DB timezone. (Exception is time data type for Postgres)
For example, we have date-time with value "2011-10-04 12:58:36", so with session timezone +00:00 or -06:00 it will return the same value "2011-10-04 12:58:36

@grishick
Copy link
Contributor

please add keep @edgao @ChristopheDuong and me as reviewers when the PR is ready

@evantahler
Copy link
Contributor

This appears to be complete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Epic team/destinations Destinations team's backlog
Projects
None yet
Development

No branches or pull requests

7 participants