Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Kernel][Writes] Write timestamp as INT64 type to Parquet data files #3084

Merged
merged 1 commit into from
May 10, 2024

Conversation

vkorukanti
Copy link
Collaborator

Description

Write the timestamp as INT64 physical format in Parquet. Currently, it is written as INT96 which is a very old method of writing timestamp and deprecated a long time ago. Also, collect statistics, for timestamp type columns.

How was this patch tested?

Update the existing tests.

@vkorukanti vkorukanti merged commit f6ebe24 into delta-io:master May 10, 2024
10 checks passed
vkorukanti added a commit that referenced this pull request May 10, 2024
…les (#3084)

## Description
Write the `timestamp` as `INT64` physical format in Parquet. Currently,
it is written as `INT96` which is a very old method of writing timestamp
and deprecated a long time ago. Also, collect statistics, for
`timestamp` type columns.

## How was this patch tested?
Update the existing tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants