-
Notifications
You must be signed in to change notification settings - Fork 16.3k
fixing file extension issue on SqlToS3Operator #54187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
564932b to
0ad29c5
Compare
6c7134c to
63bba8e
Compare
guan404ming
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, one nit.
providers/amazon/tests/unit/amazon/aws/transfers/test_sql_to_s3.py
Outdated
Show resolved
Hide resolved
44a854f to
3c0c39d
Compare
3c0c39d to
8fbebf6
Compare
|
This is going to change the user experience, but I think we can definitely catalogue this one as a bug fix. |
|
It's a breaking change. |
@bryanyang0528 Can you create a Github Issue with more details and reproduction steps. Thanks! |
Now loaded file have proper extension. refer below table to understand for extension
| File Format | Extension |
|------------------|--------------|
| CSV |
.csv|| CSV (gzip) |
.csv.gz|| JSON |
.json|| JSON (gzip) |
.json.gz|| Parquet |
.parquet|| Parquet (gzip) |
.parquet|updated and refactor unittest case for the same and added more
closes: #53976
related: #53976
POW:-
groupby_kwargspow_without_groupby.mp4
groupby_kwargshttps://github.com/user-attachments/assets/fa110550-72f9-40ec-bf05-e4d49dd58b7c
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.