S3DeleteObjectsOperator Added ability to filter keys by last modified time
#39151
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boring-cyborg
bot
added
area:providers
provider:amazon-aws
idantepper
force-pushed
the
add-options-to-S3DeleteObjectsOperator
branch
from
34c788d
to
d5ecc23
Compare
vincbeck
reviewed
Apr 22, 2024
ferruzzi
reviewed
Apr 22, 2024
idantepper
force-pushed
the
add-options-to-S3DeleteObjectsOperator
branch
from
d5ecc23
to
27be7e5
Compare
|
hi, I added the at least one function and changed the validation of params to use it |
vincbeck
reviewed
Apr 23, 2024
added 6 commits
May 4, 2024 18:31
…o TYPE_CHECKING section
idantepper
force-pushed
the
add-options-to-S3DeleteObjectsOperator
branch
from
27be7e5
to
bdf2275
Compare
romsharon98
reviewed
May 4, 2024
romsharon98
approved these changes
May 4, 2024
ferruzzi
approved these changes
May 7, 2024
eladkal
approved these changes
May 7, 2024
66 tasks
pateash
pushed a commit
to pateash/airflow
that referenced
this pull request
May 13, 2024
…ed time (apache#39151) * (S3DeleteObjectsOperator): Added ability to delete keys by last modified time * (operator/s3.py): changed the import class to datetime and moved it to TYPE_CHECKING section * (utils/helpers.py): added at_least_one function * (operators/s3.py): changed validations to use at_least_one helper * (operators/s3.py): changed validation of given variables in init and execute * (operators/s3.py): changed validation of given variables in init and execute --------- Co-authored-by: idantepper <idantepper@github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
expose cabability from the s3 hook to the delete operator
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.