-
Notifications
You must be signed in to change notification settings - Fork 14.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add start execution from triggerer support to dynamic task mapping #39912
Add start execution from triggerer support to dynamic task mapping #39912
Conversation
6c9bbfd
to
5c2b6de
Compare
The code here is not yet well structured but has been tested locally and seems to work fine. It would be great if I could get some reviews and suggestions. Thanks! |
245c685
to
4ff77db
Compare
This PR is ready to be reviewed but blocked by #39585 so I'll keep it as draft for now. |
590b172
to
e1f7aee
Compare
18f4ddc
to
403ced5
Compare
f0343d2
to
e30bb4c
Compare
Probably shouldn’t be an approval (the exception class likely needs discussion, and I would prefer the function in AbstractOperator to be a part of this PR), but close enough. |
1671692
to
06c45f4
Compare
The latest comments from @uranusjr have been addressed. I'll go ahead and merge this one. |
…expand start trigger
…art_from_trigger and _expand_start_trigger_args
…ct operator to unify how base op and mapped op retrieve start_from_trigger
…ct operator to unify how base op and mapped op retrieve start_trigger_args
…d validate start_trigger_args passed to _defer_task" to TaskDeferralError
06c45f4
to
636265e
Compare
…pache#39912) * feat(dagrun): add start_from_trigger support to mapped operator * feat(mapped_operator): add partial support to start_trigger_args * feat(mappedoperator): do not include xcom when expanding start trigger args and flag
This is a follow-up PR of #39585, which enables the mapped operator to start execution from triggerer
Why
Dynamic task mapping is not yet supported in #38674 and #39585.
What
Allow DAG authors to provide argument
trigger_kwargs
andstart_from_trigger
in__init__
which can be used inexpand
.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.