Skip to content

Conversation

@seanghaeli
Copy link
Contributor

Currently there's a bug that prevents MwaaTaskSensor from working in deferrable mode. It refers to an attribute end_from_trigger, the intention is for the trigger to complete without spinning up a call to execute_complete. However, end_from_trigger is a new attribute not yet implemented for this trigger, so I removed it. Without this change, the example_mwaa.py system integration test fails when deferrable=True on the MwaaTaskSensor, and after this change the same test passes.

@boring-cyborg boring-cyborg bot added area:providers provider:amazon AWS/Amazon - related issues labels Aug 15, 2025
@eladkal
Copy link
Contributor

eladkal commented Aug 15, 2025

However, end_from_trigger is a new attribute not yet implemented for this trigger, so I removed it.

Can you further explain?
If this is not avaliable in all airflow versions than we should workaround for the unsupported versions.

@Lee-W
Copy link
Member

Lee-W commented Aug 15, 2025

However, end_from_trigger is a new attribute not yet implemented for this trigger, so I removed it.

Can you further explain? If this is not avaliable in all airflow versions than we should workaround for the unsupported versions.

I just traced the code a bit. This argument does not exist in MwaaDagRunCompletedTrigger, AwsBaseWaiterTrigger, and BaseTrigger. It's a wrong example in example Dag so I think we're good to just remove it

@ferruzzi
Copy link
Contributor

is a new attribute not yet implemented for this trigger

To echo Elad's concern, does that mean the attribute is not in a released version? If it's an unreleased thing I'm fine with just pulling it from here. If the public has seen it, we're stuck with workarounds and deprecation.

@seanghaeli
Copy link
Contributor Author

seanghaeli commented Aug 15, 2025

So based on these docs I incorrectly assumed the end_from_trigger parameter would be supported by the BaseTrigger. Because of this, MwaaTaskSensor doesn't work in deferrable mode. In this PR I remove the usage of end_from_trigger and the sensor now works. There would not be any versioning concerns because this is the standard way a Trigger is used.

Copy link
Contributor

@ferruzzi ferruzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I think I see. Thanks for explaining.

@ferruzzi ferruzzi merged commit 4f3243f into apache:main Aug 15, 2025
77 checks passed
@ferruzzi ferruzzi deleted the ghaeli/mwaa-task-sensor-deferrable-fix branch August 15, 2025 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants