Skip to content

Comments

Restrict EC2CreateInstanceOperator cleanup to waiter failures and add guard flag#61272

Merged
vincbeck merged 1 commit intoapache:mainfrom
SameerMesiah97:EC2CreateInstanceOperator-Cleanup-Flag
Feb 5, 2026
Merged

Restrict EC2CreateInstanceOperator cleanup to waiter failures and add guard flag#61272
vincbeck merged 1 commit intoapache:mainfrom
SameerMesiah97:EC2CreateInstanceOperator-Cleanup-Flag

Conversation

@SameerMesiah97
Copy link
Contributor

@SameerMesiah97 SameerMesiah97 commented Jan 30, 2026

Description

This PR follows up on PR #60904 by refining cleanup behavior in EC2CreateInstanceOperator. Cleanup behavior is guarded by a flag and is opted in by default. Cleanup is only triggered for post-start EC2 instance failures (including WaiterError), ensuring termination is attempted only when an instance was successfully created and avoiding interception of non-AWS exceptions.

Rationale

Restricting cleanup to post-creation EC2 instance failures prevents unintended termination in unrelated failure paths while still addressing orphaned instances created during execution, and aligns EC2 cleanup semantics with recent changes to other AWS resource creation operators proposed in PRs #61145, #61051, and #61010 .

Tests

Existing tests continue to validate cleanup behavior. The new terminate_instance_on_failure flag is now explicitly set to True in applicable tests for explicitness.

Documentation

The docstring for EC2CreateInstanceOperator has been updated with a brief description of the new flag term.

Backwards Compatibility

A new flag, terminate_instance_on_failure, has been added to EC2CreateInstanceOperator with a default value of True. Best-effort cleanup will now be attempted if a post-creation failure (including WaiterError) occurs after the EC2 instance has been successfully created.

@boring-cyborg boring-cyborg bot added area:providers provider:amazon AWS/Amazon - related issues labels Jan 30, 2026
@SameerMesiah97
Copy link
Contributor Author

@vincbeck

Sorry for tagging you here but this is effectively the same pattern for the other resource-leaking AWS operators applied retrospectively to EC2CreateInstanceOperator. Since you reviewed them, I figure you might be interested in reviewing this PR.

@vincbeck vincbeck merged commit 1491ee1 into apache:main Feb 5, 2026
90 checks passed
jhgoebbert pushed a commit to jhgoebbert/airflow_Owen-CH-Leung that referenced this pull request Feb 8, 2026
…61272)

Co-authored-by: Sameer Mesiah <smesiah971@gmail.com>
Ratasa143 pushed a commit to Ratasa143/airflow that referenced this pull request Feb 15, 2026
…61272)

Co-authored-by: Sameer Mesiah <smesiah971@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants