Clearing a task group re-triggers task instances that were not selected for clearing

[okayhooni]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.8.4

What happened?

We received a report from a user of our Airflow system stating that when they cleared a task group (without selecting the "Past" or "Future" options), task instances from other DagRuns, which were newly added to the task group but not selected for clearing, were also re-triggered.

Upon reviewing the audit logs, we confirmed that the parameters passed during the clear operation indicated that both the "Past" and "Future" options were set to False.

Although we were unable to reproduce the issue in our testing environment, we have confirmed a bug where tasks newly added to a task group (which were not included in the "Affected Tasks" list of the clear UI) are still executed unexpectedly.

We suspect that this issue might be caused by the filtering of tasks to be cleared, particularly due to the fact that only the current state of the task group is stored in the metadata database, rather than the historical changes, which may have led to this bug.

tasks_to_set_state = [task for task in task_group.iter_tasks() if isinstance(task, BaseOperator)]

• https://github.com/apache/airflow/blob/2.8.4/airflow/models/dag.py#L2151
• https://github.com/apache/airflow/blob/2.8.4/airflow/api/common/mark_tasks.py#L140

Additionally, we reviewed the latest code in the main branch but could not find any fixes related to this bug.

What you think should happen instead?

Only the task instances within the target task group and the specified DagRun should be cleared.

How to reproduce

Step 1) Create a DAG with task groups and generate multiple DagRuns (with catchup=True).
Step 2) Add a new task to a specific task group (and also add new tasks to other task groups).
Step 3) Clear a specific task group via the web UI.

Operating System

Debian GNU/Linux 12 (bookworm)

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct