CLI 'kubernetes cleanup-pods' should only clean up Airflow-created Pods #15193
Labels
area:CLI
kind:bug
This is a clearly a bug
provider:cncf-kubernetes
Kubernetes provider related issues
Comments
XD-DENG
added
area:CLI
kind:feature
|
@dimberman @mik-laj may you please share your inputs on if this makes sense to you or not? Let me know if I overlooked anything. Thanks |
|
in my opinion this is a bug and by default we should only manage pods created by Airflow i.e. we don't need extra option. |
My thoughts exactly! |
|
Hey, thanks for this issue @XD-DENG . I added this feature and we use it in Astronomer too -- to cleanup any hanging PODs and their sidecars. So yeah we can just restrict it to pod launched via Airflow. |
|
Thanks everyone. I think we have come to a consensus. So I will change |
XD-DENG
added
kind:bug
XD-DENG
changed the title
kubernetes cleanup-pods should allow "only clean up Pods created by Airflow"
XD-DENG
added a commit
that referenced
this issue
Apr 8, 2021
…eated Pods (#15204) closes: #15193 Currently condition if the pod is created by Airflow is not considered. This commit fixes this. We decide if the Pod is created by Airflow via checking if it has all the labels added in PodGenerator.construct_pod() or KubernetesPodOperator.create_labels_for_pod().
ashb
pushed a commit
that referenced
this issue
Apr 15, 2021
…eated Pods (#15204) closes: #15193 Currently condition if the pod is created by Airflow is not considered. This commit fixes this. We decide if the Pod is created by Airflow via checking if it has all the labels added in PodGenerator.construct_pod() or KubernetesPodOperator.create_labels_for_pod(). (cherry picked from commit c594d9c)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Update
We discussed and decide this CLI should always only clean up pods created by Airflow.
Description
Currently command
kubernetes cleanup-podscleans up Pods that meet specific conditions in a given namespace.Underlying code: https://github.com/apache/airflow/blob/2.0.1/airflow/cli/commands/kubernetes_command.py#L70
Use case / motivation
The problem to me is: users may have other non-Airflow stuff running in this specific namespace, and they may want to have different strategy/logic for Pod cleaning-up for these non-Airflow pods.
We may want to have an additional boolean flag for this command, like
--only-airflow-pods, so that users can decide if they only want to clean up Airflow-created Pods.It should not be hard to identify Airflow-created pods, given the specific labels Airflow adds to the Pods it creates (https://github.com/apache/airflow/blob/2.0.1/airflow/kubernetes/pod_generator.py#L385)
The text was updated successfully, but these errors were encountered: