Repository navigation
Kubernetes Pod Operator Task keeps Pod with Pending/Running state in detele_succeded_pod #59083
Description
Apache Airflow version
3.1.3
If "Other Airflow 2/3 version" selected, which one?
No response
What happened?
Hi!
We have detected that the Airflow Worker does not delete the Kubernetes Pod Operator for Pending/Running State while using the on_finish_action with DELETE_SUCCEEDED_POD.
What you think should happen instead?
I believe it should delete Pods with Pending state. Otherwise, they will consume resources. Only the failed should be Kept, so, we should change the logic to DELETE_NON_FAILED_POD.
How to reproduce
In case you spin up a KPO Pod without machine using DELETE_SUCCEEDED_POD for on_finish_action, and Airflow Worker timeout in the spin up, the Pod will be kept there waiting for a machine and if the machine become available it will start to run
Operating System
apache:airflow-3.1.3 on Kubernetes
Versions of Apache Airflow Providers
Kubernetes Provider only.
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
[-]Kubernetes Pod Operator Task keeps Pod with Pending state in detele_succeded_pod[/-][+]Kubernetes Pod Operator Task keeps Pod with Pending/Running state in detele_succeded_pod[/+]shubham36deshpande commentedon Dec 7, 2025
Hi @potiuk, this seems reasonable should we implement this?
if yes, can I pick this issue up?
SameerMesiah97 commentedon Dec 7, 2025
@shubham36deshpande I am about to submit a PR for this. But if you have something ready, I think you can open it.
SameerMesiah97 commentedon Dec 7, 2025
The pull request #59160 has been opened to close this issue. I understand that the bug reporter requested the logic for
'delete_succeeded_pod' to be changed to delete all pods that have not 'Failed' but this might risk breaking backwards compatibility. Therefore, I propose adding a new flag 'delete_active_pods' to handle the cleanup of zombie pods in 'Running' or 'Pending' state on task completion or termination.
@wolvery @shubham36deshpande — please feel free to review or comment if you believe any scenarios are missing, or if you'd still prefer to propose an alternative approach.
wolvery commentedon Dec 14, 2025
hey @jscheffl, I believe we should delete the pods for all scenarios and not failed Pods.
I created this PR, If you could review it, please:
#59413