When a Batch job fails, the trigger remains in the deferred state, noting that the status in "FAILED" but not marking the task as done. The task only is marked as failed once it times out.
[2023-08-26, 23:31:28 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:32:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:33:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:34:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:35:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:36:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:37:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:38:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:39:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:40:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:41:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:42:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:43:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:44:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:45:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:46:29 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:47:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:48:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:49:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:50:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:51:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:52:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:53:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:54:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:55:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:56:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:57:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:58:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-26, 23:59:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-27, 00:00:30 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-27, 00:01:31 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
[2023-08-27, 00:02:31 UTC] {waiter_with_logging.py:129} INFO - Batch job 85f49c11-c0ab-4a05-b361-1d06a0c0b81d not ready yet: ['FAILED']
When a Batch job has failed, the Airflow task should be marked as failed immediately.
Apache Airflow version
2.7.0
What happened
When a Batch job fails, the trigger remains in the deferred state, noting that the status in "FAILED" but not marking the task as done. The task only is marked as failed once it times out.
What you think should happen instead
When a Batch job has failed, the Airflow task should be marked as failed immediately.
How to reproduce
Set
deferrable=Trueon a batch operator that starts a task that fails. The BatchOperator will continue checking the task until the operator times out.Operating System
Linux
Versions of Apache Airflow Providers
No response
Deployment
Virtualenv installation
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct