Language SDK task gets no task log when its runtime cannot launch #75557

Under which category would you file this issue?

Task SDK

Apache Airflow version

3.4.0b2

What happened and how to reproduce it?

supervise_task reports a launch failure to the API server and the task log only when the coordinator raises TaskLaunchError. SubprocessCoordinator.execute_task raises it only for native Dag files. These failures raise other exceptions:

• @task.stub task whose artifact is missing: _resolve_artifact_bundle and _build_execute_task_command raise FileNotFoundError (e.g. no JAR in the task_handler_bundle_name bundle).
• Runtime exits or times out before connecting (bad Java classpath, Go executable needing a newer glibc, missing Node module, ...): _accept_connections raises RuntimeError / TimeoutError. This applies to native Dags too.
• Runtime cannot be executed at all (java not installed, executable built for another architecture, no execute permission): subprocess.Popen raises OSError. A runtime declaring a supervisor schema version the worker does not support raises ValueError the same way.

The exception reaches the executor before task_instances.start(), so:

• the try has no task log;
• the task is failed through the scheduler's "state mismatch" path (_process_executor_events), which looks like an executor problem;
• the runtime's stdout/stderr, already read into drained by _accept_connections, is discarded. On a successful start it would have been forwarded to the task log by _register_pipe_readers(data=drained), so the actual error (e.g. Error: Could not find or load main class ...) ends up in no log at all.

Reproduction

A script in Breeze calls the real supervise_task with a JavaCoordinator, mocking only the API client (create_autospec(Client)), _is_native_dag_file (returns False, i.e. a stub task) and _resolve_artifact_bundle (a temp directory):

• Case A: the bundle directory holds no JAR.
• Case B: the runtime command is sh -c "echo 'Error: Could not find or load main class com.example.MyTasks' >&2; exit 1".
• Case C (control): the same failure raised as TaskLaunchError, as the native Dag path does.

===== Case A: no JAR in the task_handler_bundle_name bundle =====
supervise_task raised FileNotFoundError: cannot find a JAR with Main-Class metadata in /tmp/tmppcubeudw
task_instances.start called:  False
task_instances.finish called: False
task_instances.retry called:  False
task log size: 0 bytes

===== Case B: runtime exits before connecting =====
[info     ] Starting subprocess            [coordinators.subprocess] loc=_subprocess.py:372 pid=21
supervise_task raised RuntimeError: process exited with 1 before connecting
task_instances.start called:  False
task_instances.finish called: False
task_instances.retry called:  False
task log size: 0 bytes
runtime stderr in task log: False

===== Case C (control): the same failure raised as TaskLaunchError =====
[warning  ] Cannot launch the task's runtime [supervisor] reason='cannot find a JAR with Main-Class metadata in /bundle'
supervise_task returned normally
task_instances.start called:  True
task_instances.retry called:  True
task log size: 231 bytes
task log:
{"level":"error","event":"Cannot launch the task's runtime","reason":"cannot find a JAR with Main-Class metadata in /bundle","logger":"task"}

In case B, the Error: Could not find or load main class ... line printed by the runtime appears nowhere in the output.

What a user sees

Cases A and B follow the executor and scheduler code paths (not checked in a running UI):

What you think should happen instead?

Raise any failure before the task instance is started as TaskLaunchError, so the task is started, fails with the reason in its task log, and the runtime's captured output is included.

These should be retryable, like a native Dag file that is missing or cannot run: an artifact can appear once the bundle finishes syncing during a deployment, and a runtime that timed out on a busy worker can start on the next try. A routing mistake, which TaskLaunchError already reports as not retryable, stays that way.

Operating System

No response

Deployment

None

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

No response

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

Related: #74036 introduced TaskLaunchError for native Dag files; #73970 moved artifacts into Dag bundles, which makes the missing-artifact case likely during upgrades.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct