Make the default task failure email say what failed by rjgoyln · Pull Request #73221 · apache/airflow

rjgoyln · 2026-09-15T19:14:18Z

Summary

The default task failure email does not say what failed. Its subject renders {{ ti }}, which on Airflow 2 gave the TaskInstance repr and now dumps every field of a Pydantic model:

Airflow alert: id=UUID('019c3eb1-…') task_id='fail' dag_id='test-email-on-failure' run_id='manual__2026-02-08T19:18:47.078356+00:00' try_number=1 dag_version_id=UUID('019c3ea3-…') map_index=-1 context_carrier={} task=<Task(_PythonDecoratedOperator): fail> bundle_instance=GitHubAppDagBundle(name=github-app) … state=<TaskInstanceState.FAILED: 'failed'>

Neither body names the Dag or the task; the notifier's bundled template renders {{ ti.state }}, an enum whose str() is its Python name; and both offer a "Mark success" link that goes to the log, since Airflow 3 aliases mark_success_url to log_url.

both paths use the subject [Airflow] <dag_id>.<task_id> <state> - Run <run_id>
both bodies name the Dag, task, run, state and timestamps, and render the state's value
the "Mark success" link is gone unless the URL actually differs from the log URL
the bundled template's heading follows the state, so it still reads on the success and retry callbacks that share it
RuntimeTaskInstance.__str__ mirrors the scheduler-side TaskInstance.__repr__

The default subject no longer interpolates {{ ti }}, but subject_template files carried over from Airflow 2 do. It is __str__ rather than __repr__ so those read cleanly while a debugger still shows every field.

Emails sent from the Dag processor are for tasks that never reached their own callbacks, and that request carries no state, so they take it from the email type instead.

The template work is adapted from #67931, stale since July, where the heading was hardcoded to a failure. #73271 is folded in here because it rewrites the same template blocks.

Behavior change

Both subject formats change; providers/smtp/docs/changelog.rst carries the note for anyone whose mail filters match on the notifier's.

closes: #61667

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

rjgoyln · 2026-09-27T11:08:55Z

Thanks, all addressed.

I’ve updated the state handling, removed the [Airflow] prefix, added autospec=True to the new mocks, and simplified the template assertions.

For the render test, I also switched to fixed create_runtime_ti values and literal expected output, so the test is independent of the implementation details.

The default email subject renders {{ ti }}, which on Airflow 2 produced the TaskInstance repr. On Airflow 3 the worker-side task instance is a Pydantic model with no textual form of its own, so every field — UUIDs, the task object, the bundle instance — lands in the subject line, and operators can no longer tell at a glance which task failed.

Adapted from apache#67931 by Wilmer Dooley, whose PR went stale. The bundled body never named the Dag or the task, and the state field rendered the Python enum repr rather than the state itself. The heading is derived from the state so the template still reads correctly on the success and retry callbacks that share it.

Review follow-up. The parametrized test took over the tail of the defaults test, the state rationale was repeated in both templates, and the changed subject needs a note for anyone whose mail filters match on it.

The legacy email path never said which Dag or task had failed, and neither template carried the timestamps operators need to line the failure up against the rest of a run.

The Dag-processor callback path builds its task instance from the callback request alone, so it has no dates at all, and the Dag's Jinja environment is StrictUndefined -- a bare truth test on the attribute aborts the whole email.

The scheduler-side column is not nullable, but the Task SDK model declares map_index as optional and the surrounding code already guards for None, so mirroring the scheduler check alone would print "map_index=None".

The legacy path still put the task instance repr in the subject, UUID and all, which is what the issue reported; the two paths also disagreed on the shape of the subject and on whether the body named the state.

Airflow 3 has no mark-success endpoint, so mark_success_url returns log_url and the email offered the same destination twice under a label that promises something it cannot do. Folded in from apache#73271, which this supersedes.

Review follow-up: the new variable is usable from custom subject and content templates, so it belongs in the email configuration guide and the release note rather than appearing unexplained in an example.

Those emails are for tasks that never reached their own callbacks, and the callback request carries no state, so every one of them said the state was unknown -- while the request's email type is exactly the reason it fired.

Drop the subject prefix, report a missing state as the UI does, and set the Dag-processor state where the email type is already branched on. The templates are no longer compared verbatim in tests; the rendered output is.

Rebase adaptation: the Python 3.10 drop moved ruff's target version, and it now flags timezone.utc in favour of the alias.

boring-cyborg Bot added the area:task-sdk label Sep 15, 2026

rjgoyln changed the title ~~Fix task failure emails dumping raw task instance fields~~ Fix default task failure emails dumping Python internals Sep 17, 2026

rjgoyln changed the title ~~Fix default task failure emails dumping Python internals~~ Make the default task failure email say what failed Sep 17, 2026

rjgoyln mentioned this pull request Sep 17, 2026

Stop labelling the task log link as "Mark Success" in alert emails #73271

Closed

1 task done

rjgoyln force-pushed the fix/runtime-ti-readable-str branch 2 times, most recently from 3d9ab8b to bee3249 Compare September 17, 2026 13:15

rjgoyln marked this pull request as ready for review September 17, 2026 13:31

rjgoyln requested review from XD-DENG, amoghrajesh, ashb, ephraimbuddy, hussein-awala, jedcunningham and kaxil as code owners September 17, 2026 13:31

ashb reviewed Sep 25, 2026

View reviewed changes

rjgoyln force-pushed the fix/runtime-ti-readable-str branch from 9918d30 to 8ad2d18 Compare September 27, 2026 11:07

rjgoyln force-pushed the fix/runtime-ti-readable-str branch 2 times, most recently from d6d4c05 to 8ad2d18 Compare September 27, 2026 11:14

rjgoyln added 13 commits October 8, 2026 18:58

Add newsfragment

2f28a21

Restore the try-number assertion the new SMTP test displaced

d1f7a08

Review follow-up. The parametrized test took over the tail of the defaults test, the state rationale was repeated in both templates, and the changed subject needs a note for anyone whose mail filters match on it.

Name the Dag, task, run and timestamps in task failure emails

7abfaa1

The legacy email path never said which Dag or task had failed, and neither template carried the timestamps operators need to line the failure up against the rest of a run.

Carry the timestamps into the legacy task failure email

cfd20f8

The Dag-processor callback path builds its task instance from the callback request alone, so it has no dates at all, and the Dag's Jinja environment is StrictUndefined -- a bare truth test on the attribute aborts the whole email.

Treat a null map_index as unmapped in the task instance string

a2d5efd

The scheduler-side column is not nullable, but the Task SDK model declares map_index as optional and the surrounding code already guards for None, so mirroring the scheduler check alone would print "map_index=None".

Give both task failure email paths the same subject

39146c1

The legacy path still put the task instance repr in the subject, UUID and all, which is what the issue reported; the two paths also disagreed on the shape of the subject and on whether the body named the state.

Document the task_state template variable

7f6edb4

Review follow-up: the new variable is usable from custom subject and content templates, so it belongs in the email configuration guide and the release note rather than appearing unexplained in an example.

Report the state on emails sent from the Dag processor

9144ec4

Those emails are for tasks that never reached their own callbacks, and the callback request carries no state, so every one of them said the state was unknown -- while the request's email type is exactly the reason it fired.

Address review on the default task failure email

c2f5774

Drop the subject prefix, report a missing state as the UI does, and set the Dag-processor state where the email type is already branched on. The templates are no longer compared verbatim in tests; the rendered output is.

Use the datetime.UTC alias in the SMTP notifier tests

9635de5

Rebase adaptation: the Python 3.10 drop moved ruff's target version, and it now flags timezone.utc in favour of the alias.

rjgoyln force-pushed the fix/runtime-ti-readable-str branch from b52535a to 9635de5 Compare October 8, 2026 11:22

Navigation Menu

Repository navigation

Make the default task failure email say what failed - #73221

Make the default task failure email say what failed#73221
rjgoyln wants to merge 13 commits into
apache:mainfrom
rjgoyln:fix/runtime-ti-readable-str

rjgoyln commented Sep 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rjgoyln commented Sep 27, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

rjgoyln commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Behavior change

Was generative AI tooling used to co-author this PR?

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rjgoyln commented Sep 27, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rjgoyln commented Sep 15, 2026 •

edited

Loading