Skip to content

Move retry policy decision outside post execute log group block - #73023

Merged
amoghrajesh merged 1 commit into
apache:mainfrom
astronomer:retry-policy-improvement-1-log-group
Sep 15, 2026
Merged

amoghrajesh merged 1 commit into
apache:mainfrom
astronomer:retry-policy-improvement-1-log-group

Conversation

@amoghrajesh

Copy link
Copy Markdown
Contributor

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

What

When a retry policy is configured, the decision it makes ("retry" or "fail", and why) is only ever logged as a single structlog line: "Retry policy decision". That line renders inside the ::group::Post Execute collapsed section of the task log viewer, so a Dag author or a user viewing logs has to expand a collapsed group to see why their task did or didn't retry, which is the one place this information surfaces today.

Current behaviour

_evaluate_retry_policy logs the decision unconditionally after the Post Execute group is opened by the calling exception handler, so the log viewer hides it by default.

Proposed change

Close the Post Execute group immediately before logging the retry decision, so the line renders visibly instead of nested inside a collapsed section. Nothing meaningful is logged between the retry decision and the end of the task run in any failure path (_finalize_task_failure logs nothing, and run()'s finally block only logs on rare supervisor-send failures), so the group is left closed rather than reopened.

Testing

  • Ran the example_llm_retry_policy DAG

Before:

image image

Now:

image
  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@kaxil

kaxil commented Sep 12, 2026

Copy link
Copy Markdown
Member

2nd screenshot also still shows it in "post execute"

@amoghrajesh

Copy link
Copy Markdown
Contributor Author

@kaxil first and second screenshots were before the changes. Third is after

@kaxil

kaxil commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

@kaxil first and second screenshots were before the changes. Third is after

Second and third looks same to me apart from your highlight/selected text

Oh I see it now!

@amoghrajesh amoghrajesh added this to the Airflow 3.4.0 milestone Sep 15, 2026
@amoghrajesh
amoghrajesh merged commit 1e50b89 into apache:main Sep 15, 2026
107 checks passed
@amoghrajesh
amoghrajesh deleted the retry-policy-improvement-1-log-group branch September 15, 2026 03:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants