Skip to content

Add deferred task resume flow to task lifecycle diagram - #65323

Closed
yuseok89 wants to merge 3 commits into
apache:mainfrom
yuseok89:docs/task-lifecycle-deferred-flow
Closed

yuseok89 wants to merge 3 commits into
apache:mainfrom
yuseok89:docs/task-lifecycle-deferred-flow

Conversation

@yuseok89

@yuseok89 yuseok89 commented Apr 15, 2026 •

Copy link
Copy Markdown
Contributor

closes: #28580

The deferred branch of the task lifecycle diagram asked "A triggerer task?", a question about task type, at exactly the point where the reader needs the event timeline. Waiting, the trigger firing, and the return to scheduled were all collapsed behind that one branch, so the path out of deferred existed but could not be followed.

Diagram, deferred region

The branch now reads as a sequence of events.

From Branch
deferred → "Trigger fired?" NO → back to deferred, the wait loop
YES → "Task completes in the triggerer?"
"Task completes in the triggerer?" NO → Scheduler, an ordinary deferrable task resumes on a worker
YES → the existing task-end handling, which is the end_from_trigger path

"Task completes in the triggerer?" is not a rename for its own sake. It names the BaseTaskEndEvent overload of handle_event_submit, whose user-facing subclasses are TaskSuccessEvent, TaskFailedEvent and TaskSkippedEvent.

A task that raises a defer signal now transitions straight to deferred rather than to the Triggerer node. The worker writes DEFERRED and the trigger row in a single Execution API call, and a triggerer claims the trigger afterwards, so the old edge drew a synchronous hand-off that does not happen. The edge carries a trigger registered label so the triggerer is still part of the story.

Diagram, upstream failure check

Separate from the deferred work, Required upstream task(s) failed? had its branches backwards. NO pointed at upstream_failed and YES continued toward scheduled. Its two sibling diamonds are oriented correctly, so this was an authoring slip rather than a house convention. It sits on the single path every task takes out of the Scheduler box, and the deferred resume path this PR documents runs straight through it, so it is fixed here. Happy to split it into its own PR if reviewers prefer.

Prose

deferring.rst credited the scheduler with the deferred to scheduled write. The triggerer performs it, and the scheduler's own deferred to scheduled write is the trigger-timeout failure path. The same sentence promised a return to a worker unconditionally, which is not true for end_from_trigger. Both are corrected, and the paragraph that restated the bullets above it is reduced to the cross-reference it was carrying.

Screenshots

Deferred region

deferred_before_after

Upstream failure check

upstream_before_after
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    • Opus 5

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@yuseok89
yuseok89 marked this pull request as ready for review April 16, 2026 01:31
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Apr 22, 2026
@yuseok89
yuseok89 force-pushed the docs/task-lifecycle-deferred-flow branch from b6ad7b5 to ffa29cc Compare June 12, 2026 14:52
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Jul 30, 2026
@yuseok89 yuseok89 changed the title Docs: Add deferred task resume flow to task lifecycle diagram Add deferred task resume flow to task lifecycle diagram Aug 1, 2026
@github-actions github-actions Bot removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Aug 3, 2026
@yuseok89
yuseok89 force-pushed the docs/task-lifecycle-deferred-flow branch from cf74efa to 38a269a Compare September 14, 2026 04:24
@yuseok89
yuseok89 marked this pull request as draft September 14, 2026 04:36
@yuseok89
yuseok89 marked this pull request as ready for review September 15, 2026 14:24
@potiuk potiuk added the closed because of open PR limit Closed as a one-time step of introducing the open pull request limit label Sep 25, 2026
@potiuk

potiuk commented Sep 25, 2026

Copy link
Copy Markdown
Member

Hello @yuseok89 - thank you for your contributions to Apache Airflow!

The Airflow community has introduced a limit of 5 open pull requests at a time for contributors without write access to the repository. You currently have 11 open pull requests, so - as a one-time step of introducing the limit - we closed the ones where maintainers have not engaged yet:

These pull requests stay open because maintainers are already engaged in them - they count towards your limit:

This is not a judgement of you or of your changes. We never told contributors before that opening many pull requests at once was a problem, so there is nothing to feel bad about - and nothing is lost: your branches, commits and the review history stay where they are.

What we ask you to do is to make your first prioritization decision: choose which of the pull requests above matter most to you, and reopen them (up to 5 open at a time, including the ones still open) with the "Reopen pull request" button or gh pr reopen <PR_NUMBER> --repo apache/airflow. Reopen the ones you are ready to follow through - keep them rebased, respond to review comments and fix failing checks.

While your pull requests are waiting for review, the most valuable thing you can do is help in other ways - reviewing other contributors' pull requests, helping with issues, and taking part in the discussions on the devlist and Slack.

Why we introduced the limit, what it means for you and how to reopen or restore a pull request is explained in https://github.com/apache/airflow/blob/main/contributing-docs/32_open_pull_request_limit.rst.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

closed because of open PR limit Closed as a one-time step of introducing the open pull request limit kind:documentation ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update Task Life Cycle Diagram to include flow for Deferred Operators

2 participants