Skip to content

Fix short-circuit inside a mapped task group not skipping downstream - #73959

Merged
kaxil merged 2 commits into
apache:mainfrom
astronomer:fix-short-circuit-in-mapped-task-group
Oct 1, 2026
Merged

kaxil merged 2 commits into
apache:mainfrom
astronomer:fix-short-circuit-in-mapped-task-group

Conversation

@kaxil

@kaxil kaxil commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

A short-circuit task inside a mapped task group stopped skipping the task after it in 3.2.0.

Each map index of a task in a mapped task group runs as its own task instance, so SkipMixin writes the skip decision to XCom under that map index. #62287 changed NotPreviouslySkippedDep to read that XCom at map_index=-1 unless the parent is a MappedOperator, to fix an unmapped LatestOnlyOperator upstream of a mapped task. A task inside a mapped task group is a plain operator whose is_mapped is False, so the dep read -1, found no decision, and let the downstream task run.

The dep now checks get_needs_expansion(), which is true for a mapped operator and for any task inside a mapped task group: the tasks that write XCom per map index. An unmapped parent still reads -1, and the regression test from #62287 passes unchanged.

A gate that returns False for odd map indexes, gate(v) >> printstr(v) inside a task group expanded over [0, 1, 2, 3], run by the scheduler before and after the change:

Before: group_1.printstr runs for every map index

After: group_1.printstr is skipped for map indexes 1 and 3

related: #55225


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

…tasks

NotPreviouslySkippedDep looked up the SkipMixin decision at map_index -1 unless the
parent was a MappedOperator. A task inside a mapped task group is expanded per map
index and writes its skip XCom per map index, so the decision was never found and
the downstream task ran. Use get_needs_expansion() so both cases read the child's
map index.
@kaxil
kaxil merged commit 884e687 into apache:main Oct 1, 2026
78 checks passed
@kaxil
kaxil deleted the fix-short-circuit-in-mapped-task-group branch October 1, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants