Skip to content

Go SDK: delete the XComs of earlier tries before a task runs - #74178

Merged
jason810496 merged 2 commits into
apache:mainfrom
FrankYang0529:airflow-go-sdk-clear-earlier-try-xcoms
Oct 5, 2026
Merged

jason810496 merged 2 commits into
apache:mainfrom
FrankYang0529:airflow-go-sdk-clear-earlier-try-xcoms

Conversation

@FrankYang0529

Copy link
Copy Markdown
Member

Follow-up for #74075 (comment).

When a task instance starts a new try, Airflow keeps the XComs it already has and lists their keys in ti_context.xcom_keys_to_clear for the runtime to delete. The Python task runner deletes them before it renders templates. The Go runtime never read the field, so a retried Go task still saw the XComs of its previous try. With this change the Go runtime deletes them too, and the retried task no longer sees them.

  • RunTask sends a DeleteXCom for each key after it finds the task and before it binds arguments or runs the task.
  • If a delete fails, the task does not run. It ends with a RetryTask when should_retry is set and with a FAILED TaskState otherwise.
  • dag.If no longer writes {"skipped": []} to skipmixin_key before the condition runs, since the delete now removes the list from an earlier try.

Was generative AI tooling used to co-author this PR?
  • Yes - Claude Code

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, the TS and Java SDK runtime didn't respect the xcom_keys_to_clear as well.
Please feel free to raise follow-up PRs for them as well, thanks.

@FrankYang0529

Copy link
Copy Markdown
Member Author

IIRC, the TS and Java SDK runtime didn't respect the xcom_keys_to_clear as well. Please feel free to raise follow-up PRs for them as well, thanks.

Thanks for the review. Created two follow-up PRs:

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks. Except the reusability comment, rest of the change LGTM.

Comment thread go-sdk/pkg/execution/client.go Resolved
Signed-off-by: PoAn Yang <payang@apache.org>
…XCom

Signed-off-by: PoAn Yang <payang@apache.org>
@FrankYang0529
FrankYang0529 force-pushed the airflow-go-sdk-clear-earlier-try-xcoms branch from 411d274 to e8ac4e0 Compare October 5, 2026 10:29
@jason810496
jason810496 merged commit e08216d into apache:main Oct 5, 2026
90 checks passed
@FrankYang0529
FrankYang0529 deleted the airflow-go-sdk-clear-earlier-try-xcoms branch October 5, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants