Repository navigation
Resolve a trigger's XCom requests as its deferred task instance - #74772
Merged
Merged
Conversation
ashb
requested review from
amoghrajesh,
dstandish,
hussein-awala and
kaxil
as code owners
October 14, 2026 22:25
boring-cyborg
Bot
added
area:API
Note: _None_ of this is a security boundary, and should not be treated as such. The Execution API resolves XCom reads and writes for a task inside a loop to the caller's own iteration, using the task identity in the caller's JWT. Triggers have no such identity: the triggerer talks to an in-process API that treats every request as the all-zero task instance. A trigger's XCom requests therefore resolve as an anonymous caller rather than as the deferred task it runs for, so it can neither read (a 400) nor write the XComs of a task inside a loop. Triggers can also share one runner coroutine and one comms channel (the "shared streams" idea), so only the asyncio task running a trigger's code knows which trigger it is; a context variable carries that trigger's task instance id to each request frame. Shared-stream groups are started from whichever trigger subscribes first but serve several triggers, so they start with it cleared rather than inheriting that trigger's id. This isn't ideal, but will do "for now". The trigger runner process (the one running async code) sets the task instance id, and since that process runs user code a trigger can claim to be any task instance. The (in-process) API therefore checks the claim. A write acts as the task instance only when it is deferred and its trigger is already assigned to the calling triggerer. A read acts as it whenever that try is still live, because after a user marks a deferred task failed or successful the trigger row is gone and the triggerer still runs the trigger's on_kill(). The most a misbehaving trigger can do is write as another task that is deferred on the same triggerer, or read as any live task, and it can already read and write every task's XComs by coordinates through the in-process API. As stated, this is _not_ a security boundary we can rely on Requests that fail the check are rejected rather than falling back to the anonymous identity, deliberately: a trigger whose task instance is no longer deferred, or whose trigger was reassigned, should not still be writing its task's XComs, and falling back would only work for tasks outside loops. Clearing a deferred task archives its try before on_kill() runs, so on_kill() after a clear cannot read its task's XComs; a trigger that needs a job id there should keep it on the trigger, as the in-tree triggers already do. Only the in-process API that the triggerer creates knows which triggerer is calling, so every other API rejects the header rather than trusting it. A triggerer that eventually talks to a real API server will need its own credential before this can work there. Only the XCom routes honour the trigger's task instance id for now. get_previous_task_instance also resolves relative to the caller, so a trigger asking for its previous task instance still gets the latest pass rather than the one at its own loop position. Every other request from the triggerer keeps its current identity, so access rules and the live-attempt check on those routes are unchanged.
ashb
force-pushed
the
triggerer-ti-identity
branch
from
October 14, 2026 22:38
85b8f08 to
2dc8f37
Compare
kaxil
reviewed
Oct 15, 2026
Pin the HEAD and DELETE rules in the trigger claim check and assert which check denied each 403, so loosening the read-method test no longer passes.
Ignore a trigger failure reported by a triggerer that no longer owns the trigger. The XCom write check now answers 403 to a stale copy on its previous owner, which raises before yielding and would otherwise have Trigger.submit_failure send the task to __fail__ while the new owner is still running it.
kaxil
approved these changes
Oct 15, 2026
Contributor
|
Hi maintainer, this PR was merged without a milestone set.
|
Contributor
Backport failed to create: v3-4-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker 61f1d4b v3-4-testThis should apply the commit to the v3-4-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
ashb
added a commit
that referenced
this pull request
Oct 17, 2026
* Resolve a trigger's XCom requests as its deferred task instance Note: _None_ of this is a security boundary, and should not be treated as such. The Execution API resolves XCom reads and writes for a task inside a loop to the caller's own iteration, using the task identity in the caller's JWT. Triggers have no such identity: the triggerer talks to an in-process API that treats every request as the all-zero task instance. A trigger's XCom requests therefore resolve as an anonymous caller rather than as the deferred task it runs for, so it can neither read (a 400) nor write the XComs of a task inside a loop. Triggers can also share one runner coroutine and one comms channel (the "shared streams" idea), so only the asyncio task running a trigger's code knows which trigger it is; a context variable carries that trigger's task instance id to each request frame. Shared-stream groups are started from whichever trigger subscribes first but serve several triggers, so they start with it cleared rather than inheriting that trigger's id. This isn't ideal, but will do "for now". The trigger runner process (the one running async code) sets the task instance id, and since that process runs user code a trigger can claim to be any task instance. The (in-process) API therefore checks the claim. A write acts as the task instance only when it is deferred and its trigger is already assigned to the calling triggerer. A read acts as it whenever that try is still live, because after a user marks a deferred task failed or successful the trigger row is gone and the triggerer still runs the trigger's on_kill(). The most a misbehaving trigger can do is write as another task that is deferred on the same triggerer, or read as any live task, and it can already read and write every task's XComs by coordinates through the in-process API. As stated, this is _not_ a security boundary we can rely on Requests that fail the check are rejected rather than falling back to the anonymous identity, deliberately: a trigger whose task instance is no longer deferred, or whose trigger was reassigned, should not still be writing its task's XComs, and falling back would only work for tasks outside loops. Clearing a deferred task archives its try before on_kill() runs, so on_kill() after a clear cannot read its task's XComs; a trigger that needs a job id there should keep it on the trigger, as the in-tree triggers already do. Only the in-process API that the triggerer creates knows which triggerer is calling, so every other API rejects the header rather than trusting it. A triggerer that eventually talks to a real API server will need its own credential before this can work there. Only the XCom routes honour the trigger's task instance id for now. get_previous_task_instance also resolves relative to the caller, so a trigger asking for its previous task instance still gets the latest pass rather than the one at its own loop position. Every other request from the triggerer keeps its current identity, so access rules and the live-attempt check on those routes are unchanged. * fixup! Resolve a trigger's XCom requests as its deferred task instance Pin the HEAD and DELETE rules in the trigger claim check and assert which check denied each 403, so loosening the read-method test no longer passes. * fixup! Resolve a trigger's XCom requests as its deferred task instance Ignore a trigger failure reported by a triggerer that no longer owns the trigger. The XCom write check now answers 403 to a stale copy on its previous owner, which raises before yielding and would otherwise have Trigger.submit_failure send the task to __fail__ while the new owner is still running it. (cherry picked from commit 61f1d4b) Adapted to v3-4-test's CurrentTIToken: the typed execution tokens from the Dag processor scoped access work (#74370) are not on this branch.
ashb
added a commit
that referenced
this pull request
Oct 17, 2026
…) (#74868) * Resolve a trigger's XCom requests as its deferred task instance Note: _None_ of this is a security boundary, and should not be treated as such. The Execution API resolves XCom reads and writes for a task inside a loop to the caller's own iteration, using the task identity in the caller's JWT. Triggers have no such identity: the triggerer talks to an in-process API that treats every request as the all-zero task instance. A trigger's XCom requests therefore resolve as an anonymous caller rather than as the deferred task it runs for, so it can neither read (a 400) nor write the XComs of a task inside a loop. Triggers can also share one runner coroutine and one comms channel (the "shared streams" idea), so only the asyncio task running a trigger's code knows which trigger it is; a context variable carries that trigger's task instance id to each request frame. Shared-stream groups are started from whichever trigger subscribes first but serve several triggers, so they start with it cleared rather than inheriting that trigger's id. This isn't ideal, but will do "for now". The trigger runner process (the one running async code) sets the task instance id, and since that process runs user code a trigger can claim to be any task instance. The (in-process) API therefore checks the claim. A write acts as the task instance only when it is deferred and its trigger is already assigned to the calling triggerer. A read acts as it whenever that try is still live, because after a user marks a deferred task failed or successful the trigger row is gone and the triggerer still runs the trigger's on_kill(). The most a misbehaving trigger can do is write as another task that is deferred on the same triggerer, or read as any live task, and it can already read and write every task's XComs by coordinates through the in-process API. As stated, this is _not_ a security boundary we can rely on Requests that fail the check are rejected rather than falling back to the anonymous identity, deliberately: a trigger whose task instance is no longer deferred, or whose trigger was reassigned, should not still be writing its task's XComs, and falling back would only work for tasks outside loops. Clearing a deferred task archives its try before on_kill() runs, so on_kill() after a clear cannot read its task's XComs; a trigger that needs a job id there should keep it on the trigger, as the in-tree triggers already do. Only the in-process API that the triggerer creates knows which triggerer is calling, so every other API rejects the header rather than trusting it. A triggerer that eventually talks to a real API server will need its own credential before this can work there. Only the XCom routes honour the trigger's task instance id for now. get_previous_task_instance also resolves relative to the caller, so a trigger asking for its previous task instance still gets the latest pass rather than the one at its own loop position. Every other request from the triggerer keeps its current identity, so access rules and the live-attempt check on those routes are unchanged. * fixup! Resolve a trigger's XCom requests as its deferred task instance Pin the HEAD and DELETE rules in the trigger claim check and assert which check denied each 403, so loosening the read-method test no longer passes. * fixup! Resolve a trigger's XCom requests as its deferred task instance Ignore a trigger failure reported by a triggerer that no longer owns the trigger. The XCom write check now answers 403 to a stale copy on its previous owner, which raises before yielding and would otherwise have Trigger.submit_failure send the task to __fail__ while the new owner is still running it. (cherry picked from commit 61f1d4b) Adapted to v3-4-test's CurrentTIToken: the typed execution tokens from the Dag processor scoped access work (#74370) are not on this branch.
vatsrahul1001
pushed a commit
that referenced
this pull request
Oct 20, 2026
…) (#74868) * Resolve a trigger's XCom requests as its deferred task instance Note: _None_ of this is a security boundary, and should not be treated as such. The Execution API resolves XCom reads and writes for a task inside a loop to the caller's own iteration, using the task identity in the caller's JWT. Triggers have no such identity: the triggerer talks to an in-process API that treats every request as the all-zero task instance. A trigger's XCom requests therefore resolve as an anonymous caller rather than as the deferred task it runs for, so it can neither read (a 400) nor write the XComs of a task inside a loop. Triggers can also share one runner coroutine and one comms channel (the "shared streams" idea), so only the asyncio task running a trigger's code knows which trigger it is; a context variable carries that trigger's task instance id to each request frame. Shared-stream groups are started from whichever trigger subscribes first but serve several triggers, so they start with it cleared rather than inheriting that trigger's id. This isn't ideal, but will do "for now". The trigger runner process (the one running async code) sets the task instance id, and since that process runs user code a trigger can claim to be any task instance. The (in-process) API therefore checks the claim. A write acts as the task instance only when it is deferred and its trigger is already assigned to the calling triggerer. A read acts as it whenever that try is still live, because after a user marks a deferred task failed or successful the trigger row is gone and the triggerer still runs the trigger's on_kill(). The most a misbehaving trigger can do is write as another task that is deferred on the same triggerer, or read as any live task, and it can already read and write every task's XComs by coordinates through the in-process API. As stated, this is _not_ a security boundary we can rely on Requests that fail the check are rejected rather than falling back to the anonymous identity, deliberately: a trigger whose task instance is no longer deferred, or whose trigger was reassigned, should not still be writing its task's XComs, and falling back would only work for tasks outside loops. Clearing a deferred task archives its try before on_kill() runs, so on_kill() after a clear cannot read its task's XComs; a trigger that needs a job id there should keep it on the trigger, as the in-tree triggers already do. Only the in-process API that the triggerer creates knows which triggerer is calling, so every other API rejects the header rather than trusting it. A triggerer that eventually talks to a real API server will need its own credential before this can work there. Only the XCom routes honour the trigger's task instance id for now. get_previous_task_instance also resolves relative to the caller, so a trigger asking for its previous task instance still gets the latest pass rather than the one at its own loop position. Every other request from the triggerer keeps its current identity, so access rules and the live-attempt check on those routes are unchanged. * fixup! Resolve a trigger's XCom requests as its deferred task instance Pin the HEAD and DELETE rules in the trigger claim check and assert which check denied each 403, so loosening the read-method test no longer passes. * fixup! Resolve a trigger's XCom requests as its deferred task instance Ignore a trigger failure reported by a triggerer that no longer owns the trigger. The XCom write check now answers 403 to a stale copy on its previous owner, which raises before yielding and would otherwise have Trigger.submit_failure send the task to __fail__ while the new owner is still running it. (cherry picked from commit 61f1d4b) Adapted to v3-4-test's CurrentTIToken: the typed execution tokens from the Dag processor scoped access work (#74370) are not on this branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note: None of this is a security boundary, and should not be treated as
such.
The Execution API resolves XCom reads and writes for a task inside a loop
to the caller's own iteration, using the task identity in the caller's JWT.
Triggers have no such identity: the triggerer talks to an in-process API
that treats every request as the all-zero task instance. A trigger's XCom
requests therefore resolve as an anonymous caller rather than as the
deferred task it runs for, so it can neither read (a 400) nor write the
XComs of a task inside a loop.
Triggers can also share one runner coroutine and one comms channel (the
"shared streams" idea), so only the asyncio task running a trigger's code
knows which trigger it is; a context variable carries that trigger's task
instance id to each request frame. Shared-stream groups are started from
whichever trigger subscribes first but serve several triggers, so they start
with it cleared rather than inheriting that trigger's id. This isn't ideal,
but will do "for now".
The trigger runner process (the one running async code) sets the task
instance id, and since that process runs user code a trigger can claim to be
any task instance. The (in-process) API therefore checks the claim. A write
acts as the task instance only when it is deferred and its trigger is
already assigned to the calling triggerer. A read acts as it whenever that
try is still live, because after a user marks a deferred task failed or
successful the trigger row is gone and the triggerer still runs the
trigger's on_kill(). The most a misbehaving trigger can do is write as
another task that is deferred on the same triggerer, or read as any live
task, and it can already read and write every task's XComs by coordinates
through the in-process API. As stated, this is not a security boundary we
can rely on
Requests that fail the check are rejected rather than falling back to the
anonymous identity, deliberately: a trigger whose task instance is no
longer deferred, or whose trigger was reassigned, should not still be
writing its task's XComs, and falling back would only work for tasks
outside loops. Clearing a deferred task archives its try before on_kill()
runs, so on_kill() after a clear cannot read its task's XComs; a trigger
that needs a job id there should keep it on the trigger, as the in-tree
triggers already do.
Only the in-process API that the triggerer creates knows which triggerer
is calling, so every other API rejects the header rather than trusting it.
A triggerer that eventually talks to a real API server will need its own
credential before this can work there.
Only the XCom routes honour the trigger's task instance id for now.
get_previous_task_instance also resolves relative to the caller, so a
trigger asking for its previous task instance still gets the latest pass
rather than the one at its own loop position. Every other request from the
triggerer keeps its current identity, so access rules and the live-attempt
check on those routes are unchanged.
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.