Skip to content

Scope plugin macros to their team in multi-team mode - #73224

Merged
o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:onikolas/multi-team-plugins/macro-scoping
Sep 22, 2026
Merged

o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:onikolas/multi-team-plugins/macro-scoping

Conversation

@o-nikolas

Copy link
Copy Markdown
Contributor

A team-scoped plugin's macros were reachable from every task's templates, so any team could call macros written to talk to another team's systems.

The worker decides this from a new multi_team flag on the run context rather than from its own config, which is not guaranteed to match the scheduler's (misconfigured or intentional attempt to read macros from other teams): reading it as disabled while it is in fact enabled would drop scoping and apply a team's macros everywhere. The existing team_name cannot carry that signal, being None both for a teamless task and for every task when multi-team is off, so a plugin declaring a team would otherwise lose its macros in single-team deployments.

Scoping is applied at attribute access rather than when macros are loaded, because the loader mutates a process-wide module and a worker process can serve tasks from more than one team.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:task-sdk labels Sep 15, 2026
@o-nikolas
o-nikolas force-pushed the onikolas/multi-team-plugins/macro-scoping branch from 6ff3bb2 to e8ff1c6 Compare September 18, 2026 02:31
A team-scoped plugin's macros were reachable from every task's templates,
so any team could call macros written to talk to another team's systems.

The worker decides this from a new ``multi_team`` flag on the run context
rather than from its own config, which is not guaranteed to match the
scheduler's (misconfigured or intentional attempt to read macros from
other teams): reading it as disabled while it is in fact enabled would drop
scoping and apply a team's macros everywhere. The existing ``team_name``
cannot carry that signal, being ``None`` both for a teamless task and for
every task when multi-team is off, so a plugin declaring a team would
otherwise lose its macros in single-team deployments.

Scoping is applied at attribute access rather than when macros are loaded,
because the loader mutates a process-wide module and a worker process can
serve tasks from more than one team.
@o-nikolas
o-nikolas force-pushed the onikolas/multi-team-plugins/macro-scoping branch from e8ff1c6 to 5d30dfc Compare September 21, 2026 21:51
@o-nikolas
o-nikolas merged commit 63b615b into apache:main Sep 22, 2026
156 checks passed
@o-nikolas
o-nikolas deleted the onikolas/multi-team-plugins/macro-scoping branch September 22, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:task-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants