Skip to content

[v3-4-test] Stop the secrets backend lookup from importing the task runner outside tasks (#74641) - #74980

Merged
vatsrahul1001 merged 1 commit into
v3-4-testfrom
backport-f46a370-v3-4-test
Oct 18, 2026
Merged

vatsrahul1001 merged 1 commit into
v3-4-testfrom
backport-f46a370-v3-4-test

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

ensure_secrets_backend_loaded() imported task_runner to check whether
SUPERVISOR_COMMS was set. Outside a task, that loaded the task runtime
just to learn the answer was no. Read the module from sys.modules
instead, as mask_secret already does.

That import also loaded the task runner in the supervisor before it
forked the task process, and before callback processes when remote
logging has a connection, so those children inherited it. Import it
explicitly before a bare fork of a task or callback process instead, so
each child does not import it again. If the import fails there, the
child imports it again and reports the error in its own log, as before.
(cherry picked from commit f46a370)

Co-authored-by: Kaxil Naik kaxilnaik@gmail.com

…unner outside tasks (#74641)

ensure_secrets_backend_loaded() imported task_runner to check whether
SUPERVISOR_COMMS was set. Outside a task, that loaded the task runtime
just to learn the answer was no. Read the module from sys.modules
instead, as mask_secret already does.

That import also loaded the task runner in the supervisor before it
forked the task process, and before callback processes when remote
logging has a connection, so those children inherited it. Import it
explicitly before a bare fork of a task or callback process instead, so
each child does not import it again. If the import fails there, the
child imports it again and reports the error in its own log, as before.
(cherry picked from commit f46a370)

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@vatsrahul1001
vatsrahul1001 marked this pull request as ready for review October 18, 2026 09:03
@vatsrahul1001
vatsrahul1001 merged commit 7669248 into v3-4-test Oct 18, 2026
124 of 135 checks passed
@vatsrahul1001
vatsrahul1001 deleted the backport-f46a370-v3-4-test branch October 18, 2026 10:00
@github-actions github-actions Bot added this to the Airflow 3.4.1 milestone Oct 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor Author

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.4.1 based on: merged to version branch
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

vatsrahul1001 pushed a commit that referenced this pull request Oct 20, 2026
…unner outside tasks (#74641) (#74980)

ensure_secrets_backend_loaded() imported task_runner to check whether
SUPERVISOR_COMMS was set. Outside a task, that loaded the task runtime
just to learn the answer was no. Read the module from sys.modules
instead, as mask_secret already does.

That import also loaded the task runner in the supervisor before it
forked the task process, and before callback processes when remote
logging has a connection, so those children inherited it. Import it
explicitly before a bare fork of a task or callback process instead, so
each child does not import it again. If the import fails there, the
child imports it again and reports the error in its own log, as before.
(cherry picked from commit f46a370)

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants