Repository navigation
Fix DeadlockImminentError when a connection is resolved inside an async task - #71890
Merged
Merged
Conversation
2 tasks done
2 tasks done
dabla
force-pushed
the
fix/deadlock-mask-secret-get-uri
branch
from
August 20, 2026 15:12
99c8738 to
db8a1c7
Compare
…rror in async tasks get_uri() accesses extra_dejson, which calls the synchronous mask_secret() → comms.send() from within the event-loop thread. Any async hook or task that calls aget_hook() / aget_connection() triggers this path, and Airflow 3.3.1's DeadlockImminentError detection surfaces the bug. Add aextra_dejson() — an async method that awaits amask_secret() instead of the blocking mask_secret() — and aget_uri(), which delegates URI assembly to a new shared _build_uri() helper and calls await self.aextra_dejson(). This keeps the entire connection-serialisation path safely on the async stack. The sync get_uri() / extra_dejson are unchanged; _build_uri() is the single source of truth for the URI format, shared by both paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dabla
force-pushed
the
fix/deadlock-mask-secret-get-uri
branch
from
August 20, 2026 19:29
db8a1c7 to
9663c6a
Compare
1 task done
amoghrajesh
reviewed
Sep 10, 2026
amoghrajesh
left a comment
Contributor
There was a problem hiding this comment.
Directionally fine, I have a few qns / todos here.
- Can we also update the doc: https://airflow.apache.org/docs/task-sdk/stable/deferred-vs-async-operators.html? The example mentions
hook = KiotaRequestAdapterHook.get_hook(conn_id="msgraph_default") - Should we be also covering the other methods like
to_dict,get_extra_dejsonhere?
kaxil
reviewed
Sep 10, 2026
kaxil
approved these changes
Sep 10, 2026
Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
Contributor
Author
Don't know if we still want to implement an async version of |
1 task done
vatsrahul1001
added a commit
that referenced
this pull request
Sep 10, 2026
…nc task (#71890) (#72895) * Add aget_uri and aextra_dejson to Connection to fix DeadlockImminentError in async tasks get_uri() accesses extra_dejson, which calls the synchronous mask_secret() → comms.send() from within the event-loop thread. Any async hook or task that calls aget_hook() / aget_connection() triggers this path, and Airflow 3.3.1's DeadlockImminentError detection surfaces the bug. Add aextra_dejson() — an async method that awaits amask_secret() instead of the blocking mask_secret() — and aget_uri(), which delegates URI assembly to a new shared _build_uri() helper and calls await self.aextra_dejson(). This keeps the entire connection-serialisation path safely on the async stack. The sync get_uri() / extra_dejson are unchanged; _build_uri() is the single source of truth for the URI format, shared by both paths. (cherry picked from commit d057784) Co-authored-by: David Blain <info@dabla.be> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
imrichardwu
pushed a commit
to imrichardwu/airflow
that referenced
this pull request
Sep 11, 2026
…nc task (apache#71890) * Add aget_uri and aextra_dejson to Connection to fix DeadlockImminentError in async tasks get_uri() accesses extra_dejson, which calls the synchronous mask_secret() → comms.send() from within the event-loop thread. Any async hook or task that calls aget_hook() / aget_connection() triggers this path, and Airflow 3.3.1's DeadlockImminentError detection surfaces the bug. Add aextra_dejson() — an async method that awaits amask_secret() instead of the blocking mask_secret() — and aget_uri(), which delegates URI assembly to a new shared _build_uri() helper and calls await self.aextra_dejson(). This keeps the entire connection-serialisation path safely on the async stack. The sync get_uri() / extra_dejson are unchanged; _build_uri() is the single source of truth for the URI format, shared by both paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
xvega
pushed a commit
to xvega/airflow
that referenced
this pull request
Sep 13, 2026
…nc task (apache#71890) * Add aget_uri and aextra_dejson to Connection to fix DeadlockImminentError in async tasks get_uri() accesses extra_dejson, which calls the synchronous mask_secret() → comms.send() from within the event-loop thread. Any async hook or task that calls aget_hook() / aget_connection() triggers this path, and Airflow 3.3.1's DeadlockImminentError detection surfaces the bug. Add aextra_dejson() — an async method that awaits amask_secret() instead of the blocking mask_secret() — and aget_uri(), which delegates URI assembly to a new shared _build_uri() helper and calls await self.aextra_dejson(). This keeps the entire connection-serialisation path safely on the async stack. The sync get_uri() / extra_dejson are unchanged; _build_uri() is the single source of truth for the URI format, shared by both paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dabla
added a commit
to dabla/airflow
that referenced
this pull request
Oct 3, 2026
get_conn() resolves the connection with get_connection() and reads extra_dejson, whose secret masking sends to the supervisor synchronously; inside an async task with another async SDK call in flight both raise DeadlockImminentError. aget_conn() returns an AsyncOpenAI client configured exactly like get_conn()'s, from get_async_connection() with the extra from Connection.aextra_dejson() (apache#71890, Airflow 3.3.2+), or the raw extra via json.loads on older versions (as in apache#72130). The client settings move into _client_kwargs(), shared by both paths. acreate_embeddings() is the async counterpart of create_embeddings(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dabla
added a commit
to dabla/airflow
that referenced
this pull request
Oct 3, 2026
get_conn() resolves the hook's connection and every fallback connection with get_connection() and reads extra_dejson, whose secret masking sends to the supervisor synchronously; inside an async task with another async SDK call in flight both raise DeadlockImminentError, although the agent the hook builds is async-native. aget_conn() fetches the connection and its fallbacks with get_async_connection() (extra from Connection.aextra_dejson(), apache#71890, Airflow 3.3.2+, or the raw extra via json.loads on older versions as in apache#72130), seeds the existing connection cache and builds the model with the unchanged get_conn(); acreate_agent() does the same before create_agent(), spec_file included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 task done
dabla
added a commit
to dabla/airflow
that referenced
this pull request
Oct 3, 2026
get_conn() resolves the connection with get_connection() and reads extra_dejson, whose secret masking sends to the supervisor synchronously; inside an async task with another async SDK call in flight both raise DeadlockImminentError. aget_conn() returns an AsyncOpenAI client configured exactly like get_conn()'s, from get_async_connection() with the extra from Connection.aextra_dejson() (apache#71890, Airflow 3.3.2+), or the raw extra via json.loads on older versions (as in apache#72130). The client settings move into _client_kwargs(), shared by both paths. acreate_embeddings() is the async counterpart of create_embeddings(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dabla
added a commit
to dabla/airflow
that referenced
this pull request
Oct 3, 2026
get_conn() resolves the hook's connection and every fallback connection with get_connection() and reads extra_dejson, whose secret masking sends to the supervisor synchronously; inside an async task with another async SDK call in flight both raise DeadlockImminentError, although the agent the hook builds is async-native. aget_conn() fetches the connection and its fallbacks with get_async_connection() (extra from Connection.aextra_dejson(), apache#71890, Airflow 3.3.2+, or the raw extra via json.loads on older versions as in apache#72130), seeds the existing connection cache and builds the model with the unchanged get_conn(); acreate_agent() does the same before create_agent(), spec_file included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 3, 2026
dabla
added a commit
to dabla/airflow
that referenced
this pull request
Oct 3, 2026
get_conn() resolves the hook's connection and every fallback connection with get_connection() and reads extra_dejson, whose secret masking sends to the supervisor synchronously; inside an async task with another async SDK call in flight both raise DeadlockImminentError, although the agent the hook builds is async-native. aget_conn() fetches the connection and its fallbacks with get_async_connection() (extra from Connection.aextra_dejson(), apache#71890, Airflow 3.3.2+, or the raw extra via json.loads on older versions as in apache#72130), seeds the existing connection cache and builds the model with the unchanged get_conn(); acreate_agent() does the same before create_agent(), spec_file included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dabla
added a commit
that referenced
this pull request
Oct 3, 2026
…74147) Connection.extra_dejson masks the extra's secrets with a synchronous call to the supervisor, which raises DeadlockImminentError when a hook reads it on an event loop with another async call in flight. Airflow 3.3.2+ has Connection.aextra_dejson() (#71890), but providers that still support older versions cannot call it directly, so async hooks fall back to json.loads(conn.extra) and skip the masking (#72130). get_async_extra_dejson(conn) awaits Connection.aextra_dejson() when it exists, and otherwise runs extra_dejson in a worker thread, where blocking on the supervisor is safe, as get_async_connection() does for get_connection(). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes a
DeadlockImminentErrorraised in Airflow 3.3.1 when a connection is resolved inside an async task viaawait BaseHook.aget_hook(...)orawait BaseHook.aget_connection(...).Problem
The call chain when fetching a connection inside an async task is:
The error message says "use the async equivalents" — but the caller already is.
The bug is inside
Connectionitself:get_uri()andextra_dejsonhave no async counterparts, so secret masking always goes through the blockingcomms.send().amask_secret()(which usesasend()) already existed inairflow.sdk.logbut nothing in
Connectioncalled it.Changes
task-sdk/src/airflow/sdk/definitions/connection.py_build_uri(self, extra_dejson: dict)helper so sync and async paths share a single implementation.get_uri()toreturn self._build_uri(self.extra_dejson)— behaviour is identical to before.aextra_dejson()— async method that mirrorsextra_dejsonbut awaitsamask_secret()instead of calling the blockingmask_secret().aget_uri()— async method that callsawait self.aextra_dejson()and delegates to_build_uri().task-sdk/tests/task_sdk/definitions/test_connection.pytest_aget_uri—aget_uri()produces the same URI asget_uri()and delegates toaextra_dejson.test_aextra_dejson_calls_amask_secret—aextra_dejson()usesamask_secret(async) and never calls the syncmask_secret.test_aextra_dejson_no_extra—aextra_dejson()returns{}without callingamask_secretwhenextraisNone.Usage
Was generative AI tooling used to co-author this PR?
Claude Sonnet 4.6 (GitHub Copilot)
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.