Skip to content

Migrate execution API reads to async sessions - #73403

Merged
ashb merged 5 commits into
apache:mainfrom
subhramit:api-async-migrate
Sep 30, 2026
Merged

ashb merged 5 commits into
apache:mainfrom
subhramit:api-async-migrate

Conversation

@subhramit

@subhramit subhramit commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Migrate a small group of Execution API read endpoints from sync DB access to AsyncSessionDep.

This updates direct ORM read handlers that can safely switch to async without pulling in sync-only helpers, including:

Each converted handler does a single small read plus Pydantic validation, so the CPU-bound work that moves onto the event loop is negligible and accepted for this batch.

Tests

Ran

uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/execution_api/ -xvs

results:

=============== 597 passed, 4 skipped, 2 warnings in 1157.70s (0:19:17) ===============

(warnings were unrelated JWT reissue middleware failed to refresh token: Not enough segments)


Was generative AI tooling used to co-author this PR?
  • Yes

Assisted-by: Zed GPT-5.4 following the guidelines. I used it to navigate code and manually instructed it after going through the above linked issue and PR.

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:task-sdk labels Sep 20, 2026
@subhramit
subhramit force-pushed the api-async-migrate branch 2 times, most recently from d465b8d to 0672157 Compare September 20, 2026 11:41
@subhramit
subhramit marked this pull request as ready for review September 20, 2026 12:29
@subhramit

Copy link
Copy Markdown
Contributor Author

Related - #73407 so removed the overlap.

Comment thread airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_assets.py Outdated

@Dev-iL Dev-iL left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking up this batch of endpoints! Please forgive my delay in reviewing this.

The route selection is well judged: every converted handler does direct ORM reads with no sync helper, provide_session or commit on its path, which is exactly the kind of route that can switch cleanly. The conversions themselves are careful. Buffered results are consumed correctly ((await session.scalars(...)).one(), .first(), .mappings()), NoResultFound still maps to 404, or 0 keeps its meaning, and nothing touches a lazy attribute after the await (TI.dag_run is lazy="joined", and the response models read only columns).

I confirmed the rebased branch passes on SQLite, PostgreSQL and MySQL. The two comments below are both about test code, mostly because main moved on after the branch was cut.


Regarding the PR description itself

  • once the fixture is gone, the following sentence no longer applies:

    The tests for these routes now opt into async engine reconfiguration so the per-test FastAPI event loop uses a matching async DB engine.

  • The migration guidance also asks for backend, driver, command and result on SQLite, PostgreSQL and MySQL; the listed run is a host uv run on the default SQLite backend. A breeze run --backend <backend> pass over the touched test classes should be quick; they were green on all three backends for me.

  • Finally, one line on the CPU-bound work that moves onto the event loop would close the last gap. Each converted handler does a single small read plus Pydantic validation, so "negligible, accepted" is enough.


P.S. I've been updating the skill in #73405 in light of recent feedback and changes on main. You might find it useful for future migrations.

Comment thread airflow-core/tests/unit/api_fastapi/execution_api/conftest.py
Comment thread airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_assets.py Outdated
Signed-off-by: subhramit <subhramit.bb@live.in>
Signed-off-by: subhramit <subhramit.bb@live.in>
Signed-off-by: subhramit <subhramit.bb@live.in>
@subhramit
subhramit requested a review from Dev-iL September 30, 2026 19:21
@subhramit

subhramit commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor Author

@Dev-iL Thanks for the review. have addressed your comments and updated the PR description.

@ashb
ashb merged commit 489316f into apache:main Sep 30, 2026
80 checks passed
@ashb ashb added this to the Airflow 3.4.0 milestone Sep 30, 2026
@subhramit
subhramit deleted the api-async-migrate branch September 30, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:task-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants