Skip to content

Avoid loading timetables for dags in Execution API endpoints - #75765

Merged
jason810496 merged 5 commits into
mainfrom
lazy-serialized-dag-timetable
Oct 28, 2026
Merged

jason810496 merged 5 commits into
mainfrom
lazy-serialized-dag-timetable

Conversation

@ashb

@ashb ashb commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

For a few reasons/places now in the Exec API we now need the serialized
dag, but we don't need the timetable (which could involve loading) plugins
or user code.

This should let the ExecAPI run without user code if a deployment admin
wishes to try that.

Spotted by @feluelle


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

For a few reasons/places now in the Exec API we now need the serialized
dag, but we don't need the timetable (which could involve loading) plugins
or user code.

This should let the ExecAPI run without user code if a deployment admin
wishes to try that.
@ashb
ashb requested a review from bolkedebruin as a code owner October 27, 2026 13:53
@ashb ashb added the backport-to-v3-4-test Backport to v3-4-test label Oct 27, 2026
@vatsrahul1001 vatsrahul1001 added this to the Airflow 3.4.0 milestone Oct 27, 2026
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

The re-run of the jobs stopped by EC2 Spot interruptions finished: attempt 3 failed.

`weight_rule` on SerializedDag is already a property that decodes/inflates
strings into the object
@ashb
ashb requested a review from XD-DENG as a code owner October 27, 2026 17:49

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#75842 should solve the CI failure (thanks @amoghrajesh for the info), will rebase this PR once #75842 get merged.

@jason810496
jason810496 enabled auto-merge (squash) October 28, 2026 07:40
@jason810496
jason810496 merged commit 402ab11 into main Oct 28, 2026
130 checks passed
@jason810496
jason810496 deleted the lazy-serialized-dag-timetable branch October 28, 2026 08:02
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-4-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-4-test PR Link

vatsrahul1001 pushed a commit that referenced this pull request Oct 28, 2026
…nts (#75765) (#75846)

* Avoid loading timetables for dags in Execution API endpoints

For a few reasons/places now in the Exec API we now need the serialized
dag, but we don't need the timetable (which could involve loading) plugins
or user code.

This should let the ExecAPI run without user code if a deployment admin
wishes to try that.

* Stop eagerly decoding custom weight_rules

`weight_rule` on SerializedDag is already a property that decodes/inflates
strings into the object

* Fix reviews

* fixup! Avoid loading timetables for dags in Execution API endpoints

---------
(cherry picked from commit 402ab11)

Co-authored-by: Ash Berlin-Taylor <ash@astronomer.io>
Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants