Skip to content

Fix email callback bundle_version for unpinned Dag runs - #72370

Closed
ayanhussain81 wants to merge 2 commits into
apache:mainfrom
ayanhussain81:fix-email-bundle-version-unpinned-run
Closed

ayanhussain81 wants to merge 2 commits into
apache:mainfrom
ayanhussain81:fix-email-bundle-version-unpinned-run

Conversation

@ayanhussain81

Copy link
Copy Markdown

PR #66485 fixed scheduler-emitted TaskCallbackRequests (external kill, heartbeat
timeout, stuck-in-queued) to source bundle_version from dag_run.bundle_version
instead of DagVersion.bundle_version, so that callbacks for Dags with
disable_bundle_versioning=True stay unpinned and run against the same on-disk
code the task did, instead of pinning to a version the run was never pinned to.

That fix touched three call sites but missed a fourth: the EmailRequest built a
few lines below the TaskCallbackRequest in process_executor_events's
external-kill path. It still falls back to DagVersion.bundle_version whenever
dag_version is set, regardless of whether dag_run.bundle_version is None.

For a Dag with disable_bundle_versioning=True and email_on_failure/
email_on_retry configured, when a task is detected as externally killed, the
failure/retry email ends up pinned to a stale bundle version the run was never
pinned to — causing the Dag Processor to check out an unnecessary
versions/<sha>/ working tree for that email callback (the exact class of
problem #66485 set out to fix, just through the one path it didn't touch).

This applies the same guard used at the other three call sites (and used by the
_resolve_ti_callback_bundle_info helper, whose own docstring says
process_executor_events "inlines the same resolution" — this brings it back in
sync), and adds a regression test mirroring the existing
test_external_kill_callback_bundle_version_follows_dag_run for the EmailRequest
path, since the existing TestSchedulerCallbackBundleInfoDagVersionNullable suite
verifies a reimplementation of the logic rather than the real per-callsite code,
which is how this one diverged unnoticed.

Verified locally: the new test fails against the pre-fix code
(AssertionError: -'abc123-sha' +None) and passes with the fix; the full
bundle_version / process_executor_events / heartbeat-timeout test surface in
test_scheduler_job.py passes with no regressions; ruff check and
ruff format --check are clean on both changed files.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Claude Code was used to investigate the root cause (via git blame/git log on the
original fix, and tracing EmailRequest.bundle_version through to
BundleVersionLock), implement the fix, write the regression test, and run/verify
the test suite. All findings and the diff were reviewed and understood before
submission.


  • Read the Pull Request Guidelines.
  • No related issue — this was found via source/history review rather than a bug report.
  • Newsfragment ({pr_number}.bugfix.rst) will be added as a follow-up commit once this PR's number is known.

PR apache#66485 fixed scheduler-emitted TaskCallbackRequests (external kill,
heartbeat timeout, stuck-in-queued) to source bundle_version from
dag_run.bundle_version instead of DagVersion.bundle_version, so callbacks
for Dags with disable_bundle_versioning=True stay unpinned and run
against the same code the task did.

The EmailRequest built a few lines below the TaskCallbackRequest in
process_executor_events' external-kill path was not covered by that fix
and still falls back to DagVersion.bundle_version whenever dag_version is
set, regardless of whether dag_run.bundle_version is None. For an
unpinned run, this pins the failure/retry email to a stale bundle
version the run was never pinned to, causing the Dag Processor to check
out an unnecessary versions/<sha>/ working tree for that email callback.

Apply the same guard used at the other three call sites, and add a
regression test mirroring test_external_kill_callback_bundle_version_follows_dag_run
for the EmailRequest path.
@boring-cyborg

boring-cyborg Bot commented Sep 1, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@bbovenzi

bbovenzi commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Closing in favor of #71425 & #73915

@bbovenzi bbovenzi closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants