Skip to content

Fix manual runs getting a zero-length data interval across DST - #70089

Merged
Lee-W merged 5 commits into
apache:mainfrom
harjothkhara:fix-cron-get-prev-dst
Aug 12, 2026
Merged

Lee-W merged 5 commits into
apache:mainfrom
harjothkhara:fix-cron-get-prev-dst

Conversation

@harjothkhara

@harjothkhara harjothkhara commented Jul 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

On the two days a year the clocks change, a manually triggered run can get a data interval that is zero length, or that ends after the run time.

With CronDataIntervalTimetable("0 2 * * *", timezone="America/New_York"), a manual run at 2024-03-10 07:30Z:

manual    : 2024-03-10 07:00Z -> 2024-03-10 07:00Z   (zero length)
scheduled : 2024-03-09 07:00Z -> 2024-03-10 07:00Z   (correct)

croniter steps back on naive wall-clock time, but make_aware moves a tick that falls in the spring-forward gap onto the next real instant. When the previous tick is in the gap it lands on the current time, so _get_prev returns the time it was asked about.

That zero-length interval then costs a run. next_dagrun_info reads it as a point-in-time run and skips a period (#66132, correct for the case it was written for):

after the bad interval : 2024-03-11 06:00Z -> 2024-03-12 06:00Z
if the interval is ok  : 2024-03-10 07:00Z -> 2024-03-11 06:00Z

The fall-back side overshoots instead. With 0 1 * * * in America/New_York at 2024-11-03 05:30Z, _get_prev returns 06:00Z — half an hour in the future — so CronTriggerTimetable with run_immediately=True fires a tick that has not happened yet.

Also reproduces in Europe/Zurich, Australia/Sydney and Australia/Lord_Howe.

What

_get_prev now keeps stepping cron.get_prev until the result is strictly earlier than the time asked about. get_prev is strictly decreasing on naive time, so it terminates.

The _covers_every_hour branch is untouched — it uses a wall-clock delta rather than make_aware, so it cannot hit this.

Only values that were already wrong change. Nothing outside a DST transition window is affected.

Testing

Three regression tests, all failing before and passing after:

  • test_cron_infer_manual_data_interval_dst — gap and fall-back, including Australia/Lord_Howe for its 30-minute offset
  • test_cron_next_dagrun_info_after_dst_manual_run — the skipped period
  • test_cron_trigger_run_immediately_does_not_pick_future_run — CronTriggerTimetable, which shares the mixin

airflow-core/tests/unit/timetables/ is 246 passed.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Opus 4.8 following the guidelines

…lier

croniter steps back on naive wall-clock time, but make_aware can map a tick
inside a DST transition forward onto the current instant or later. _get_prev
then returned a "previous" tick that was not strictly earlier, so a manual run
got a zero-length data interval on spring-forward, or one ending after
run_after on fall-back. A zero-length interval also trips the point-in-time
guard in next_dagrun_info and silently skips a scheduled period.
@boring-cyborg

boring-cyborg Bot commented Jul 19, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@harjothkhara
harjothkhara marked this pull request as ready for review July 19, 2026 04:55
_skip_to_latest is the only caller that steps _get_prev twice, so a
regression there would silently reappear as a run proposed for a period
that has not elapsed. Nothing in the branch covered that path.
@harjothkhara
harjothkhara requested a review from uranusjr August 9, 2026 22:22

@Lee-W Lee-W left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one nit improvement. overall looks good

The fix lives in CronMixin._get_prev, shared by MultipleCronTriggerTimetable
and CronPartitionTimetable, so a regression there would hit them too even
though only CronTriggerTimetable had a DST case.
@harjothkhara
harjothkhara requested review from Lee-W and uranusjr August 10, 2026 17:32

@Lee-W Lee-W left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good after parameterization is done. Thanks!

The three timetables share CronMixin._get_prev and were being exercised by
three near-identical test bodies differing only in construction and call
signature.
@harjothkhara
harjothkhara requested a review from Lee-W August 12, 2026 05:06
@Lee-W
Lee-W merged commit 6194df6 into apache:main Aug 12, 2026
78 checks passed
@boring-cyborg

boring-cyborg Bot commented Aug 12, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants