Repository navigation
Fix backup schedule timeout and scheduling state - #818
Merged
Merged
Conversation
Use persisted status and Job annotations as the scheduling cursor, and measure timeouts from the Job start time. Delete timed-out Jobs with foreground propagation to prevent overlapping replacements. Signed-off-by: Matt Lord <mattalord@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR adjusts VitessBackupSchedule scheduling and timeout behavior to use durable state (persisted status and Job annotations) as the scheduling cursor, and to measure job timeouts from the Kubernetes Job start time. It also changes timed-out Job deletion semantics to use foreground propagation to prevent overlapping replacement jobs.
Changes:
- Use
vbsc.Status.LastScheduledTimes(and repair it from Job annotations when newer) as the scheduling cursor. - Measure timeout from
job.Status.StartTimeand delete timed-out Jobs with foreground propagation. - Add regression tests for scheduling cursor persistence/repair and timeout semantics; update API docs to reflect the new timeout definition.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/controller/vitessbackupschedule/vitessbackupschedule_controller.go | Implements status/annotation cursor repair and switches timeout calculation + deletion propagation for timed-out Jobs. |
| pkg/controller/vitessbackupschedule/vtctldclient_backup_test.go | Updates existing timeout test to set Job Status.StartTime and pass now into removeTimeoutJobs. |
| pkg/controller/vitessbackupschedule/scheduling_regression_test.go | Adds regression coverage for status-as-cursor behavior and startTime-based timeouts (including propagation policy). |
| pkg/apis/planetscale/v2/vitessbackupschedule_types.go | Updates API field documentation for JobTimeoutMinutes to specify startTime-based measurement. |
| docs/api/index.html | Regenerates/updates rendered API docs for the new timeout definition. |
| docs/api.md | Regenerates/updates markdown API docs for the new timeout definition. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Signed-off-by: Matt Lord <mattalord@gmail.com>
maxenglander
approved these changes
Aug 28, 2026
mhamza15
approved these changes
Aug 28, 2026
mattlord
added a commit
that referenced
this pull request
Sep 2, 2026
Backport of #818 to release-2.17. Signed-off-by: Matt Lord <mattalord@gmail.com>
mattlord
added a commit
that referenced
this pull request
Sep 2, 2026
[release-2.17] Fix backup schedule timeout and scheduling state (#818)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #817
Fix backup-schedule create/delete loops caused by stale cron timestamps and lost Job history. The controller now uses persisted per-strategy status as its scheduling cursor and measures timeouts from the actual Job start time.
This prevents completed or timed-out Jobs from resetting scheduling state, including when
successfulJobsHistoryLimit: 0or the operator catches up a missed cron slot after downtime.Changes
status.lastScheduledTimesas the durable scheduling cursor.jobTimeoutMinutefromJob.status.startTime; Jobs that have not started cannot time out.jobTimeoutMinute: -1to disable timeout cleanup. The default remains 10 minutes.concurrencyPolicy: Forbid, a terminating Job remains active until Kubernetes removes its Pods, blocking a replacement.VitessBackupSchedulestatus only when it changes.