Repository navigation
Feature: Priority Queue #2374
Description
Tasks
- v0.20
- v0.21
- 🌱 Add priority label to PQ depth metric #3156
- 🐛Fix behavior of rate limit option in priorityqueue.AddWithOpts #3103
- 🌱 Add debug logging for the state of the priority queue #3075
- ✨ Leverage isInInitialList of ResourceEventHandler.OnAdd, Add IsInInitialList to TypedCreateEvent #3162
- 🌱 Remove redundant WithLowPriorityWhenUnchanged in builder #3168
- ✨ Controller: Retain the priority #3167
- 🐛Implement priorityqueue as default on handlers if using priorityqueue interface #3111
- 🌱 Handlers: Use low priority when object is unchanged and priority queue #3152
- 🌱 Followups to default low priority in mappers #3160
- 🌱 TypedEnqueueRequestForOwner: Decrease priority when unchanged #3174
- v0.22
- ⚠️ Priorityqueue: Make priority opt a pointer #3289
- ✨ WithLowPriorityWhenUnchanged: Set Priority for all add methods #3290
- 🐛 Priorityqueue: Shutdown on shutdown #3250
- 🐛 Priorityqueue: Don't block on Get when queue is shutdown #3243
- [release-0.22] 🐛 priority queue: properly sync the waiter manipulation #3371
- v0.23 (enabled per default)
- 🐛 Don't block on Get when queue is shutdown (2nd try) #3337
- ✨ Add optional Priority field to reconcile.Result #3333
- ✨ Priorityqueue: Enable per default #3332
- 🐛 Fix a bug where the priorityqueue would sometimes not return high-priority items first #3330
- 🌱 Prioriyqueue tests: Add and use newQueueWithTimeForwarder #3336
- 🌱 Priorityqueue tests: Use synctest #3350
- 🐛 priority queue: properly sync the
waitermanipulation #3368 - 🌱 Add some more tests to the priorityqueue #3387
- ✨ Fix unused priorityqueue test: TestWhenAddingMultipleItemsWithRatelimitTrueTheyDontAffectEachOther #3395
- 🐛 Priorityqueue: Do FIFO ordering within priorities and not across #3408
- ✨ Use a buffer to optimize priority queue AddWithOpts performance #3415
- 🌱 Priorityqueue: Use separate b-trees for ready and non-ready items #3416
- 🐛 Limit depthWithPriorityMetric cardinality to 25 #3419
- v0.24
- v0.25
- v0.26
Ideas:
- Add Priority parameter to reconcile.Request (xref: 📖 Add a design for a priority queue #3013 (comment))
- Won't implement until we have a clear use case for it
Feature enablement
The feature can be enabled by setting the UsePriorityQueue option to true, e.g.:
ctrlOptions := ctrl.Options{
Controller: config.Controller{
UsePriorityQueue: ptr.To[bool](true),
},
...
}
mgr, err := ctrl.NewManager(restConfig, ctrlOptions)Original issue description
While discussing #857 (comment) an idea come up about having something like a priority queue, so we can give resync events a lower priority while assigning to other events a higher priority.
This could help when there are many objects of the same type and at every resync period there is a storm of events being added to the queue.
Reacted by Stefan Büringer, Jessica Fiedler, Casey Buto, Nikolai Osipov, Benjamin Somhegyi, Weiwei Tang, yy, Nathan Perkins and Archandan
Reacted by Alvaro Aleman, Nathan Perkins, Tsuzuki Tsuchiya and merusso
Activity
alvaroaleman commentedon Jun 12, 2023
While resync events are one example, I think that it makes sense in general to allow specifying a priority when adding items to the workqueue.
One challenge here is the fact that as of today, there is no priority queue implementation in upstream kube.
troy0820 commentedon Aug 25, 2023
/kind feature
k8s-triage-robot commentedon Jan 27, 2024
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/remove-lifecycle stale/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
k8s-triage-robot commentedon Feb 26, 2024
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/remove-lifecycle rotten/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
k8s-triage-robot commentedon Mar 27, 2024
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/reopen/remove-lifecycle rottenPlease send feedback to sig-contributor-experience at kubernetes/community.
/close not-planned
k8s-ci-robot commentedon Mar 27, 2024
@k8s-triage-robot: Closing this issue, marking it as "Not Planned".
Details
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
sbueringer commentedon Mar 28, 2024
/remove-lifecycle rotten
/reopen
k8s-ci-robot commentedon Mar 28, 2024
@sbueringer: Reopened this issue.
Details
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
3 remaining items
alvaroaleman commentedon Nov 16, 2024
/assign
alvaroaleman commentedon Nov 17, 2024
All,
I've written a small design doc and POC for this. Would appreciate everyones feedback
sbueringer commentedon Jan 2, 2025
I've updated the issue so we can use it as an umbrella issue for the feature going forward
[-]Implement priority queue as an option for reconcile loops[/-][+]Feature: Priority Queue[/+]lingdie commentedon Apr 8, 2025
Can we provide more example codes on priority queue feature?
sbueringer commentedon Apr 8, 2025
What example code are you looking for? To use the feature you only have to set a boolean field to true
lingdie commentedon Apr 8, 2025
Is that all? Can't believe. :)
sbueringer commentedon Apr 8, 2025
Yup. Literally just that. We try to implement this in a way that it just works.
(I added a short config snippet to the issue description)
vishalbalaji-v commentedon Aug 13, 2025
Hi everyone,
Is there timeline on when this will be generally available? The doc for priority queue says it's currently in beta (link).
We have a use case for it, but people are worried about putting it in production since the doc says it's in beta. I'd be happy to contribute to something if required. The design also doesn't seem to have been updated in a while.
alvaroaleman commentedon Aug 13, 2025
The main reason for that is that we don't want to call it GA until we are sure we don't want to change the interface definition. The priority queue is being used in production in places where it matters and there are no known issues with it.
sbueringer commentedon Oct 6, 2025
And PQ is enabled per default 🎉
Thx to everyone that is contributing to this feature!
fabriziopandini commentedon Nov 7, 2025
Awesome job folks!