Add rate_limiting_active field to TaskQueueStats#818
Conversation
When a task queue's whole-queue rate limit is blocking dispatches, auto-scaling systems should suppress scale-up since more workers cannot increase throughput. This boolean surfaces that state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| // worker instance. | ||
| float tasks_dispatch_rate = 4; | ||
|
|
||
| // Whether the task queue's whole-queue rate limit is currently blocking dispatches. When true, adding more |
There was a problem hiding this comment.
What are the semantics of "currently" here? Will this flicker on and off if worker throughput is nearly equal to the rate limit? I think adding a bit more detail to the doc would be useful here.
And, internally, if that is possible, it might make sense to apply some smoothing/debounce or something so scalers don't oscillate
There was a problem hiding this comment.
(updated comment) This is instantaneous value, and can flicker. We expect the consumer to handle any smoothing. I didn't want to add complexity right now to server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| // This reflects instantaneous state at the time of the query. If worker throughput is close to the rate | ||
| // limit, this value may alternate between true and false across successive queries. Consumers should | ||
| // apply their own smoothing or hysteresis to avoid oscillating scaling decisions. |
There was a problem hiding this comment.
wondering if this is the right semantics or more of a sliding window (say 30s to be consistent with task add/dispatch rate fields)?
Also, what instantaneous mean? does it mean that at this moment a task is blocked behind the limiter.allow?
There was a problem hiding this comment.
if the concern is complexity in server, we already have sliding window taskTracker used for add/dispatch rates. so it shouldn't be much work.
The flag represents whether rate limiting is blocking dispatches, without distinguishing which limiter is responsible. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3 participants
What
Adds
bool rate_limiting_activetoTaskQueueStatsproto message.Why
Auto-scaling systems need to know when backlog growth is caused by rate limiting rather than insufficient worker capacity. When the whole-queue rate limit is blocking dispatches, scaling up workers doesn't help — the bottleneck is the rate limit, not worker count.
Server PR
🤖 Generated with Claude Code