Resume runs after rate-limit / provider 5xx like after server restart #23399

Summary

After a server restart, T3 resumes interrupted runs on its own (restart continuation). After a rate limit (HTTP 429, usage_limit, quota exhaustion) or a transient provider error (5xx, overloaded, stream closed), it doesn't: the run ends failed or interrupted, and the thread stays idle until a human sends a message.

Please treat these failures like a restart: resume the run once after a backoff, at the provider's reset time when the error carries one.

Evidence

Data from four self-hosted T3 servers over three days (2026-10-19 to 2026-10-22). "Never resumed" means no later run on that thread.

Most of the limit runs were delegated child threads (mostly Codex 429s). Each one that wasn't resumed left its parent waiting on a result that never arrived.

Proposed behaviour

• When a run fails with a retryable error (failure.retryable, a 429/5xx class or a resetAt), schedule one continuation using the same mechanism as restart continuation.
• Wait until resetAt (plus a small margin) if present. Otherwise use a backoff, e.g. 15 min for limits and 3 min for 5xx.
• Never resume a run a human stopped, a thread a human settled or archived, or a thread with a newer human message.
• Make it idempotent per failed run, and show "resuming at HH:MM" in the thread.

Related asks from the same analysis

1. Keep a completion wake when continuing a child with thread_send. Continuing a delegated child via t3_thread_send, instead of a new delegate_task, registers no completion wake, so the parent sleeps after the child finishes. This happened in 7 of 113 delegating sessions we sampled.
2. Show native background work as pending. A Claude native background subagent or Monitor in a delegated child isn't counted as pending work. The parent sees the child as idle, even though the child will wake itself.
3. Give ContextHandoffBudgetError a structured code. "Insufficient context allowance for the provider handoff" is only available as message text (22 failed runs). A code would let tooling detect it and compact or retry it.

Related: #22026, #16815 and #14169 (usage-limit auto-resume), and #17503 (a Codex 429 retry hang). This issue asks for the general case: limit and 5xx failures resumed like a restart.

We are running a local workaround meanwhile: a timer reads the state DB and sends one resume message per failed run. A built-in path would make it unnecessary.