Skip to content

feat(server): continue a delegated child its provider cut off before failing the task - #1

Open
jagadhis wants to merge 3 commits into
feat/sidebar-session-groupsfrom
feat/delegated-child-provider-resume
Open

jagadhis wants to merge 3 commits into
feat/sidebar-session-groupsfrom
feat/delegated-child-provider-resume

Conversation

@jagadhis

@jagadhis jagadhis commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Problem

When a provider cut a delegated child's run short with an error that clears up by itself (overloaded, rate limited, dropped connection, 5xx), T3 marked the parent's delegated task failed and woke the parent right away. The parent then had to notice the failure and start the work again itself.

Solution

  • Verdict. providerFailureVerdict is a rule-for-rule port of Cockpit's providerVerdict (jagadhis/cockpit Guard git quick actions for diverged or stale branch status pingdotgg/t3code#47/Update cached branch list immediately after checkout pingdotgg/t3code#48) and uses the same categories. These pass: Provider overloaded, Model at capacity, Rate limited, Connection dropped, Provider server error. These do not, and fail the task at once: Signed out, Out of allowance (T3's usage-limit recovery handles that case), Context window full, Model not available, Provider refused it, and any error it does not recognise.
  • Hold and continue. When a delegated child's run fails with a passing error, a provider-runtime.retry outbox effect is written. It becomes due 1, 2, 4 and 8 minutes after each cut in a row. While the effect is pending or running, the task status and the parent's wake are held by the same path that holds them across a T3 restart (childAwaitsRestartContinuation). When the wait ends, the child is sent a carry-on message as a new run. That run keeps the same delegatedTaskId. If the continued run succeeds, the parent gets a normal completion.
  • Release. The hold is released, and the task is marked failed with the provider's own error text, in any of these cases: the error is lasting, a fifth cut comes in a row, the thread is archived or deleted, another run took over, or the task is stopped. thread.stop and task_cancel both end a pending wait at once.
  • No repeated outward steps. If the project's git root has .agents/scripts/resume-guard.ts (as ~/agents does), the worker runs resume-guard.ts arm before the carry-on, the same way Cockpit does. It points the script at this server's own state database. The steps arm returns are listed in the carry-on message, and the workspace hook blocks an exact repeat. If arm fails, the turn is not continued and the task fails as it is. When the task settles (success, final failure, Stop or task_cancel), a provider-runtime.retry-release effect runs resume-guard.ts disarm with the same root and state database, so later turns in that thread are not blocked. A failed disarm is logged and does not fail the task. A workspace without the script still gets a carry-on message that tells the child to check what it already did before repeating any comment, review, PR, Linear or Slack write, or thread launch.

Size of the change

Files Lines
Production 10 +782 / −5
Tests 2 +482 / −2
Mocks and fixtures 0 +0 / −0

Anything to be aware of

Warning

  • Takes effect only after a T3 rebuild and restart. Restarting ends live sessions, so Jags should choose when to do it.
  • Stacked on feat/sidebar-session-groups, the branch the live T3 checkout runs. The fork's main is far behind upstream.

Note

  • Tested with unit tests on the verdict, and with orchestrator tests on hold, continue and release. Those tests cover: a success after a retry reaching the parent once, a lasting error failing at once, tries running out and failing with the provider's reason, Stop ending the wait, arm both listing steps and failing, and a disarm being written on every settle after a retry (none after a lasting error), with a failed disarm only logged. The tests write the failure events directly. No real provider overload has gone through a built T3 yet.
  • resume-guard.ts arm and disarm have not been run from T3 against a real thread; the tests use stubs.
  • When tries run out, the parent is told the provider's error text. It does not get Cockpit's "still after N tries" wording.

Ada, from Jags's crew

Port of Cockpit's providerVerdict, kept rule for rule: overloaded, rate
limited, a dropped connection and a 5xx pass; signed out, out of
allowance, a refused request and anything unrecognised do not. Adds the
1, 2, 4 and 8 minute waits and the carry-on message that lists the
outward steps a cut-off turn already took.
…failing the task

A delegated child whose run failed on a passing provider error (overload,
dropped connection, rate limit, 5xx) used to fail its task and wake the
parent at once. Now the task and the parent's wake are held, as they are
for a restart continuation, and a provider-runtime.retry effect continues
the child with the same delegatedTaskId after 1, 2, 4 and 8 minutes. A
lasting or unrecognised error, a fifth cut in a row, or Stop releases the
hold and fails the task with the provider's reason; a later success
reaches the parent as a normal completion.

Before continuing, the worker runs the workspace's
.agents/scripts/resume-guard.ts arm when the project has one, so the
continued turn is told which outward steps it already completed and the
workspace hook blocks exact repeats. If arm fails the turn is not
continued and the task fails as it is.
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL labels Nov 3, 2026
…ettles

The resume-guard marker armed before a provider-retry carry-on stayed
until its 24-hour TTL, so a later turn in that thread could not repeat
an outward step it was asked to. When a task whose child a provider
retry touched settles (success, final failure, Stop or task_cancel),
the orchestrator now writes a provider-runtime.retry-release effect,
and the worker runs resume-guard.ts disarm with the same root and
T3_STATE_PATH as arm. A failed disarm is logged and never fails the
task.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown

Thread transfer impact

⚠️ The latest CI run did not produce a thread transfer result for b412005.

This comment will update automatically after the next completed run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant