Skip to content

A turn with no path to a terminal event: Pi's settle probe never re-arms, OpenCode drops unresolvable child asks #16846

Closed as not planned

Description

@Adamulek123

Problem

Two paths leave a turn with no route to a terminal event.

  • PiAdapterV2.ts:1919-1927 — a settle probe that succeeds but reports isStreaming, a compact in progress, or a non-zero pendingMessageCount returns without calling finalizeTurn and without calling scheduleSettleProbe. So the probe fires once, decides the work is still busy, and never re-arms. The turn stays open forever. Contrast the probeFailed branch at 1908-1913, which does retry.
  • OpenCodeAdapterV2.ts:1996-2005 — an unresolvable child ask is retried 5 times with min(200 * 2**retry, 2000) backoff, about 5 seconds total, then abandoned. Effect.ensuring only clears the dedupe key; no failure event reaches the UI. The user asked for something and gets silence.

Why these are unambiguous

Unlike the interrupt-timeout and Stop questions, neither of these needs a product decision — both are simply cases where the retry/settle loop fails to re-arm. A reviewer can approve these without choosing a policy.

Impact

Both produce a turn or request that is silently abandoned: no terminal event, no error surfaced. For the Pi probe this is indistinguishable from a genuinely stuck provider.

Environment

Found by static audit of the merged orchestrator-v2 work (#2829). Re-verified present at main. Grouped with A6b because both are "a turn with no path to terminal", which is the same class of defect even though they sit in different adapters.

Not filed as a PR only because the audit grouped seven findings into one bundle; these two are the subset with a single correct answer.

Activity

  1. juliusmarminge commented on Oct 7, 2026

    @juliusmarminge
    Member

    Note

    Grok responding on behalf of Julius.

    Triage against current main (apps/server/src/orchestration-v2/Adapters/).

    OpenCode: child ask dropped silently (confirmed)

    • OpenCodeAdapterV2.ts:2035-2045: if routeRuntimeRequest can't resolve an owner inline, it retries 5 times (min(200 * 2**retry, 2000) ms, about 5 s in total) and then the fiber just ends. Effect.ensuring only removes the id from pendingChildRequestRoutes. There's no log, no emitRuntimeRequest, no reply to OpenCode and no failure event. The child session keeps waiting on a permission or question that the UI never shows.
    • Fix: when the retries run out, either reject or cancel the native request through the SDK and emit a turn failure or warning on the nearest known thread, or keep the request parked and route it once session.created or the task part registers the relation. At minimum, log it.
    • Open PR fix(server): settle a turn when its provider session closes #16181 ("settle a turn when its provider session closes") changes both adapters, but it targets settling turns when a session closes, not this give-up path.

    Pi: settle probe that sees busy state doesn't re-arm (not confirmed as a hang)

    • PiAdapterV2.ts:1918-1928: it's true that a successful probe reporting isStreaming, isCompacting or pendingMessageCount > 0 returns without calling finalizeTurn or scheduleSettleProbe, unlike the probeFailed retry at 1904-1917.
    • However, re-arming is driven by events. Every agent_settled bumps the generation and schedules a new probe (1782-1792), and every compaction_end (completed, failed or aborted) re-probes when settleWhenIdle is set (1651-1697). Pending or streaming work should end in another agent_settled, and a compaction should end in compaction_end. So in the normal case the turn still settles.
    • The stall would only happen if Pi reports busy and then never emits another agent_settled or compaction_end. We haven't confirmed that can happen. A repro or Pi RPC log showing it would help. A bounded re-probe (for example, reusing SETTLE_PROBE_MAX_ATTEMPTS/SETTLE_PROBE_RETRY_DELAY on the busy branch) would be cheap defense in depth.
  2. added
    bugSomething is broken or behaving incorrectly.
    via-triageFiled through npx t3 triage
    on Oct 7, 2026
  3. Adamulek123 commented on Oct 7, 2026

    @Adamulek123
    ContributorAuthor

    Thanks for the triage -- both halves were useful, and one of them corrected me.

    Pi: you are right and my report was wrong. I claimed the settle probe "fires once, decides the work is still busy, and never re-arms. The turn stays open forever." That was an inference from the missing scheduleSettleProbe call in the busy branch at PiAdapterV2.ts:1921-1929, and I did not check whether a second re-arm mechanism existed. It does: agent_settled bumps settleProbeGeneration and re-probes (1782-1792), and compaction_end re-probes (1665/1675/1696). So in the normal case the turn settles, and a stall requires Pi to report busy and then emit neither event. I have not shown that is reachable and neither have you.

    I am reframing that half as needing a repro or an RPC log, with a bounded re-probe on the busy branch as cheap defense in depth, using the existing SETTLE_PROBE_MAX_ATTEMPTS / SETTLE_PROBE_RETRY_DELAY. Not filing it as a confirmed hang.

    OpenCode: confirmed, and I checked it is not already handled. Still present on current main at OpenCodeAdapterV2.ts:2038-2045. I also checked that nothing in flight covers it: none of our open branches touch it, #15778 works at the ProviderRuntimeRecoveryService / ProviderSessionManager layer during recovery and detach rather than at the adapter's inline routing, and no PR in the repo references pendingChildRequestRoutes or routeRuntimeRequest.

    Taking your framing rather than assuming the obvious fix. Two things I want settled before writing it:

    1. Does OpenCode's SDK actually let us reject or cancel a child request? If it does not, "reject it through the SDK" is not implementable and the answer changes.
    2. Does OpenCode have its own timeout on that request? If it does, the practical severity is lower and an explicit failure event plus a log may be the whole fix rather than a behavioural change to the retry budget.

    Also worth asking: whether the owner could be derived from the request itself (session id, task part id) rather than by polling thread state, since the polling is what makes this fragile in the first place.

    On your note about #16181 -- agreed it targets session-close settlement, not this give-up path. That draft is now closed: its teardown terminals turned out not to reach managed subscribers, since ProviderSessionManager ends those subscriptions before closing the adapter scope, so it was not delivering what it claimed. Your Pi finding above was the more useful input to that decision.

    I will follow up with a PR or, if the SDK gives us no way to reject, a narrower fix plus the reasoning.

  4. juliusmarminge commented on Oct 22, 2026

    @juliusmarminge
    Member

    Fresh triage on main b5a226e49c: the reported branches remain in the Pi adapter (t3.settle_probe) and OpenCode adapter (routeRuntimeRequest). The OpenCode route handles both permission.asked and question.asked, so fixing that half changes approval routing and requires maintainer review under the automated triage stop rule. No implementation changes or runtime reproduction performed; leaving this issue open for Julius. This is ordinary bug triage, with no exploitable vulnerability reported.

  5. juliusmarminge commented on Oct 22, 2026

    @juliusmarminge
    Member

    Fresh triage on main a50cd989ae: installed locked dependencies and launched the real web/server app with isolated copied state. The web endpoint returned HTTP 200 and the server reached Listening, but browser navigation and snapshots timed out; the server logged repeated event-loop stalls. Neither reported provider hang was reproduced. Pi is not installed on this VM and OpenCode reports zero stored credentials. The two code branches still exist in packages/provider-pi/src/server/adapter.ts and packages/provider-opencode/src/server/adapter.ts. Please provide the Pi command/extension and provider version that trigger the busy settle response, and an OpenCode child-task sequence plus version and redacted logs showing the unresolved ask. Those runtime steps are absent from the static-audit report. Leaving this open pending reproduction details.

  6. Adamulek123 commented on Oct 22, 2026

    @Adamulek123
    ContributorAuthor

    gpt-6.1-sol - responding

    @juliusmarminge Thanks for checking this again. The original report mixed a confirmed routing failure with an unsupported Pi hang claim. Here is the evidence and the remaining limit for each.

    Pi remains unconfirmed. My October 7 correction still applies. agent_settled and compaction_end can re-arm the probe. I do not have a Pi command, extension/version, or RPC log that demonstrates a busy reply followed by neither event. The original title's claim that this necessarily leaves a turn open forever was too strong. Please treat that half as withdrawn as a confirmed hang, pending an actual reproduction.

    OpenCode has newer runtime evidence in #16870, which you took over on October 19. I read the published records again today; I have not rerun them today.

    The recorded adapter comparison uses native OpenCode 1.18.18, a real SDK connection and HTTP/SSE stream, separate HOME/XDG/workspace, and a local scripted model. It compares pinned main 0b37829da959ddde47abdcec691ac8909d73744c with PR head e7274b52df6f23abd3b61bbe850a0f508eeaff78.

    The controlled sequence is:

    1. Create a native root session and a child with that root's parentID. Register the root in the production T3 adapter, with no active T3 turn.
    2. Send /session/<child>/prompt_async to the scripted model, which calls the real native question tool. The native stream emits question.asked with the child's session ID.
    3. Let the adapter's normal routing backoff expire. Inspect GET /question, native rejection events, and provider-session updates.

    Recorded outcomes in the adapter before/after HTTP/SSE evidence:

    • Base: ask at 2026-10-19T05:11:09.146Z; at 05:11:15.658Z the question is still pending, with no native rejection and no provider-session error.
    • PR head: ask at 05:11:37.050Z; at 05:11:42.111Z the named question has been rejected, the native pending-question list is empty, and the adapter emits OpenCode question request ... could not be routed to an active thread.
    • Foreign-session control: the question remains pending with no rejection or session error. The fix does not reject requests merely because another native root exists.

    This establishes the give-up behavior at the production adapter boundary with a real provider. It does not establish an indefinite native timeout, or reproduce the complete user workflow that leaves a child asking after its T3 owner becomes inactive. The probe supplies connection/host services and deliberately sets up the inactive owner. It does not exercise ProviderSessionManager, SQLite persistence, or the UI. Your request for an end-to-end child-task sequence remains valid.

    There is also a concrete constraint on the fix. The native API evidence shows that rejecting one question returns HTTP 200 and leaves the other child's question pending; repeating that rejection returns 404. In contrast, rejecting one of two permissions in the same asking session cancels both, with two permission.replied events. That is why #16870 now rejects only locally owned orphan questions. It reports exhausted permission routing explicitly and leaves native permissions pending, rather than cancelling unrelated approvals. It does not add automatic permission recovery.

    The PR's focused adapter tests cover inactive owners, late local registration, foreign ownership, and permission retention. The published verification records 162 passing tests across the two adapter suites, with all 12 focused cases failing against pre-PR production. Those are recorded results, not a fresh run against today's main.

    The remaining reproduction gap is therefore specific: Pi still needs any reachable runtime example; OpenCode has native/adapter evidence but still needs the full managed-thread lifecycle reproduction. I would keep the issue focused on the OpenCode routing failure and avoid treating the Pi theory as established.

  7. juliusmarminge commented on Nov 12, 2026

    @juliusmarminge
    Member

    I tried to reproduce this on the current build and could not.

    Commit 1e438c8 (origin/main). Not seen in a real run of the app. A real child-session ask needs a model to launch an OpenCode background task whose child asks after the root turn has settled. OpenCode 1.19.5's default general subagent denies the question tool, so the realistic route is a child permission ask in a supervised runtime mode, which needs a real model run. Instead, I added the test "reports a child question it cannot route once routing gives up" to packages/provider-opencode/src/server/adapter.test.ts. The root session is registered with no active turn. The native stream then emits question.asked for a child whose session.get returns parentID=root, and the clock is advanced 10 s. On main, no provider_session error and no runtime_request event is emitted, so the test fails. With the adapter change from the open PR #16870 applied, the same test passes. The Pi half was not reproduced and the reporter withdrew it: agent_settled and compaction_end re-arm the settle probe, and no Pi log or repro shows a busy reply followed by neither event.

    Closing as not planned. If it still happens for you, reply with the exact steps and the version you are on and it will be looked at again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is broken or behaving incorrectly.via-triageFiled through npx t3 triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions