Skip to content

Prime Agent active turns cannot survive a Pylon server restart #84

Description

@rynfar

Problem

A daemon-backed Prime Agent turn cannot survive a restart of the Pylon server process. The projected thread may still show a running turn and the Prime daemon may still own native work, but startup reconciliation cannot adopt that execution. Pylon marks the provider session orphaned and requires a new user message.

This is separate from the in-process daemon transport reconnect handled by #79. In that path the Pylon process, adapter context, active turn id, event cursor, MCP ownership, and settlement guards remain alive.

Current constraints

  • ProviderSessionDirectory.runtimePayload.activeTurnId is written after adapter.sendTurn returns, so it is late or stale during an active Prime turn.
  • startup reconciliation marks projected active sessions missing from adapter memory as orphaned
  • ordinary Prime resume continues a transcript but does not adopt an existing native execution
  • the daemon manager retires an existing daemon on a new Pylon process start
  • supervised sessions intentionally reject restored active execution

Validated cross-repository dependency

The current public Prime SDK/daemon API cannot transfer a live client_owned worker to a new Pylon process. Prime creates a fresh in-memory protocol owner for each daemon client, hides client-owned workers from other owners, rejects their attach, and starts owned-worker cleanup after disconnect. Persisting native ids alone cannot cross that boundary. Promoting the worker to a resident session would remove private owner scoping and is not an acceptable recovery mechanism.

The required fork primitive is tracked by pylon-code/prime-agent#27. It must use a private, capability-gated recovery handle to atomically adopt and attach the same worker after the old owner is gone, fence the old owner, transfer scoped MCP ownership, and return exact generation/correlation continuity. Wrong, stale, concurrent, partial, or unavailable continuity fails without disclosing the worker.

Dependency order and version-one boundary

  1. Land exact admission/session-incarnation ownership in Provider teardown can leave later turns stuck starting #175 / PR fix(server): reconcile stalled provider starts #192.
  2. Land snapshot-generation isolation in prime-agent#21 / Prime PR chore(web): clear stale T3 test copy and two lint false positives #19.
  3. Establish the provenance-checked fork artifact/install path tracked by Prime provider parity: tasks, automation, and upstream watch #114 and prime-agent#1.
  4. Land the Prime recoverable-owned-session capability in prime-agent#27.
  5. Add Pylon's private recovery ledger, daemon retention/detach path, active lifecycle persistence, startup adoption, and exact-once completion/checkpoint recovery as focused Pylon PRs.

Version one is deliberately narrow: full-access sessions on macOS/Linux, the same retained daemon/supervisor generation, exact complete replay or snapshot continuity, and no unresolved UI interaction. Approval-required, Windows native transport, copied state, daemon replacement, partial replay, or ambiguous identity retain the existing precise orphan failure.

Hard invariants

  • Native prompt admission remains exactly one; recovery never calls prompt submission.
  • The canonical turn keeps one turn.started, terminal settlement, admission acceptance, and checkpoint.
  • The old owner cannot mutate after adoption and two Pylon processes cannot publish the same lineage.
  • Scoped MCP credentials and the managed extension are restored before recovered activity is visible.
  • Recovery handles, native ids, correlations, paths, cursors, snapshots, prompts, tool data, and transport errors never enter public contracts, events, receipts, or logs.
  • Private recovery state is removed only after terminal projection, checkpoint quiescence, and authoritative native cleanup.

Expected behavior

When durable state and a private native recovery handle authoritatively identify the same live full-access Prime execution, a restarted Pylon server should adopt it under the existing canonical turn id without replaying the prompt or emitting duplicate turn lifecycle events. If identity or event continuity cannot be proven, Pylon should retain the precise orphaned-session failure.

Acceptance coverage

  • persist and clear active provider turn ownership from runtime lifecycle events, not after sendTurn returns
  • adopt a compatible surviving full-access Prime daemon/session by private identity
  • restore scoped MCP ownership before publishing recovered activity
  • no duplicate turn.started, turn.completed, prompt admission, or checkpoint
  • terminal native state reached during Pylon downtime reconciles once
  • ambiguous identity, unavailable event continuity, and supervised mode fail closed
  • no orphaned child, queue, MCP credential, provider-session binding, or daemon ownership
  • focused provider-directory, startup-reconciliation, manager, adapter, and SQLite integration tests

Related: #79, #80.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions