Skip to content

feat(orchestration-v2): import an existing Claude Code or Codex session by id, with two-way transcript sync - #5499

Open
Bil0000 wants to merge 215 commits into
pingdotgg:t3code/codex-turn-mappingfrom
Bil0000:feat/import-session-v2
Open

feat(orchestration-v2): import an existing Claude Code or Codex session by id, with two-way transcript sync#5499
Bil0000 wants to merge 215 commits into
pingdotgg:t3code/codex-turn-mappingfrom
Bil0000:feat/import-session-v2

Conversation

@Bil0000

@Bil0000 Bil0000 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Reimplements #4617 against the orchestration V2 runtime, as requested in #4617 (comment) — plus a safe two-way sync between the imported thread and the provider's own on-disk transcript.

Why

T3 Code can only continue sessions it started. A Claude Code CLI session or a codex thread already on the machine is fully resumable, but there was no way to open it here. The V1 attempt (#4617) was closed because #2829 replaced the orchestration layer it was built on; this is the V2 reimplementation, targeting the #2829 branch.

What

Import session... in the command palette takes one session id.

  • Resolve first. orchestration.resolveImportSession reads the session's working directory and title before anything is created. If a project covers that directory, the thread lands there regardless of which project is open; if not, the dialog relabels to Add project & import and nothing is created without a second press. Unlike V1, Codex sessions resolve too (workspace + title come from the rollout file).
  • Import is all-or-nothing. The transcript is read before anything is written — Claude via the agent SDK (getSessionInfo/getSessionMessages), Codex from its rollout .jsonl under $CODEX_HOME/sessions (no live app-server needed). History is appended as synthetic message.updated/turn-item.updated pairs with deterministic ids — the same mechanism as the legacy V1 thread importer — under a new historyOrigin: "provider_import". There is no create-then-compensate dance: a failed import leaves nothing behind.
  • The next turn resumes the real session. The import synthesizes a provider thread whose nativeThreadRef is the external session id (strength: "strong"; for Claude, nativeConversationHeadRef is set to the last transcript message so the adapter takes the resume path instead of colliding on sessionId). ProviderTurnStartService then resumes natively — the model keeps its full context; nothing is replayed.

Two-way sync

  • T3 → provider comes free. Because the thread resumes the original native session, Claude appends every T3 turn to the original ~/.claude/projects/**/<id>.jsonl, and Codex to its own rollout.
  • Provider → T3 is a lazy sync. On thread reads, ensureSynced re-reads the transcript (throttled by source mtime, serialized per thread, skipped while a run is active) and appends turns made outside T3. Idempotency and safety come from three dedupe layers: deterministic event ids, provider-native item ids already present as turn items, and exact role+text match against native run messages — so T3's own turns echoed back by the provider are never re-imported. Sync state lives in orchestration_v2_session_imports (migration 046).

Verification

  • 8 new server tests: pure transcript-mapper tests plus an integration test that imports a real on-disk Codex rollout into an in-memory V2 store, checks the projection (history origin, messages, provider thread binding), refuses duplicates, and round-trips the provider→T3 sync.
  • 6 new CommandPalette.logic tests (17/17 in the file).
  • fmt, lint, typecheck clean across contracts, server, client-runtime, web (pre-existing @cursor/sdk-related failures unaffected).
  • Live, against a real Claude Code session:
    1. claude -p "Remember this fact for later: the import canary code is FERRET-4413. Reply with exactly: OK" in a project directory.
    2. Palette → Import session... → paste id → thread appears with both prior messages rendered.
    3. Ask "What is the import canary code?" — the resumed session answers FERRET-4413. The canary only exists in the on-disk transcript, so the real session resumed rather than the history being replayed.
    4. The question and answer from step 3 appear in the original ~/.claude/projects/**/<id>.jsonl (T3 → provider).
    5. claude -p --resume <id> "…second canary is BADGER-7788…" in the terminal, reopen the thread in T3 — the CLI turn appears (provider → T3), and T3's own turns are not duplicated.

Demo

https://drive.google.com/file/d/16rBmArbhFZyHfzLWxNY7pxJNyqt7D9hs/view?usp=sharing

Known limits

  • Only plain user/assistant text is imported Solved: tool activity now survives the import — Claude thinking blocks become reasoning items, Bash calls become command items paired with their results, Edit/Write calls become file-change items (other tools get a compact command rendering); same for Codex rollouts (reasoning summaries, function calls paired by call id). Verified live against a real session with tool use.
  • Turns synced from the CLI render above native T3 runs Solved: sync-appended entries take position ordinals after the thread's current maximum, so CLI turns made after T3 turns render chronologically. Verified live (CLI turn after a native T3 turn lands at the bottom).
  • $CODEX_HOME only Solved: rollout discovery resolves the codex instance's configured home from its resolved home layout first, then $CODEX_HOME, then ~/.codex.
  • Codex live resume remains unexercised end-to-end (no Codex auth on the test machine); transcript parsing, resolve, import and sync are covered by the integration test against a real rollout file, and resume goes through the existing thread/resume adapter path.

Note

Add import and two-way sync of Claude Code and Codex sessions into orchestration-v2 threads

  • Adds a full import workflow (SessionImportService) that reads an external Claude Code or Codex session, maps its transcript to deterministic OrchestrationV2 events, and stores it as a new thread with historyOrigin: 'provider_import'.
  • Adds incremental sync (ensureSynced) that detects provider-side conversation added after import, backfills command outputs, and is called automatically on getThreadProjection and subscribeThread.
  • Adds two new WS RPC endpoints (resolveImportSession, importSession) with auth scopes and client-runtime command functions, plus an ImportSessionDialog UI accessible from the command palette.
  • Extends ProjectionStore ordinal logic so that provider-imported runless items are only included in history up to the correct run band when forking.
  • Adds migration 046 creating the orchestration_v2_session_imports tracking table with a unique index on (driver, external_id).
  • Risk: getThreadProjection and subscribeThread now perform a file-system or network read (ensureSynced) on every call for imported threads; failures are swallowed with a warning but add latency.

Macroscope summarized d265cee.

juliusmarminge and others added 30 commits August 5, 2026 15:51
Co-authored-by: codex <codex@users.noreply.github.com>
- Initialize provider as unchecked in a pending state
- Update initial probe message to reflect session-local status
- Type the runtime effect with `Scope`
- Build the ACP session runtime without wrapping it in `Effect.scoped`
- Use strict TurnId and ProviderItemId parsing in Codex session routing
- Decode in-memory stdio chunks in streaming mode to avoid split UTF-8 corruption
- Transfer session-owned scopes into adapter state
- Ensure runtime scopes close on stop and startup failure
- Add regression coverage for scoped lifecycle cleanup
- Close the managed native event logger when the adapter layer tears down
- Make session runtime close idempotent with an atomic closed flag
- Add coverage for flushing thread native logs on shutdown
- Use codex app-server snapshots for auth, models, and skills
- Remove legacy CLI/config discovery paths and related helpers
- Update tests for the new provider status flow
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
- Document the target orchestration graph, IDs, lifecycles, and capability model
- Add Codex app-server probe fixtures and update the probe test harness
- Introduce orchestration v2 service interfaces and error types
- Add replay runtime, fixtures, and integration coverage
- Update shared contracts and probe transcripts

Co-authored-by: codex <codex@users.noreply.github.com>
- Add Codex adapter and replay harness wiring
- Introduce in-memory orchestration projections and provider registry
- Expand orchestration contracts for turn and runtime events
Co-authored-by: codex <codex@users.noreply.github.com>
- Add context transfer IDs, schemas, and projections
- Support cheap fork creation and Codex native fork rollback
- Cover fork idempotency and replay behavior in tests
- Track remaining projection, context transfer, rollback, capability, and subagent work
- Clarify current V2 baseline and debugger-only follow-ups
- Map fork and merge-back turns into stored handoffs and transfer resolutions
- Add shell snapshot projection support plus coverage tests
- Update replay fixtures and web contracts for the new turn flow
Co-authored-by: codex <codex@users.noreply.github.com>
- Move Codex replay recording into `apps/server`
- Add Claude Agent SDK replay fixtures and test harness
- Update orchestration-v2 fixture scenarios and docs
- Move Claude provider runtime logic into its own module
- Share the SDK query runner between live and replay paths
- Add replay driver error wrapping for unexpected failures
- persist the selected model on run records
- surface run model selection in the debug UI
- update replay fixtures and contracts for the new field
- Record Claude SDK transcripts across multiple prompts and restart/query modes
- Add approval and tool-call replay coverage for new orchestration fixtures
- Update Claude adapter testkit to model open/prompt/permission frames
- Derive Claude SDK query options from runtime policy
- Add read-only replay fixture and policy mapping tests
- Reuse shared approval-policy fixtures across orchestrator tests

Co-authored-by: codex <codex@users.noreply.github.com>
- add active steering and interrupt-restart replay fixtures
- update Claude adapter/orchestrator turn handling for steering
- refresh replay and integration test coverage
- add interrupt and mid-tool replay fixtures for Claude and Codex
- log Claude Agent SDK protocol frames to native event traces
- project Codex commandExecution start events into orchestration updates
- Map Cursor SDK agents and runs to V2 thread and turn lifecycles
- Update MCP capability, tool, and testing guidance for SDK-based injection
Co-authored-by: codex <codex@users.noreply.github.com>
juliusmarminge and others added 9 commits August 5, 2026 16:17
…sPinned

Main owns migration numbering: 036_ProjectionThreadsPinned landed on main,
so the v2 migrations shift from 036-044 to 037-045. Release path runs all
of main's migrations first, then the v2 stack.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Port thread pinning (pingdotgg#5312) into the orchestration-v2 command pipeline:
  thread.pin/unpin commands, thread.pinned/unpinned events, pinnedAt on the
  v2 thread state and projected shells, promotion semantics (pin clears
  settle/snooze, settle clears pin) matching the v1 decider, and client
  pin/unpin operations in the v2 dispatch style.
- Port the regenerated-title context anchoring (pingdotgg#5365) into
  ThreadTitleRegenerationService: pin the first user message ahead of the
  retained tail when the digest is truncated.
- Re-apply the right-panel controls positioning from pingdotgg#5260 to the v2
  ChatView title bar controls.
- Repair merge artifacts: committed conflict markers in BranchToolbar,
  duplicate capability keys, duplicate CommandPalette import, v1 turn
  naming in DiffPanel's focus-refresh effect, onSend signature merge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Match progress button spacing and single-line height to static git actions
…vider_import history origin

- ORCHESTRATION_V2_WS_METHODS.resolveImportSession / .importSession with
  input/output schemas and OrchestrationV2ImportSessionError
- historyOrigin gains "provider_import" for threads whose history was
  copied from an external provider session
…V2 thread

SessionImportService resolves a session id to its workspace/project and
imports its transcript as a new thread:

- The transcript is read before anything is written (Claude via the agent
  SDK's getSessionInfo/getSessionMessages, Codex from the rollout file
  under $CODEX_HOME/sessions), so a failed import leaves nothing behind.
- History lands as synthetic message.updated/turn-item.updated pairs with
  deterministic ids, mirroring the legacy v1 transcript importer; the
  thread carries historyOrigin "provider_import" so runless items render.
- A provider thread is synthesized with nativeThreadRef = the external
  session id (strength strong) and, for Claude, nativeConversationHeadRef
  = the last transcript message, so the first turn takes the adapter's
  resume path and every T3 turn writes back into the provider's own
  transcript.
- ensureSynced re-reads the transcript on thread reads and appends turns
  made outside T3 (guarded by mtime, event-id/native-id/text dedupe, and
  skipped while a run is active), tracked in
  orchestration_v2_session_imports (migration 046).
- Duplicate imports of the same external session are refused.
resolveImportSession/importSession RPC wrappers plus command atoms with
serial concurrency keyed by (environment, external id).
Palette action opens ImportSessionDialog: provider segmented buttons
(ready Claude Code/Codex instances only), session id input, two-phase
resolve with "Add project & import" when the session's workspace is not
a project yet, and navigation to the imported thread.
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 03f2d775-30e0-4dcb-9164-f8d6822ba669

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). labels Aug 6, 2026

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Effect service conventions review of the new SessionImportService and its contract error. Four convention violations found; details inline.

Posted via Macroscope — Effect Service Conventions

Comment thread packages/contracts/src/orchestrationV2.ts
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts
Comment thread apps/web/src/components/ImportSessionDialog.tsx
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
Comment thread apps/server/src/orchestration-v2/ProjectionStore.ts Outdated
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
@Bil0000
Bil0000 marked this pull request as draft August 6, 2026 12:03
@macroscopeapp

macroscopeapp Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature: importing existing Claude Code or Codex sessions with two-way transcript synchronization. It adds new UI components, RPC endpoints, a database migration, and complex orchestration logic for reading and syncing external provider transcripts. New features of this scope and complexity warrant human review.

You can customize Macroscope's approvability policy. Learn more.

- Imported transcripts now keep tool activity: Claude thinking blocks
  become reasoning items, Bash calls become command_execution items
  paired with their tool results, Edit/Write calls become file_change
  items, and other tools fall back to a compact command rendering. The
  same applies to Codex rollouts (reasoning summaries and function
  calls paired by call id). Only plain-text rows become conversation
  messages; ids stay compatible with previously imported threads.
- Transcript entries synced after the import take position ordinals
  after the thread's current maximum instead of the import's 0-band, so
  turns made in the CLI after T3 turns render chronologically instead
  of inside the history block.
- Codex rollout discovery resolves the instance's configured home from
  its continuation key (codex:home:<path>) before falling back to
  $CODEX_HOME / ~/.codex.
- Sync failures now log a pretty-printed cause.
@Bil0000

Bil0000 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Pushed f00d38658 addressing three of the four known limits:

  • Rich import — tool calls, thinking/reasoning, and file edits from the original session now import as command/reasoning/file-change turn items (Claude and Codex), with tool outputs paired to their calls. Message ids are unchanged, so threads imported before this commit sync cleanly.
  • Chronological sync — transcript entries synced after the import now take position ordinals past the thread's current maximum, so CLI turns made after T3 turns render in order instead of inside the history block.
  • Codex home — rollout discovery uses the instance's resolved home layout (falling back to $CODEX_HOME, then ~/.codex).

All verified live against real Claude Code sessions (tool-use session import renders the executed command with output; a CLI turn added after a native T3 turn appears at the bottom of the thread). 10 server tests + 6 palette tests pass; typecheck/lint/fmt clean.

The one remaining limit is Codex live resume, which needs a machine with Codex auth — covered by unit/integration tests up to the thread/resume adapter boundary.

Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts Outdated
Correctness (Macroscope review):
- Import write phase (positions, events, imports row) now runs in one
  transaction under the per-thread lock ensureSynced uses, so a failure
  can no longer leave a thread partially imported, and the negative
  sync cache can no longer race the import and permanently disable
  syncing (the import clears the cache entry under the same lock).
- Fork inheritance gates provider-imported runless items by position
  ordinal, so forking from an earlier run no longer copies transcript
  entries synced after that run.
- Codex rollout lookup matches the full `-<id>.jsonl` segment (a short
  id can no longer collide with another rollout's tail), stats before
  reading (an append between read and stat can no longer be skipped by
  the next sync), and only treats a missing sessions directory as
  not-found — other filesystem errors surface as read failures.
- The role+text sync dedupe backstop is a multiset: each T3 run message
  absorbs one transcript echo, so a genuinely repeated prompt in the
  CLI still imports.
- The web dialog deletes the project it created when the import half of
  "Add project & import" fails.

Conventions:
- OrchestrationV2ImportSessionError now carries a structured reason
  discriminator and the externalId; errors are constructed at each
  failure boundary instead of through a helper.
- SessionImportService declares its interface inline in Context.Service
  and acquires FileSystem/Path from the environment in make.

CI: migration contiguity test now expects 46 entries (046 added).
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts
The active-run guard read the projection before the transcript read and
several queries, so a run dispatched in that window could interleave
with the sync write. The run-state check now repeats inside the write
transaction; on an active run the sync aborts without touching the
imports row, so the next thread read retries.
Comment thread apps/server/src/orchestration-v2/SessionImportService.ts
A tool call and its result are separate transcript lines, so a sync
could import the command before its output existed and the deterministic
event id then kept the completed output out forever. Sync now re-emits
imported command items whose output has since appeared — same item id
and ordinal, one deterministic revision event — and the projector
upserts them in place. Covered by an integration-test scenario.
@Bil0000
Bil0000 marked this pull request as ready for review August 6, 2026 13:30
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch 3 times, most recently from 72e3863 to a186d64 Compare August 11, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants