Repository navigation
docs(harness): Research which Harness tools survive an ACP attach - #669
Conversation
Architect gate @
|
| Item | Verdict |
|---|---|
Research note under docs/research/ with Fact/Inference/Assumption |
Pass |
| Per-Harness tool-class matrix | Pass |
| Ranked options + ADR consequences | Pass |
| Smallest falsifying probe | Pass — two ACP frames; cheap half needs no macOS/subscription |
| Follow-up issues only after accept | Pass — titles listed, not filed |
Standards
Primary sources cited (adapter 4deace4, Agent SDK argv, Claude binary strings, Anthropic docs, our harness.rs / acp_wire.rs). Load-bearing claims table is the right shape. Honest about Assumption cells.
Product correction (Architect accepts for gating)
Prior Architect diagnosis overstated the loss as “native tools gone.” Note’s stronger claim: claude_code preset survives; attach loses computer use, project/local MCP scoped to cwd, and AskUserQuestion — largely because we attach with cwd = app data folder, not because ACP strips the preset. Independent gotcha: default ~20s turn budget can cancel permission-gated tools before Chat answers.
Soft
- Windows CI still pending on this tip at gate time — do not merge until CLEAN if you care; content gate does not depend on it.
- ADR-0003 Consequences should gain the interactive-session gate when spike(harness): Keep the Harness's own tools under ACP (computer use, web, user MCP) #668 is accepted (note says so; not this PR’s job to edit the ADR unless Oded wants it in-tree with the note).
Gate
CLEARED for the research note. I will not merge — Oded merges when ready. Next product move after accept: cheap cwd MCP-diff probe, then decide whether to implement option #2 (attach cwd) + README honesty (#1).
— Cursor agent (Architect), on @omesser's behalf.
Spike for #668. Co-authored-by: Oded Messer <odedmesser@gmail.com>
87589bd to
11e3a23
Compare
Live probes against claude-agent-acp 0.78.0, grok, hermes, opencode and pi-acp, recorded with Fact / Inference / Assumption labels. Companion to the desk research in #669. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Live probes against claude-agent-acp 0.78.0, grok, hermes, opencode and pi-acp, recorded with Fact / Inference / Assumption labels. Companion to the desk research in #669. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Live probes against claude-agent-acp 0.78.0, grok, hermes, opencode and pi-acp, recorded with Fact / Inference / Assumption labels. Companion to the desk research in #669. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Why
#668 asks whether ACP attach demotes a Harness to a chat brain plus buddy MCP. If it does, the Harness-versus-Model-API bet loses its main reason to exist, and ADR-0003 — which shipped no Executor because "the Harness now genuinely brings its own" — is resting on a capability the product cannot actually reach.
The claim needed checking against primary sources rather than against the symptom, because "the Harness did nothing" is what a missing tool, an unanswered permission prompt, and an expired turn budget all look like from Chat.
What changed
One new note,
docs/research/harness-native-tools-under-acp.md. No product code, and no probe script — the probe in the note is two JSON-RPC frames and needs neither.The finding is that the premise is half wrong, and that the surviving half is narrower and more fixable than the issue assumed.
claudeattach keeps the fullclaude_codetool preset. Zed's adapter passestools: { type: "preset", preset: "claude_code" }andsettingSources: ["user", "project", "local"], and it merges ai-buddy's MCP server over the user's rather than replacing it.Bash,Read/Write/Edit,WebSearch,WebFetch, subagents, skills andCLAUDE.mdare all present.AskUserQuestion.cwd— not ACP, not the adapter, not ADR-0023/0026. ai-buddy attaches in its own app data folder (harness.rs:1492-1501), and every one of the five Harnesses keys project configuration offcwd: Claude's local-scope MCP servers, Claude's.mcp.json, Claude'senabledMcpServerscomputer-use opt-in (recorded per project in~/.claude.json), Grok's.grok/config.tomlwalk to the git root, opencode's project config, Hermes' tool roots. The single highest-value lever in the note is a directory we already choose and currently choose badly.~/.claude.jsonis read regardless ofsettingSourcesand "PassingmcpServers: {}does not suppress the connectors". The mail-and-calendar case in the issue works already if the user added it as a connector or at--scope user.fs/terminalwould not help: those are methods the agent calls on us. Advertisingterminalwould make ai-buddy a shell Executor, one input class over from what ADR-0003 refuses.elicitation.formis the one capability that demonstrably buys a tool back.Deliberately not done: no ADR was written or superseded, and no issues were filed. The note ranks six options with their ADR-0003 / 0010 / 0018 / 0022 / 0023 / 0026 consequences and proposes follow-up titles only, per the ticket. It surfaces rather than routes around one ADR disagreement: ADR-0003's consequences are incomplete, because they name the Pro/Max gate and not the interactive-session gate that no subscription clears. That is flagged for reopening, not superseded here.
How to verify
The note is the deliverable, so verification is that its claims hold. Every load-bearing claim is labelled [Fact] / [Inference] / [Assumption] inline and collected in one table near the top, so a reviewer can check the labels rather than re-read the argument.
Scenarios covered by the note, each answering one of #668's eight spike questions:
Optionsobject andtoolspreset resolution, quoted from@agentclientprotocol/claude-agent-acp0.76.0 at4deace4.enabledMcpServers, per project in~/.claude.json, absent from the settings reference), plus the constant and the enablement check read out of the shipped@anthropic-ai/claude-code2.1.270 binary. Blocker attributed: Anthropic platform, with a scoping accident stacked on top.fs/terminalare actually for, and the one adapter line where an omitted capability removes a tool.claude,opencode,hermes,grok,pi, with per-row vendor citations and honestconditional/n/acells.Nothing here was run against a live Harness, and the note says so: the computer-use cell is the one genuinely unknown value in the matrix. The recommended probe reads the Agent SDK's
initmessage — which carries the authoritativetoolsandmcp_serverslists — through the adapter's documentedemitRawSDKMessagesextension, with the success signals tabulated and the cheap half (diffmcp_serversacross twocwdvalues) callable out as the one to run first, since it needs neither macOS nor a subscription.Checks run:
pre-commit run --files docs/research/harness-native-tools-under-acp.md— all applicable hooks pass, the rest skip as no-files.Notes
Closes nothing. #668 stays open until Architect and Oded accept the spike; its Done-when includes filing follow-up issues, which this pull request proposes as titles only.
Sources are primary throughout: Anthropic's Claude Code and Agent SDK documentation, the ACP v1 specification,
claude-agent-acpandpi-acpsource at named commits, the@anthropic-ai/claude-agent-sdk0.3.257 argv assembly, strings from the@anthropic-ai/claude-code2.1.270 binary, opencode / Hermes / xAI vendor docs, and ai-buddy's ownharness.rs,acp_wire.rs, README and ADRs ate070216.Two findings worth a reviewer's attention beyond the ranking.
README.md:157oversells thepirow — "Chat-only (no MCP)" is true of our MCP and misleading about pi's own tools, which are intact and local, and the cause ispi-acpacceptingmcpServerswithout wiring them through. And the shippedclaudebinary carries an undocumentedALLOW_ANT_COMPUTER_USE_MCPenvironment variable; it is recorded as a finding, not proposed as a plan, because setting an undocumented vendor flag into a Harness we spawn sits close to ADR-0010 rules 4 and 5.— Cursor agent, on @omesser's behalf.