Skip to content

fix(harness): Serve the MCP endpoint from the probe before it attaches - #1035

Merged
omesser merged 3 commits into
mainfrom
fix/984-probe-harness-mcp
Sep 26, 2026
Merged

omesser merged 3 commits into
mainfrom
fix/984-probe-harness-mcp

Conversation

@omesser

@omesser omesser commented Sep 26, 2026

Copy link
Copy Markdown
Owner

Why

scripts/probe-harness.sh returned from main.rs before mcp_http::serve ran, so mcp_http::endpoint() was None on the probe path. choose_mcp then skipped the HTTP branch even for a Harness that advertised mcpCapabilities.http and handed over the --mcp-stdio shim with an empty environment, which exits on its first call (ADR-0026). The probe printed mcp http true next to that shim and said nothing, and every per-Harness MCP claim in the README was read off that silence.

Confirmed at runtime before the change, against Goose:

  mcp http     true
  mcp          /private/tmp/ai-buddy-984/target/debug/ai-buddy --mcp-stdio

The probe now serves the endpoint before it attaches, so choose_mcp takes the branch the app takes, answers any tool call through the same dispatch the frame loop uses, and reports under mcp listed whether the Harness fetched tools/list. A probe whose loopback bind fails says so in capitals under mcp.

Closes #984.

Scope

  • harness::probe serves mcp_http::serve before attach, spawns answer_probe_calls, and prints mcp listed after the turn and mcp call per tool call.
  • harness::answer_probe_calls dispatches through ai_buddy_core::dispatch::dispatch with an empty roster, no ExpressionHandle, and StubWindowSource. speak comes back success: false; remember writes the real memory.md, which the probe already documented.
  • mcp_http::tools_listed counts tools/list answers.
  • ai_buddy_core::window_source::StubWindowSource loses its cfg(not(macos)); nothing used it on any platform.
  • The fake ACP agent records mcp=http or mcp=stdio from session/new, and a test asserts the probe hands an http-advertising Harness the loopback URL.
  • README: the note under the tools table says what the stock probe proves, and the Pi row states the measured outcome. docs/DEVELOPMENT.md and the script header describe the new lines.

Tradeoffs

The issue's second route, a loud warning with no endpoint, was rejected once route one measured as small: one channel, one thread, and the existing dispatch. The receiver is not a stub, which ADR-0026 rules out, and not a second tool implementation, which ADR-0023 rules out.

PROBE_PROMPT stays fixed. Delivery of the tool list is observed server side through mcp listed, so it does not depend on a model being up or obeying. Which tools a Harness then shows its model came from the tool-class research note's own client, and the README note now says so.

Blast Radius

Probe path and its docs only. The app's serve call in main.rs is untouched; tools_listed is a read-only counter it never reads. StubWindowSource becoming unconditional compiles on macOS and adds one unused-by-the-app type elsewhere.

Built and tested on macOS only. src-tauri/src/platform/x11/ does not compile on this machine; the change touches no platform module.

Verification

  • cargo test -p ai-buddy (588 passed) and cargo test -p ai-buddy-core (494 passed); cargo clippy --workspace --all-targets -- -D warnings clean; node --test tests/ 343 passed; pre-commit green including shfmt and shellcheck on the script.
  • the_probe_serves_the_loopback_endpoint_before_it_attaches fails on the first commit when run alone (mcp=http count 0) and passes after the fix.
  • Stock scripts/probe-harness.sh after the fix:
Harness mcp mcp listed turn
goose http://127.0.0.1:55848/mcp yes, 1 request Goose's own provider on this machine is down, so the reply is its network error; the list was fetched before it
codex http://127.0.0.1:55878/mcp yes, 1 request end_turn, Wave | Hello from the probe.
copilot http://127.0.0.1:55909/mcp yes, 1 request end_turn, Wave | Hello from the probe.
pi ai-buddy --mcp-stdio (with environment) no, never asked Pi's provider retried three times and gave up; pi-acp stores session/new servers and never passes them to pi (#1019)

Goose before and after, the same machine and prompt:

before:  mcp          /private/tmp/ai-buddy-984/target/debug/ai-buddy --mcp-stdio
after:   mcp          http://127.0.0.1:55848/mcp
         mcp listed   yes, 1 tools/list request(s)

Not run here: claude, hermes, opencode, cursor-agent, grok. Nothing in this change is Harness-specific.

🤖 Generated with Claude Code

— Claude Code agent, on @omesser's behalf.

omesser and others added 3 commits September 26, 2026 06:44
Fails alone: the fake advertises http and gets no loopback URL (#984).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…Harness

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@omesser omesser left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Architect review — CLEARED

Tip: 8c5a9dcd95df11034c9cb9270e042df69e353f88
Title: fix(harness): Serve the MCP endpoint from the probe before it attaches
CI: green (ubuntu / macos / windows × PR Tests; Pages build ok, deploy skipped)
Mergeable: MERGEABLE / CLEAN
Scope: probe path + docs (+111 / −32, 6 files). App serve in main.rs untouched. Closes #984.

Verified

Claim Tip state
Probe served MCP before attach so choose_mcp takes the HTTP branch probe calls mcp_http::serve then spawns answer_probe_calls before handshake print / turn.
Same dispatch as the frame loop (ADR-0026 / ADR-0023) answer_probe_calls uses ai_buddy_core::dispatch::dispatch with empty roster, no ExpressionHandle, StubWindowSource. Not a stub receiver; not a second tool impl.
Loud failure when bind fails Capitalized NO LOOPBACK LISTENER line under mcp.
mcp listed observes tools/list server-side LISTED AtomicUsize + tools_listed(); printed after the turn.
Fake ACP records transport; regression test mcp=http / mcp=stdio recording + the_probe_serves_the_loopback_endpoint_before_it_attaches.
README / DEVELOPMENT honesty Pi row states never-asked + #1019; stock probe note says list delivery ≠ tool call; measured Goose/Codex/Copilot rows.
StubWindowSource unconditional cfg(not(macos)) removed; probe needs it on macOS.

No Hard. Probe-only blast radius matches body.

Soft residuals (non-blocking)

  • remember on the probe path still writes real memory.md (already documented). Acceptable for a stock probe.
  • Named Harnesses not re-probed here (claude / hermes / opencode / cursor-agent / grok) — body says so; not required for #984.
  • ENDPOINT OnceLock is process-wide (app already); fine for one probe process.

Verdict

CLEARED. Ready for human merge; Architect does not approve or merge.

— Cursor agent (Architect), on @omesser's behalf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(verify): probe-harness serves no MCP endpoint, so every tool claim it made is untested

1 participant