Skip to content

feat(harness): Name GitHub Copilot CLI as a preset once smoked #1016

Closed

Description

@omesser

Problem

The README's Harness Support table sends GitHub Copilot CLI through the escape hatch: "copilot --acp --stdio reaches ai-buddy this way today and earns a named row once smoked (#457)". #457 closed with Grok, Codex and OpenCode named and Copilot still unsmoked, and nothing tracks it since. A user who types copilot --acp into Custom gets no row, no Settings option, no Connect button, and no evidence anyone has run it.

What a named row is

What #971 gave Goose. One value, copilot, that every surface understands:

  • harness::launch maps copilot to its argv, and named_login and not_installed carry its sign-in line and install hint.
  • HARNESS_PRESETS lists it, so Settings offers Harness · copilot and the two disclosures naming standalone CLIs include it.
  • Chat's connect landing has a branded button, DISPLAY_NAMES and installUrlFor know it.
  • The README names it in the launch table, the tools table and the session table, and the escape-hatch row stops naming it.
  • docs/DEVELOPMENT.md's AI_BUDDY_HARNESS row lists it.
  • The two Settings fixtures are regenerated from the comparison test's dump.

What the smoke has to establish

ADR-0022 sets the bar at a turn smoked fresh and resumed, through AI_BUDDY_HARNESS=copilot scripts/probe-harness.sh. The row states each of these from the run, not from the vendor's docs:

  1. A fresh session ends in end_turn.
  2. Whether initialize advertises loadSession, and whether a second run reports the same session id. cursor-agent advertises none and every attach is fresh; the row says which this is.
  3. What initialize advertises: mcpCapabilities.http, authMethods, and the rest the session table records.
  4. Whether ai-buddy's MCP reaches the session. The stock probe serves no endpoint (fix(verify): probe-harness serves no MCP endpoint, so every tool claim it made is untested #984), so this needs a local run that does, the way the Goose tools row was measured. If it cannot be established the row says so instead of guessing.
  5. What tools it keeps under the attach, for the tools table.

The argv is checked against copilot --help before it is trusted; --stdio is not in the help text.

— Claude Code agent, on @omesser's behalf.

Activity

  1. added
    enhancementNew feature or request
    p1Next up: the default for work that should happen
    macosmacos / OSX specific
    on Sep 25, 2026
  2. added this to the V1 milestone on Sep 25, 2026
  3. self-assigned this
    on Sep 25, 2026
  4. omesser commented on Sep 25, 2026

    @omesser
    OwnerAuthor

    The smoke, copilot 1.0.88 at /opt/homebrew/bin/copilot, macOS 25.6, origin/main at e5ac66b7. Three runs of AI_BUDDY_HARNESS="copilot --acp" scripts/probe-harness.sh, in this order. The first two are the stock probe; the third is a local run patched to serve mcp_http and log every method it receives, the same instrument the Goose tools row came from. Nothing from that patch is committed.

    The argv

    copilot --help lists --acp ("Start as Agent Client Protocol server") and does not list --stdio. Both copilot --acp and copilot --acp --stdio answered initialize with the same body, so the preset is copilot --acp.

    {"jsonrpc":"2.0","id":0,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":true,"mcpCapabilities":{"http":true,"sse":true},"promptCapabilities":{"image":true,"audio":false,"embeddedContext":true},"sessionCapabilities":{"close":{},"list":{}}},"agentInfo":{"name":"Copilot","title":"Copilot","version":"1.0.88"},"authMethods":[{"id":"copilot-login","name":"Log in with Copilot CLI","description":"Run `copilot login` in the terminal","_meta":{"terminal-auth":{"command":"/opt/homebrew/Caskroom/copilot-cli/1.0.88/copilot","args":["login"],"label":"Copilot Login"}}}]}}
    

    Run 1, fresh session, exit 0

    probe-harness
      harness      copilot
      command      copilot --acp
      cwd          /Users/oded/Library/Application Support/ai-buddy
      data         /Users/oded/Library/Application Support/ai-buddy/probe
      timeout      turn 120s, attach 120s
    
    attach
      agent        Copilot
      loadSession  true
      mcp http     true
      mcp          /Users/oded/prisma/workspace/ai-buddy/.worktrees/copilot-preset/target/debug/ai-buddy --mcp-stdio
      authMethods  Log in with Copilot CLI
      session      a51b9ded-0851-432e-adde-f0febf934476
    
    turn
      prompt       Reply with exactly this one line and nothing else: Wave | Hello from the probe.
      stop         end_turn
      reply        Wave | Hello from the probe.
      proposal     Wave | Hello from the probe.
    

    Run 2, resumed session, exit 0

    Same session id as run 1, read back from probe/harness-session.json.

    probe-harness
      harness      copilot
      command      copilot --acp
      cwd          /Users/oded/Library/Application Support/ai-buddy
      data         /Users/oded/Library/Application Support/ai-buddy/probe
      timeout      turn 120s, attach 120s
    
    attach
      agent        Copilot
      loadSession  true
      mcp http     true
      mcp          /Users/oded/prisma/workspace/ai-buddy/.worktrees/copilot-preset/target/debug/ai-buddy --mcp-stdio
      authMethods  Log in with Copilot CLI
      session      a51b9ded-0851-432e-adde-f0febf934476
    
    turn
      prompt       Reply with exactly this one line and nothing else: Wave | Hello from the probe.
      stop         end_turn
      reply        Wave | Hello from the probe.
      proposal     Wave | Hello from the probe.
    

    Run 3, patched to serve MCP, exit 0

    MCPREQ lines are the patched mcp_http::handle naming each method Copilot sent. The proposal: no at the end is expected: this run's prompt asks for a tool list, not a Behavior.

    probe: mcp endpoint served = true
    probe-harness
      harness      copilot
      command      copilot --acp
      cwd          /Users/oded/Library/Application Support/ai-buddy
      data         /Users/oded/Library/Application Support/ai-buddy/probe
      timeout      turn 120s, attach 120s
    
    attach
    MCPREQ server/discover
    MCPREQ initialize
      agent        Copilot
      loadSession  true
      mcp http     true
      mcp          http://127.0.0.1:61569/mcp
      authMethods  Log in with Copilot CLI
      session      a51b9ded-0851-432e-adde-f0febf934476
    
    turn
      prompt       Answer in exactly two sections and nothing else.
    TOOLS: every tool you can call in this session, by exact name, comma separated.
    RESOURCES: every MCP resource you can read, by exact URI, comma separated. Write RESOURCES: none if you cannot list or read any.
    MCPREQ tools/list
      stop         end_turn
      reply        TOOLS: bash, read_bash, stop_bash, list_bash, view, create, edit, web_fetch, search_code_subagent, skill, sql, session_store_sql, read_agent, list_agents, write_agent, grep, glob, task, github-mcp-server-get_copilot_space, github-mcp-server-get_file_contents, github-mcp-server-list_copilot_spaces, github-mcp-server-search_code, github-mcp-server-search_users, ai-buddy-describe_screen, ai-buddy-list_instances, ai-buddy-list_windows, ai-buddy-play_behavior, ai-buddy-recall, ai-buddy-remember, ai-buddy-speak, parallel
    RESOURCES: none
      proposal     no, the first line is not a Behavior name
    

    What that establishes

    Question Answer Evidence
    Fresh session ends in end_turn yes run 1
    loadSession, resumed session advertised true; run 2 reused a51b9ded… and ended in end_turn runs 1 and 2
    initialize mcpCapabilities.http and sse both true; one auth method, copilot-login, a terminal command (copilot login); promptCapabilities.image and embeddedContext; sessionCapabilities.close and list the JSON above
    ai-buddy's MCP reaches it yes, over HTTP. It called server/discover, initialize and tools/list on the served endpoint and listed all seven ai-buddy tools under an ai-buddy- prefix. No call into a tool was made, so this confirms delivery of the list, not a call. run 3
    Tools kept under the attach bash, read_bash, stop_bash, list_bash, view, create, edit, grep, glob, web_fetch, search_code_subagent, skill, sql, session_store_sql, read_agent, list_agents, write_agent, task, parallel, five github-mcp-server-* tools from its bundled server. No web-search tool, no ask-user tool. run 3

    Not established: whether a running Copilot session re-reads ~/.copilot/mcp-config.json after a copilot mcp add, and no speak through the Settings-generated registration has been drawn in a bubble. copilot mcp add --transport http … --header "Authorization: Bearer …", get and remove were run against the CLI directly; a second add of a held name exits 1 with "already exists", which is why the generated snippet removes first.

    — Claude Code agent, on @omesser's behalf.

  5. omesser commented on Sep 25, 2026

    @omesser
    OwnerAuthor

    Marks for the rows that lack one, measured against Simple Icons' 3470 slugs: pi exists and is the right product (source https://pi.dev/favicon.svg), so the README row now carries it. No OpenAI, xAI, Grok, Block or Goose mark exists there; the near matches (OpenAI Gym, ngrok, Mongoose) are other products. codex, grok and goose would each need a self-hosted SVG under docs/readme/ like nous.svg, with the vendor's licence carried in the tree. Not done in #1017.

    — Claude Code agent, on @omesser's behalf.

  6. omesser commented on Sep 25, 2026

    @omesser
    OwnerAuthor

    Shipped on main via #1017 (merge 52a4442).

    What landed: named copilot preset (copilot --acp), Settings option and disclosures, Chat connect button, BYO copilot mcp remove/add arm, README launch/tools/session rows (escape-hatch row no longer names it), ADR-0022 Copilot sentence as history, DEVELOPMENT.md AI_BUDDY_HARNESS list. Smoke evidence (fresh, resumed, HTTP MCP tool list) is on this issue.

    Still out of scope here, recorded as standing rather than open work: BYO registration is config generated, unverified (no speak through the Settings snippet yet); mark gaps for codex/grok/goose are #1021.

    — Cursor agent (Coder), on @omesser's behalf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestharnessmacosmacos / OSX specificp1Next up: the default for work that should happen

Projects

  • Status
    Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions