Skip to content

docs: Say ai-buddy never reads pixels and point computer use at cua-driver - #959

Merged
omesser merged 2 commits into
mainfrom
docs/882-harness-computer-use
Sep 23, 2026
Merged

omesser merged 2 commits into
mainfrom
docs/882-harness-computer-use

Conversation

@omesser

@omesser omesser commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Why

ADR-0031 and CONTEXT.md already record that ai-buddy drops Capture and keeps Free sensing, but the README never says it. A reader landing on the README gets one of two wrong answers. Either they assume the buddy screenshots them, which is alarming and false, or they assume desktop computer use is off the table, which is also false — it belongs to the Harness or to an MCP server they attach. Nothing public pointed at that path.

What changed

Two README edits, docs-only. No product code moves; the Settings surface has no capture disclosure to contradict, only the existing "Appear in screenshots and screen shares" toggle, which is about the sprite being visible in captures rather than taking them, and the new copy says so.

  • A bullet in What It Does for the reader who scans and stops there. It states the negative and links down.
  • A Computer use subsection at the end of Harness Support, next to the ACP tool-class table and the existing "Explicitly not served" line, because that is where a reader already asking about desktop control lands. It states what sensing is (window metadata, describe_screen describes layout), states the bound applies to ai-buddy's code and not to the agent attached to it, and names cua-driver as the portable option with both cua docs links.

Judgment call: the runner-up stays out of the README. minghinmatthewlam/computer-use-mcp is macOS-only and pre-1.0. A second name in user-facing copy forces a choice the reader has no basis to make, and the honest way to help them make it would be a security and maturity comparison the acceptance criteria rightly forbid. One primary option plus a link to the research note, which carries the full driver survey and the per-Harness table, unblocks the person who just wants this working and still leaves the survey one click away.

Copy checked against the three things it must not claim. It does not say the buddy ships screenshots, it makes no parity claim across drivers (only one driver is named, and the caution attached to it is about its permission model rather than a comparison), and the second paragraph exists specifically to prevent "no Capture" reading as "agents never see pixels".

How to verify

Both external links fetched today, live, not just a 200 from an SPA shell:

  • https://cua.ai/docs/how-to-guides/driver/connect-your-agent — 200, no redirect, <title>Connect your agent to Cua Driver | Cua docs</title>. Body carries per-client registration sections for Claude Code, Codex, Cursor, Prime Agent and a generic MCP JSON block, and documents the Windows and Linux runtime, which is where the multi-OS claim in the copy comes from. The issue's URL is current; no path move.
  • https://cua.ai/docs/reference/cua-driver/mcp-tools — 200, no redirect, <title>MCP Tools (macOS) | Cua docs</title>. Live. Worth knowing that this reference page is scoped to macOS even though the driver is not, which is why the copy introduces it as "lists what it exposes" rather than as the platform statement.
$ node --test tests/
1..312
# tests 318
# suites 2
# pass 318
# fail 0

pre-commit run --files README.md passes, codespell and ast-grep included.

A reviewer can check the in-repo links resolve: docs/research/capture-drop-and-harness-cu-path.md and the #computer-use anchor generated by the new ### Computer use heading.

Notes

ADR-0031 is cited, not ADR-0005. The ADR-0005 amend and the consent-string scrub are separate issues and untouched here, as are the files #924 and #916 are editing.

Closes #882

— Claude Code agent, on @omesser's behalf.

🤖 Generated with Claude Code

…river

Closes #882.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread README.md Resolved
ADRs are internal. A README reader has no use for the number and no
reason to follow it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@omesser
omesser merged commit 7b3dd09 into main Sep 23, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: Point users at cua-driver / CU MCP for harness computer use

1 participant