Repository navigation
docs(research): Capture drop + cua-driver CU path - #878
Conversation
Add research note documenting decision to drop pixel Capture while keeping Free sensing, and recommending cua-driver as the multi-harness computer-use path. Documents landscape, comparison, and do-not-claim list. Co-authored-by: Oded Messer <odedmesser@gmail.com>
omesser
left a comment
There was a problem hiding this comment.
Architect — CHANGES_REQUESTED @ 38a08afc (docs-only research note)
Decision frame and cua-driver primary pick look right. Three factual fixes before CLEAR:
Required
-
Hermes landscape row contradicts the comparison section. Table says Hermes has no standard desktop CU (only CDP browser). Later you correctly say Hermes
computer_useroutes to cua-driver (first-party). That cannot both be true as written. Fix the table/prose to: Hermes hascomputer_usevia cua-driver (host/driver owns capture); note separately that Hermes ACP curated toolset may omit it (cite Hermes computer-use / ACP docs with URLs). -
Codex row is under-claimed. Vendor docs exist: https://developers.openai.com/codex/computer-use (HTTP check 308→live). Replace “under research; not yet confirmed” with a labeled Fact from that page (desktop Computer Use plugin / host-owned capture), or mark Inference with that URL as evidence — not an empty dash.
-
zavora footnote uses the wrong URL.
https://github.com/zavora-ai/computer-use-macos404s;https://github.com/zavora-ai/computer-use-mcpreturns 200. Update the footnote (and PR body curl checklist) to the mcp repo, or drop zavora if you cannot verify platforms from that README.
Soft (non-blocking)
- Double-check the Cursor Cloud skill name
cursor-gpt-computer-useagainst current Cursor docs wording; if the public name differs, use the public name. - After fixes: mark the PR ready for review (draft is fine until then).
Not blocking: Drop Capture / keep Free sensing / do-not-claim list / follow-ups structure — good. No ADR/SPEC edits in this PR is correct.
— Cursor agent (Architect), on @omesser's behalf.
- Hermes landscape: correct to computer_use → cua-driver (first-party) - Codex: cite Computer Use plugin docs (host owns capture) - zavora: fix URL to computer-use-mcp (not computer-use-macos) - Cursor Cloud: use public 'computer use' naming (not skill name) - Update Gaps section with verified Facts - Remove stale Codex follow-up (now resolved) Co-authored-by: Oded Messer <odedmesser@gmail.com>
omesser
left a comment
There was a problem hiding this comment.
Architect — CLEARED @ d3a4ac13
All three required fixes landed:
- Hermes table + ACP curated caveat consistent with cua-driver section (vendor URL cited).
- Codex Computer Use plugin Fact with https://developers.openai.com/codex/computer-use.
- zavora footnote →
zavora-ai/computer-use-mcp(live).
Soft Cursor Cloud naming cleaned up. CI green; PR ready for review. Docs-only scope respected.
Oded may merge when ready. Follow-ups in the note (ADR-0005 supersession, consent copy scrub, optional CU guidance issue) stay out of this PR.
— Cursor agent (Architect), on @omesser's behalf.
omesser
left a comment
There was a problem hiding this comment.
Architect — CHANGES_REQUESTED (docs cleanliness)
Tip d3a4ac13 still has 25 accidental line-number prefixes ( 20|, 50||, …) baked into docs/research/capture-drop-and-harness-cu-path.md. GitHub renders them as literal text and breaks the Codex table row.
CLEAR requires:
- Tip has zero matches for
^[ \t]*[0-9]+\|in that file. - Unslop already requested (no em dashes; plain wording) still applied.
- How-to-verify boxes checked by agent with verification note (no human checklist).
A clean file was handed to the cloud agent; this gate stays until the tip proves clean.
— Cursor agent (Architect), on @omesser's behalf.
Critical fix: strip editor line-number prefixes ( N|) that were baked into markdown and would render literally on GitHub. Applied poteto-mode unslop: em dashes removed, **Fact.** period labels, landscape→options, niche→gap. Verified zero hits: grep '^[ \t]*[0-9]\+|' returns 0. Co-authored-by: Oded Messer <odedmesser@gmail.com>
omesser
left a comment
There was a problem hiding this comment.
Architect — CLEARED @ 99146c0b
Sanity check on tip (prove-it-works):
^[ \t]*[0-9]+\|→ 0 hits (line-number paste junk gone; Codex table row renders as a normal| Codex | … |line)- Em dashes → 0; no "landscape" / "architectural niche" leftover
- How-to-verify boxes are
[x]with agent Verification run note - Docs-only single file still holds
Oded may merge when ready. CI may still be settling on the new tip.
— Cursor agent (Architect), on @omesser's behalf.
…ure foreshadowing Option A locked: ScreenCast portal consent is for Wayland window titles (other apps' title / similar metadata), not Capture. Capture stays dropped (#878 / #881 / #883). Changes: - Retarget all user-facing copy: ScreenCast buys Wayland titles, not Capture - Update comments: #886 will consume usable(PortalScreenCast), not Capture - Scrub 'Capture when it ships' from Settings help, buys, pane intro - Update Cargo.toml comment to reference Wayland titles - Fix test assertions to look for 'Wayland' or 'title', not 'Capture' - Update dead_code comments to reference #886 Closes #883 (for this codebase — consent surface is scrubbed) Related #881 (ADR amendment tracks Capture drop decision) Related #886 (follow-up to implement titles consumer) Co-authored-by: Oded Messer <odedmesser@gmail.com>
Why
ai-buddy deferred pixel Capture in ADR-0005 but never documented the full decision to drop it, nor the recommended multi-harness computer-use path for users who need agent desktop control. This creates ambiguity about what sensing ships, what doesn't, and where agent CU belongs architecturally.
Decision context (Oded / Architect, 2026-09-22): KEEP Free sensing (OS metadata: frontmost app, window geometry, idle); DROP pixel Capture entirely; computer use for agents lives outside ai-buddy (harness-native or multi-harness MCP driver). Architect pick after comparison: cua-driver as primary (multi-OS, Hermes-native), minghinmatthewlam/computer-use-mcp as runner-up (macOS-only, AX-first).
What changed
Added
docs/research/capture-drop-and-harness-cu-path.mddocumenting:All claims labeled Fact. / Inference. / Assumption. Real URLs verified. Tone/structure matches
harness-tools-under-acp-probe.mdandharness-native-tools-under-acp.md: dated, explicit gaps, do-not-claim section.Deliberately did not: change ADRs, SPEC, product code, consent strings, or add research index (none exists). This PR is research note only.
Poteto-mode unslop applied (2026-09-22): em dashes removed, Fact. period labels, "landscape"→"options", "architectural niche"→"gap".
How to verify
Read the note:
Checklist:
Verify links resolve:
Docs-only:
git diff main --name-only # Returns: docs/research/capture-drop-and-harness-cu-path.mdVerification run (2026-09-22, Asia/Jerusalem timezone):
git diff main --name-onlyshows only the research note— Cursor agent, on @omesser's behalf.