Repository navigation
P2: Select a browser element and attach precise UI context to the composer #1193
Description
Outcome: point at a broken button or spacing problem in the shared Browser pane and tell the agent what to change without describing its location.
Evidence: Orca Design Mode documents element selection with HTML, computed styles, a cropped screenshot and source location when available. Solenta's src/components/BrowserPane.tsx:48 offers screenshot attachment but no element picker. electron/preview.js:160 reports URL/title/navigation state; click/type take CSS selectors. Inspected at commit 9fc8645a2472e493a94c2e4c4c8a942974bff305.
Smallest scope: an opt-in Inspect toggle; hover highlight; click to select without activating the page; a bounded context chip containing page URL, element tag/role/name, selector hint, selected layout/style properties and screenshot crop. Let the user add a comment, then insert into the existing composer. No automatic send. Support one selected element first.
Acceptance:
- Selecting a submit button does not submit its form; Escape exits inspection.
- The attachment stays bound to the originating thread, page and capture time even if selection changes.
- Preview the exact content before send; exclude password fields, hidden input values, cookies, storage and tokens. Treat captured DOM as untrusted context.
- Cap DOM/style payload and image size. Report unsupported cross-origin frames or unavailable source locations honestly.
- Keyboard selection or an accessible alternative exists; narrow splits remain usable.
- A mounted interaction check covers select/cancel/send; a packaged Browser-pane check verifies the crop and coordinates.
Scope boundary: explicit residual follow-up to #155, which already mentions element annotations but is marked plan:done after the preview shipped. Keep that parent link; do not present this as a previously untracked idea. #460 covers drawing on a still screenshot; this is live element context. No computer-use toggle or desktop control.
Activity
currentbits commentedon Oct 9, 2026
Still open on main @24d3c36b:
BrowserPane.tsx:51can attach a screenshot, but there is no element picker. Ranked #5 in the 2026-10-09 gap pass, and high value for frontend work. Include the element's selector, its outer HTML (trimmed), computed box and the source file when a dev server exposes it.currentbits commentedon Oct 15, 2026
Implemented on branch
coder/p2-select-a-browser-element-and-attach-p-ef93f1. It is uncommitted and not yet merged. Follow-up to #155.elementcontext chip on the originating thread. The chip freezes the thread, URL and capture time. Clicking the chip's label in the composer shows the exact text again. Nothing sends automatically.electron/test/preview-inspect.test.js: the real guest script under jsdom, plus the main-process plumbing;test/browserInspect.test.tsx: a mounted select, cancel and send flow;npm run verify:browser-inspect: real Electron, checking the rect, crop rect and crop pixels at zoom 1 and 1.5, and Escape. This runs in dev Electron, not inside the packaged .app, and is not wired into CI.currentbits commentedon Oct 15, 2026
Landed from Solenta thread "P2: Select a browser element and attach precise UI context t" (PR #1644). Closed on merge.