Skip to content

chore(scripts): Verify the macOS Settings window through the Accessibility API - #626

Merged
omesser merged 4 commits into
mainfrom
chore/verify-settings-macos
Sep 11, 2026
Merged

omesser merged 4 commits into
mainfrom
chore/verify-settings-macos

Conversation

@omesser

@omesser omesser commented Sep 10, 2026 •

Copy link
Copy Markdown
Owner

Why

Checking AppKit Settings meant launching the app, opening the window, and staring at a screenshot. Freeze state was a guess until someone typed into a field. The Accessibility tree already holds the strings and booleans those checks ask for.

Scope

  • scripts/ax-settings.swift presses the tray Settings row, tabs, and popups by name, then dumps the tree as one line per control.
  • scripts/verify-settings-macos.sh runs two passes in throwaway HOMEs, one with no Harness and one with AI_BUDDY_HARNESS set, and greps that dump.

Does not change the Settings renderer. Does not add a CI job. Runtime source switch in one window is #635.

Tradeoffs

The freeze oracle is AXValue settability, not AXEnabled. freeze_or_bind uses setEditable(false), which leaves the field enabled and looking live. AppKit also demotes a frozen NSTextField to AXStaticText, so a matcher that waits for AXTextField walks into the next row.

The Harness pass waits to open Settings until the child logs attached. draw() re-applies freeze (#593), but a background attach does not draw.

Blast Radius

macOS only, and only when someone runs the script. CI does not. A wrong grep fails the script, not the product.

Verification

cargo build -p ai-buddy
./scripts/verify-settings-macos.sh

A green no-Harness run PASSes section order AI > AI source > Model / API > Last user turn, live Base URL and Model, the AI-brain state line, and no is the Director's mind. The Harness pass reports SKIP, not failure, when the named CLI is missing or not signed in. Grant Accessibility to the terminal that runs it.

— Cursor agent (Grok), on @omesser's behalf.

omesser and others added 3 commits September 11, 2026 18:55
…ing it

Verifying the AppKit Settings window meant launching, clicking the tray,
clicking a tab, taking a screenshot and looking at it - minutes per
observation, and the freeze state was a guess until someone typed into a
field. Every question the verification asks is a string or a boolean the
Accessibility tree already holds, so ask that instead: both passes,
including a live Harness attach, take under twenty seconds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A non-editable NSTextField is demoted from AXTextField to plain
AXStaticText, so the matcher walked past the frozen row and read the
next control it recognised - the disabled Clear key button, which
answers false and made the freeze checks pass for the wrong reason.

Take the line straight after the label whatever its role. Adds `pick`,
which changes a popup in place, for the states that only exist between
two launches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@omesser
omesser force-pushed the chore/verify-settings-macos branch from 7e10ea9 to 8364b1c Compare September 11, 2026 16:09
Co-authored-by: Cursor <cursoragent@cursor.com>
@omesser
omesser merged commit 0985f51 into main Sep 11, 2026
5 checks passed
@omesser
omesser deleted the chore/verify-settings-macos branch September 11, 2026 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

p2 Eventually: real, but nothing waits on it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant