Skip to content

test(macos): Settings e2e + fill AppKit renderer gaps #629

Closed

Description

@omesser

Why

Native Settings is three renderers. Form data (form.rs / settings.rs) is well unit-tested; AppKit is not. Recent bugs (#625 freeze-on-source-switch, #593 disclosure, AI-source reorder) only showed up in a live window. We need a diligent macOS pass: missing automated tests and a real e2e run, not screenshots as the only oracle.

Sister issues: Linux GTK (this epic’s pair), Windows Win32 (this epic’s pair). Shared matrix below; this ticket owns AppKit only.

Related: #625 (runtime freeze — P0-class; #593 is fixing draw()), #548 / #593 (progressive disclosure), #626 (verify-settings-macos.sh AX script — land/extend, do not duplicate).

Two workstreams

1. Complete missing tests (automated)

Today: platform/macos/settings_window.rs has a tiny mod tests (a couple of cases). settings.rs / form.rs tests do not prove the renderer applied frozen, disclosure, or tab chrome.

Add (compile-time / unit, no GUI where possible):

  • draw() / apply_enabled_states (or equivalent) re-applies frozen from a fresh form::describe() — freeze and unfreeze. Table-driven: driving Harness vs Model API. This is the Settings rows never re-freeze after the AI source changes #625 regression lock.
  • Frozen vs live: HTTP rows (DIRECTOR_BASE_URL_ID, DIRECTOR_MODEL_ID, DIRECTOR_API_KEY_ID, base-url picker, Clear key) match http_row_parts.
  • AI tab section order from director_sections: AI → AI source → Model / API → Last user turn (lock feat(settings): Progressive disclosure data structures for everyday Settings #593 reorder).
  • Popup titles: Model API / Harness · {preset} / Harness · Custom via harness_options / harness_choice (already in form tests — assert the AppKit popup is filled from those, not a stale "Off" / bare "claude").
  • status / disclosure rows: if feat(settings): Progressive disclosure data structures for everyday Settings #593 paints them, assert they exist in the view tree; if not yet, skip with a comment pointing at the follow-up rather than a false green.
  • Env-owned rows stay frozen (AI_BUDDY_* override) and do not unfreeze on source switch.

Extend #626 / scripts/verify-settings-macos.sh (AX tree, not pixels) so it covers the runtime switch, not only launch-with-harness-already-attached:

  • Open Settings, Harness driving → HTTP rows AXEnabled=false (or equivalent).
  • Switch AI source to Model API in that same window → rows become enabled without relaunch (this is the Settings rows never re-freeze after the AI source changes #625 e2e).
  • Switch back to a Harness → rows freeze again.
  • AI tab section headings order.
  • Stills saved for clip-only failures (AX cannot see truncated labels).

2. Live e2e verification (must run on a Mac)

Build tip of the branch under test (cargo build / packaged debug). Throwaway HOME. Dual-display not required unless a control is display-specific.

Walk every tab (Presence, Character, AI, Privacy, Development). For each:

  • Window opens from tray / menu; close and reopen (hidden, not rebuilt — freeze must still be correct).
  • All sections reachable (no clip without a way to scroll).
  • Labels persist after switching away and back (Win32 had label-wipe; confirm AppKit does not).
  • Checkboxes, text, secure field, popups, Apply/Cancel actually commit (settings.json / keychain / live Director retarget as DEVELOPMENT.md describes).
  • AI source: Model API vs Harness · claude (or another named row) — freeze/unfreeze without relaunch.
  • Short vs truncated Speech is out of scope except Open-chat if it is on this tab (it is not).
  • Development still on the strip (feat(settings): Progressive disclosure for everyday Settings copy #548 out of scope to hide it).
  • Env override: launch with AI_BUDDY_WAKE_SECS set → First wake frozen + status/copy names the var (once strips paint).

File bugs found as separate issues (do not silently expand this ticket into a rewrite). Link them here.

Done when

  • Automated tests above are in CI for macos-latest.
  • verify-settings-macos.sh (or successor) is documented in DEVELOPMENT.md and covers runtime source switch.
  • A comment on this issue records a live pass (date, macOS version, SHA) with the checklist ticked, or links filed bugs for each fail.

No merge of a Settings UX PR that claims freeze/unfreeze without the AX runtime-switch check going green.

— Cursor agent (Architect), on @omesser's behalf.

Activity

  1. added
    ready-for-agentFully specified, ready for an AFK agent
    p1Next up: the default for work that should happen
    macosmacos / OSX specific
    on Sep 11, 2026
  2. added this to the V1 milestone on Sep 11, 2026
  3. omesser commented on Sep 11, 2026

    @omesser
    OwnerAuthor

    Sisters: Linux #630, Windows #631. Freeze bug #625; AX script #626; Settings PR #593.

    — Cursor agent (Architect), on @omesser's behalf.

  4. self-assigned this
    on Sep 11, 2026
  5. omesser commented on Sep 11, 2026

    @omesser
    OwnerAuthor

    Claiming this ticket. Stacking on #626 (chore/verify-settings-macos) so the AX script is extended rather than duplicated. AppKit unit tests and a runtime AI-source switch pass are the scope. #593 disclosure stays a skip-with-comment until that PR paints the rows.

    — Cursor agent (Grok), on @omesser's behalf.

  6. moved this from 👷 Todo to 🏗 In Progress in Fidgeton Sep 11, 2026
  7. omesser commented on Sep 11, 2026

    @omesser
    OwnerAuthor

    Live macOS pass for this ticket

    Date: Fri Sep 11 2026, 17:15 IDT. sw_vers: macOS 26.6.2 (25G83). Branch test/629-settings-macos-e2e @ 83f33f4b. Binary: worktree cargo build -p ai-buddy then ./scripts/verify-settings-macos.sh.

    Evidence: .verify/settings-macos-20260911-171612/ in that worktree (dumps + stills). Not committed.

    What ran

    Unit tests (no AppKit widgets; test_mtm is unsound for setEditable):

    • freeze_intents_follow_a_fresh_describe_for_http_rows
    • env_owned_http_rows_stay_frozen_on_a_second_describe
    • ai_tab_section_headings_match_director_sections
    • harness_popup_is_fed_off_presets_and_custom

    Red: missing freeze_intents / harness_popup_fill (error[E0425]). Green: cargo test -p ai-buddy --bin ai-buddy -- settings_window → 7 passed.

    draw() now calls apply_enabled_states from a fresh form::describe(), using description.frozen(id) (no row_frozen).

    AX script outcome

    • PASS: Presence, Character, AI, Privacy, Development dumps. Section order AI > AI source > Last user turn. Base URL, Model, and the endpoint picker live with no Harness. Those three still live after the Off pick (they were never frozen).
    • FAIL (2): state line does not say "AI brain"; "is the Director's mind" is still there. Copy left alone; filed Settings AI source state line still says mind, not AI brain #633.
    • SKIP (2): claude never answered (set but not running), so the driving freeze/unfreeze lock did not run. Second pick Harness claude failed (popup has no option claude) after Off left the control on claude; filed macOS Settings Off pick does not leave the Harness popup #634.

    I am not claiming the runtime freeze switch passed. A signed-in Harness that attaches is still required for that.

    Commits on this branch: af9e7e75 test freeze intents, 15bd421a re-apply on draw, 022da861 in-window AX switch, 83f33f4b docs.

    — Cursor agent (Grok), on @omesser's behalf.

  8. moved this from 🏗 In Progress to 🏁 Done in Fidgeton Sep 11, 2026
  9. omesser commented on Sep 22, 2026

    @omesser
    OwnerAuthor

    #706 Step 9 deletes the AppKit Settings renderer this e2e epic owned.

    The ticket is already closed via #635. Recording here so it is not reopened against a window that no longer exists: Settings is the webview, and scripts/verify-settings-macos.sh is gone. macOS sittings are the webview scripts (verify-settings-keyboard-webview.sh and the other verify-settings-webview-*).

    #871

    — Grok Build agent, on @omesser's behalf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

macosmacos / OSX specificp1Next up: the default for work that should happen

Projects

  • Status
    Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions