Repository navigation
Simplify reconnect error message - #9
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219
added a commit
that referenced
this pull request
Mar 31, 2026
agent-api.py: - Add _safe_id() to sanitize task IDs against path traversal (#2-4, #8) - Reject path traversal in /media/ before resolve (#5-7) - Sanitize mime type to prevent HTTP response splitting (#1) inline-tools.ts: - Escape backslashes in shell args for openUrl and switchApp (#9-11) conversation-server.ts: - Mask phone number in startup log (#12) - Escape meetingId/passcode in TwiML Play digits (#13) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi
pushed a commit
that referenced
this pull request
Mar 31, 2026
* Fix 13 CodeQL security alerts agent-api.py: - Add _safe_id() to sanitize task IDs against path traversal (#2-4, #8) - Reject path traversal in /media/ before resolve (#5-7) - Sanitize mime type to prevent HTTP response splitting (#1) inline-tools.ts: - Escape backslashes in shell args for openUrl and switchApp (#9-11) conversation-server.ts: - Mask phone number in startup log (#12) - Escape meetingId/passcode in TwiML Play digits (#13) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add security comments to shell escaping in inline-tools Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219
pushed a commit
that referenced
this pull request
Apr 18, 2026
Per Susan's 'general edit to correct outdated descriptions' ask. Applied Maddy's review hits that I could verify locally: - pkill pattern: `-f "src/Sutando"` → `-x Sutando` (hit #8: safer, matches the actual process name regardless of launch path). - Uninstall step 5: softened from `brew uninstall imsg wacli` to a generic 'remove via the package manager you used' (hit #3: wacli isn't necessarily brew). - Capabilities table: - Task delegation: `task-bridge.ts` → `task-bridge.ts + watch-tasks.sh + tasks/` (hit #4: reflects the actual file-based flow). - Notes: 'via CLAUDE.md' → '`notes/` directory (YAML-frontmatter markdown)' (hit #5: notes/ is the real store). - Added rows for cross-node sync, info-radar, menu-bar avatar states (hit #6: recent features missing from the matrix). - Services table: added 7845 (screen-capture), 3100 (conversation-server), 4040 (ngrok admin UI) (hit #7). - Removed stray duplicate `---` separator after the services table (hit #9). Skipped: - Hit #1 (L77 subscription line) — already removed earlier. - Hit #2 (screencapture note) — Maddy confirmed still accurate. - Hit #10 (swiftc missing `-framework AVFoundation`) — main.swift does NOT import AVFoundation (verified: import Cocoa, Carbon, UserNotifications only). Current compile line is correct. - L101 permission rewrite (Maddy's 4-bullet proposal) — substantive rewrite, deferred to Susan's specific approval before applying. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
liususan091219
added a commit
that referenced
this pull request
Apr 19, 2026
* docs(README): update 'Summon' voice trigger example to 'Summon my computer to zoom' The short form 'Summon' is false-positive-prone in voice transcripts. Owner uses the longer 'Summon my computer to zoom' phrase in practice. This docs update matches what actually triggers the Zoom summon reliably. Code-side prompts are NOT changed in this PR (per CLAUDE.md's 'prompts tuned through testing' rule) — the underlying summonTool already matches both phrases via the 'summon' keyword. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): add two troubleshooting entries (Claude context drift + phone-server error) Per Susan's Discord msg: two common failure modes worth surfacing in Troubleshooting: 1. Claude hallucinates more as context fills → tell users to restart the Claude Code session periodically. 2. Phone call answers 'We are sorry, an error has occurred' → the conversation-server on port 3100 isn't running; relaunch via startup.sh or restart.sh. Verified the correct server path: skills/phone-conversation/scripts/ conversation-server.ts (not a standalone start-phone-server.sh, which doesn't exist). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): apply Maddy's verified hits — general cleanup Per Susan's 'general edit to correct outdated descriptions' ask. Applied Maddy's review hits that I could verify locally: - pkill pattern: `-f "src/Sutando"` → `-x Sutando` (hit #8: safer, matches the actual process name regardless of launch path). - Uninstall step 5: softened from `brew uninstall imsg wacli` to a generic 'remove via the package manager you used' (hit #3: wacli isn't necessarily brew). - Capabilities table: - Task delegation: `task-bridge.ts` → `task-bridge.ts + watch-tasks.sh + tasks/` (hit #4: reflects the actual file-based flow). - Notes: 'via CLAUDE.md' → '`notes/` directory (YAML-frontmatter markdown)' (hit #5: notes/ is the real store). - Added rows for cross-node sync, info-radar, menu-bar avatar states (hit #6: recent features missing from the matrix). - Services table: added 7845 (screen-capture), 3100 (conversation-server), 4040 (ngrok admin UI) (hit #7). - Removed stray duplicate `---` separator after the services table (hit #9). Skipped: - Hit #1 (L77 subscription line) — already removed earlier. - Hit #2 (screencapture note) — Maddy confirmed still accurate. - Hit #10 (swiftc missing `-framework AVFoundation`) — main.swift does NOT import AVFoundation (verified: import Cocoa, Carbon, UserNotifications only). Current compile line is correct. - L101 permission rewrite (Maddy's 4-bullet proposal) — substantive rewrite, deferred to Susan's specific approval before applying. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): revise 'How it works' diagram — phone agent + accurate process topology Per Susan's ask ('diagram isn't quite right', 'phone agent rarely covered') and Maddy's accuracy verification: Diagram changes: - Web client → Voice agent is now WebSocket on :9900 (was implicit file bridge) - Phone (Twilio) → Conversation server added as peer entry path - results/ flow annotated as per-channel (speak via voice/phone vs text via Telegram/Discord), not one unified speak path Process topology changes: - "Two processes work together" → "Three processes" (voice-agent, conversation-server, core-agent) — verified against conversation-server.ts by Maddy (writes to tasks/ directly at lines 289, 964, 1315; not hierarchically downstream of voice-agent). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): surface phone-agent in prereqs + services list Per Susan's ask ('phone agent is rarely covered in the README, it should be mentioned when appropriate'): - Prereqs: added optional Twilio + ngrok bullet so readers know the phone path exists and what they need to enable it. - 'This starts all services' sentence: added 'phone conversation server' to the list so readers see it's a default-started daemon, not an opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): rewrite permission section (4-bullet 'why elevated perms') per Maddy's proposal Replaces the conservative CYA note ('you should review what it does. All actions are logged') with Maddy's framing that tells the reader: 1. It's local (no remote control plane) 2. You control the audience (3-tier access gating + VERIFIED_CALLERS) 3. Actions are auditable (build_log + tasks/ + results/ + logs/) 4. Hooks are your brake pedal (git-rules-guard.sh Discord 👍/👎) Susan's direction was 'lead the work and Maddy please review iteratively'. Applying Maddy's already-drafted proposal; she reviews next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): add inline-tools path to 'How it works' diagram Per Susan's ask — voice agent and conversation server have two response paths (inline tools for instant in-process calls; tasks/ for anything the core agent needs to execute), while Telegram/Discord bridges only use tasks/. — Lucy (Mac Studio bot) * docs(README): swap 'mute' for real inline tool per Maddy's review mute is a menu-bar ⌃M shortcut, not an inline tool. Replace with get_current_time (src/inline-tools.ts:376) and broaden the prose sentence to list only genuine inline tools. — Lucy (Mac Studio bot) * docs(README): add running-costs section + price-aware prereqs Per Susan's in-#dev review (consolidated 4-edit bundle, 👍'd): - Line 96: drop unqualified "free" on API key — usage-billed per Running costs - Line 97: add ~$0.01–0.03/min Twilio cost reference - New "## Running costs" section: Gemini Live $0.30–$1.30/hr, Twilio $0.0125/min (US local), free for browser/Telegram/Discord - New ffmpeg bullet in prereqs (optional, for video/audio) Sources: Gemini pricing docs, Twilio voice pricing docs. — Lucy (Mac Studio bot) * docs(README): bundle 4 follow-ups from #susan iteration Per Susan's 4× 👍 iterative review on PR #450: - Diagram: add `↻` on Core agent + legend explaining the cron-driven proactive loop (5-min fires for tasks, health, build-log work). - Running costs table: Twilio row now bidirectional — ~$0.013–$0.018/min covers inbound ($0.0085) + outbound ($0.014) + Media Streams ($0.004). - Prereq bullet: rename "Google AI Studio API key" → "Gemini API key" and inline the ~$0.30–$1.30/hour voice cost. - macOS permissions: moved from inline Quick start bullets to the Security section with a per-permission "why it's needed" expansion; Quick start keeps a one-line pointer. — Lucy (Mac Studio bot) * docs(README): hedge Gemini price claims — free tier vs paid unclear Susan and Chi are still sorting out the free-tier vs paid-tier picture; pull back the specific $0.30-$1.30/hr claim since it misrepresents the free-tier case: - Prereq bullet: "may be free on the free tier or usage-billed depending on your Google Cloud account" instead of a rate - Running costs table Gemini row: "Free tier available - paid usage varies"; notes keep the published paid rates as reference but point readers at their billing dashboard for ground truth - Drop the personal $13/602-min datapoint from the typical-call example (may conflate with non-voice Gemini spend) Twilio row unchanged - that math is from published rates. — Lucy (Mac Studio bot) * docs(README): Option B — remove Running costs section + price mentions Per Susan's "B" direction in #dev: the free-tier vs paid-tier picture for Gemini isn't settled, and leaving any price language in the PR risks misleading first-time readers. - Delete the entire "## Running costs" section (table + example + sources line). - Revert the Gemini API key prereq bullet to `(click "Get API key")` — drop the usage-billed caveat entirely. - Revert the Twilio prereq bullet to drop the `~$0.01–0.03/min` price reference. ffmpeg bullet, diagram rewrite, macOS permissions in Security section, `↻` cron legend, phone-agent surfacing — all stay. — Lucy (Mac Studio bot) --------- Co-authored-by: Xueqing Liu <xueqingliu@Xueqings-Mac-Studio.local> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
liususan091219
pushed a commit
to liususan091219/sutando-fork
that referenced
this pull request
May 28, 2026
…onichi#9) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219
added a commit
to liususan091219/sutando-fork
that referenced
this pull request
May 28, 2026
* Fix 13 CodeQL security alerts agent-api.py: - Add _safe_id() to sanitize task IDs against path traversal (sonichi#2-4, sonichi#8) - Reject path traversal in /media/ before resolve (sonichi#5-7) - Sanitize mime type to prevent HTTP response splitting (sonichi#1) inline-tools.ts: - Escape backslashes in shell args for openUrl and switchApp (sonichi#9-11) conversation-server.ts: - Mask phone number in startup log (sonichi#12) - Escape meetingId/passcode in TwiML Play digits (sonichi#13) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add security comments to shell escaping in inline-tools Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219
added a commit
to liususan091219/sutando-fork
that referenced
this pull request
May 28, 2026
* docs(README): update 'Summon' voice trigger example to 'Summon my computer to zoom' The short form 'Summon' is false-positive-prone in voice transcripts. Owner uses the longer 'Summon my computer to zoom' phrase in practice. This docs update matches what actually triggers the Zoom summon reliably. Code-side prompts are NOT changed in this PR (per CLAUDE.md's 'prompts tuned through testing' rule) — the underlying summonTool already matches both phrases via the 'summon' keyword. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): add two troubleshooting entries (Claude context drift + phone-server error) Per Susan's Discord msg: two common failure modes worth surfacing in Troubleshooting: 1. Claude hallucinates more as context fills → tell users to restart the Claude Code session periodically. 2. Phone call answers 'We are sorry, an error has occurred' → the conversation-server on port 3100 isn't running; relaunch via startup.sh or restart.sh. Verified the correct server path: skills/phone-conversation/scripts/ conversation-server.ts (not a standalone start-phone-server.sh, which doesn't exist). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): apply Maddy's verified hits — general cleanup Per Susan's 'general edit to correct outdated descriptions' ask. Applied Maddy's review hits that I could verify locally: - pkill pattern: `-f "src/Sutando"` → `-x Sutando` (hit sonichi#8: safer, matches the actual process name regardless of launch path). - Uninstall step 5: softened from `brew uninstall imsg wacli` to a generic 'remove via the package manager you used' (hit sonichi#3: wacli isn't necessarily brew). - Capabilities table: - Task delegation: `task-bridge.ts` → `task-bridge.ts + watch-tasks.sh + tasks/` (hit sonichi#4: reflects the actual file-based flow). - Notes: 'via CLAUDE.md' → '`notes/` directory (YAML-frontmatter markdown)' (hit sonichi#5: notes/ is the real store). - Added rows for cross-node sync, info-radar, menu-bar avatar states (hit sonichi#6: recent features missing from the matrix). - Services table: added 7845 (screen-capture), 3100 (conversation-server), 4040 (ngrok admin UI) (hit sonichi#7). - Removed stray duplicate `---` separator after the services table (hit sonichi#9). Skipped: - Hit sonichi#1 (L77 subscription line) — already removed earlier. - Hit sonichi#2 (screencapture note) — Maddy confirmed still accurate. - Hit sonichi#10 (swiftc missing `-framework AVFoundation`) — main.swift does NOT import AVFoundation (verified: import Cocoa, Carbon, UserNotifications only). Current compile line is correct. - L101 permission rewrite (Maddy's 4-bullet proposal) — substantive rewrite, deferred to Susan's specific approval before applying. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): revise 'How it works' diagram — phone agent + accurate process topology Per Susan's ask ('diagram isn't quite right', 'phone agent rarely covered') and Maddy's accuracy verification: Diagram changes: - Web client → Voice agent is now WebSocket on :9900 (was implicit file bridge) - Phone (Twilio) → Conversation server added as peer entry path - results/ flow annotated as per-channel (speak via voice/phone vs text via Telegram/Discord), not one unified speak path Process topology changes: - "Two processes work together" → "Three processes" (voice-agent, conversation-server, core-agent) — verified against conversation-server.ts by Maddy (writes to tasks/ directly at lines 289, 964, 1315; not hierarchically downstream of voice-agent). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): surface phone-agent in prereqs + services list Per Susan's ask ('phone agent is rarely covered in the README, it should be mentioned when appropriate'): - Prereqs: added optional Twilio + ngrok bullet so readers know the phone path exists and what they need to enable it. - 'This starts all services' sentence: added 'phone conversation server' to the list so readers see it's a default-started daemon, not an opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): rewrite permission section (4-bullet 'why elevated perms') per Maddy's proposal Replaces the conservative CYA note ('you should review what it does. All actions are logged') with Maddy's framing that tells the reader: 1. It's local (no remote control plane) 2. You control the audience (3-tier access gating + VERIFIED_CALLERS) 3. Actions are auditable (build_log + tasks/ + results/ + logs/) 4. Hooks are your brake pedal (git-rules-guard.sh Discord 👍/👎) Susan's direction was 'lead the work and Maddy please review iteratively'. Applying Maddy's already-drafted proposal; she reviews next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): add inline-tools path to 'How it works' diagram Per Susan's ask — voice agent and conversation server have two response paths (inline tools for instant in-process calls; tasks/ for anything the core agent needs to execute), while Telegram/Discord bridges only use tasks/. — Lucy (Mac Studio bot) * docs(README): swap 'mute' for real inline tool per Maddy's review mute is a menu-bar ⌃M shortcut, not an inline tool. Replace with get_current_time (src/inline-tools.ts:376) and broaden the prose sentence to list only genuine inline tools. — Lucy (Mac Studio bot) * docs(README): add running-costs section + price-aware prereqs Per Susan's in-#dev review (consolidated 4-edit bundle, 👍'd): - Line 96: drop unqualified "free" on API key — usage-billed per Running costs - Line 97: add ~$0.01–0.03/min Twilio cost reference - New "## Running costs" section: Gemini Live $0.30–$1.30/hr, Twilio $0.0125/min (US local), free for browser/Telegram/Discord - New ffmpeg bullet in prereqs (optional, for video/audio) Sources: Gemini pricing docs, Twilio voice pricing docs. — Lucy (Mac Studio bot) * docs(README): bundle 4 follow-ups from #susan iteration Per Susan's 4× 👍 iterative review on PR sonichi#450: - Diagram: add `↻` on Core agent + legend explaining the cron-driven proactive loop (5-min fires for tasks, health, build-log work). - Running costs table: Twilio row now bidirectional — ~$0.013–$0.018/min covers inbound ($0.0085) + outbound ($0.014) + Media Streams ($0.004). - Prereq bullet: rename "Google AI Studio API key" → "Gemini API key" and inline the ~$0.30–$1.30/hour voice cost. - macOS permissions: moved from inline Quick start bullets to the Security section with a per-permission "why it's needed" expansion; Quick start keeps a one-line pointer. — Lucy (Mac Studio bot) * docs(README): hedge Gemini price claims — free tier vs paid unclear Susan and Chi are still sorting out the free-tier vs paid-tier picture; pull back the specific $0.30-$1.30/hr claim since it misrepresents the free-tier case: - Prereq bullet: "may be free on the free tier or usage-billed depending on your Google Cloud account" instead of a rate - Running costs table Gemini row: "Free tier available - paid usage varies"; notes keep the published paid rates as reference but point readers at their billing dashboard for ground truth - Drop the personal $13/602-min datapoint from the typical-call example (may conflate with non-voice Gemini spend) Twilio row unchanged - that math is from published rates. — Lucy (Mac Studio bot) * docs(README): Option B — remove Running costs section + price mentions Per Susan's "B" direction in #dev: the free-tier vs paid-tier picture for Gemini isn't settled, and leaving any price language in the PR risks misleading first-time readers. - Delete the entire "## Running costs" section (table + example + sources line). - Revert the Gemini API key prereq bullet to `(click "Get API key")` — drop the usage-billed caveat entirely. - Revert the Twilio prereq bullet to drop the `~$0.01–0.03/min` price reference. ffmpeg bullet, diagram rewrite, macOS permissions in Security section, `↻` cron legend, phone-agent surfacing — all stay. — Lucy (Mac Studio bot) --------- Co-authored-by: Xueqing Liu <xueqingliu@Xueqings-Mac-Studio.local> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
yixuan-ag2
added a commit
that referenced
this pull request
Aug 14, 2026
…tures
Implements plan step 3 [A] of design-voice-p7-implementation.md (D7.2):
- src/voice-health-matrix.ts — a pure function over the engine ledger
snapshot with HONEST outcomes. Row 0: RECONNECTING/CONNECTING is a
reconnect-window; no heartbeat or no prior baseline is
insufficient-evidence, never a guess. Row 1 (client capture dead)
requires a CURRENT-epoch, unexpired gap — the open gap or a closed
episode whose interval overlaps the ingress stall — while unmuted +
connected (an old latched gap can never pair with unrelated later
silence), and discriminates the S1/S2 shapes (ctx-suspended /
post-suspension / device-recovery / main-thread-gap) from the fields
that travel. Rows 2-4 (native server dispositions) are structurally
unreachable at coverage session-only: their observable half surfaces
as insufficient-evidence naming the unobserved hop. Row 5
(model-silent, the F4 residue) is likewise gated on full coverage —
at session-only, speech-without-model-event names the blind upstream
hop instead of blaming the model. Row 5' healthy-idle: a quietly
listening user is not an incident. Row 6 fires on transmitted
evidence (bufferedAmount growth or a sendSkipped delta); row 7a needs
the server-side buffered input or stays silent; row 7b requires
scheduling to advance with zero NATURAL completions and a frozen ctx
clock (cancellations excluded by construction — barge-in is not a
playback failure).
- parseHeartbeat now decodes the full episode tuples (gap
{startMs, durationMs}, speech {onsetSeq, offsetSeq, maxRmsPm,
aboveFloorMs}) — the matrix needs intervals, not ids. The ledger
tracks epochStartApproxMs (receipt-time approximation, labeled) so
client-epoch-relative intervals can be placed on the engine clock,
and carries lastMatrixVerdict into every persisted row.
- voice-agent.ts: the 30 s tick evaluates the matrix (lastEgressAt as
the Tranche-A model-event proxy, named), logs verdict transitions,
and notes the verdict for persistence.
- FE-1/FE-2 are encoded as HYPOTHESIS-labeled fixtures (round-1 #9:
the incident counters never existed) to be validated against captured
S1/S2 runs before promotion; FE-3 belongs to D7.6-S3, not the matrix.
Tests: 20 matrix cases covering outcome totality, the
partial-coverage-never-yields-a-layer-verdict rule, row-1 overlap +
unmuted gates and shape discrimination, rows 6/7a/7b including the
cancellation exclusion, healthy-idle, and both hypothesis fixtures.
yixuan-ag2
added a commit
that referenced
this pull request
Aug 14, 2026
…tures
Implements plan step 3 [A] of design-voice-p7-implementation.md (D7.2):
- src/voice-health-matrix.ts — a pure function over the engine ledger
snapshot with HONEST outcomes. Row 0: RECONNECTING/CONNECTING is a
reconnect-window; no heartbeat or no prior baseline is
insufficient-evidence, never a guess. Row 1 (client capture dead)
requires a CURRENT-epoch, unexpired gap — the open gap or a closed
episode whose interval overlaps the ingress stall — while unmuted +
connected (an old latched gap can never pair with unrelated later
silence), and discriminates the S1/S2 shapes (ctx-suspended /
post-suspension / device-recovery / main-thread-gap) from the fields
that travel. Rows 2-4 (native server dispositions) are structurally
unreachable at coverage session-only: their observable half surfaces
as insufficient-evidence naming the unobserved hop. Row 5
(model-silent, the F4 residue) is likewise gated on full coverage —
at session-only, speech-without-model-event names the blind upstream
hop instead of blaming the model. Row 5' healthy-idle: a quietly
listening user is not an incident. Row 6 fires on transmitted
evidence (bufferedAmount growth or a sendSkipped delta); row 7a needs
the server-side buffered input or stays silent; row 7b requires
scheduling to advance with zero NATURAL completions and a frozen ctx
clock (cancellations excluded by construction — barge-in is not a
playback failure).
- parseHeartbeat now decodes the full episode tuples (gap
{startMs, durationMs}, speech {onsetSeq, offsetSeq, maxRmsPm,
aboveFloorMs}) — the matrix needs intervals, not ids. The ledger
tracks epochStartApproxMs (receipt-time approximation, labeled) so
client-epoch-relative intervals can be placed on the engine clock,
and carries lastMatrixVerdict into every persisted row.
- voice-agent.ts: the 30 s tick evaluates the matrix (lastEgressAt as
the Tranche-A model-event proxy, named), logs verdict transitions,
and notes the verdict for persistence.
- FE-1/FE-2 are encoded as HYPOTHESIS-labeled fixtures (round-1 #9:
the incident counters never existed) to be validated against captured
S1/S2 runs before promotion; FE-3 belongs to D7.6-S3, not the matrix.
Tests: 20 matrix cases covering outcome totality, the
partial-coverage-never-yields-a-layer-verdict rule, row-1 overlap +
unmuted gates and shape discrimination, rows 6/7a/7b including the
cancellation exclusion, healthy-idle, and both hypothesis fixtures.
yixuan-ag2
added a commit
that referenced
this pull request
Aug 15, 2026
…ion egress controls, capture recovery (#2902) * feat(voice): P7 step 1 — audio-progress ledger client hop + capture recovery FSM Implements plan step 1 [A] of design-voice-p7-implementation.md (D7.1 client hop, D7.5 recovery FSM, §D7.0b budget) in the canonical transport: - Epoch-scoped counters: capCallbacks, sendSkipped, sendFailed, chunksScheduled (renamed playChunkCount), chunksEnded split from chunksCancelled (flushPlayback stop() fires onended too — cancellation is not completion), lastEndedAt, scheduled depth, ctx clock. All reset per connect() so evidence never spans two calls. - Frame-path speech evidence (client corroboration): allocation-free subsampled RMS (every 4th sample) + latched speech intervals {onsetSeq, offsetSeq, maxRms, aboveFloorMs} with a 600 ms hangover. - Capture-gap watchdog: stall past max(3× frame interval, 1 s) while unmuted + connected opens a gap on the 500 ms tick; the episode latches into a preallocated 8-slot ring when capture returns, so a fast resume cannot erase it; a permanent stall travels as the open gap (og) in the heartbeat. - audio_health heartbeat every 4th stats tick (2 s), first on the first tick: compact wire schema with per-connection nonce (Tranche A epoch mapping — the engine mints the server epoch on first sight), absolute counters, idempotent last-4 episode window (ACK-free; server dedups by id), episodeOverflow for evidence loss, bufferedAmount current/high-water, hard ≤300 B/frame by construction (window trims oldest-first until the frame fits). - Egress backpressure: bufferedAmount > 256 KiB skips the frame into sendSkipped instead of deepening a stalled socket (FE-1); ws.send wrapped in try/catch (sendFailed) so a dying socket can't throw off the audio graph. - Recovery FSM: independent captureGen nested under attemptGen (bumps on every attempt, timeout, and teardown; all async capture continuations validate both gens), single-flight with reacquire escalation, statechange→resume (≤3 attempts, 0/1s/4s backoff, bounded timeout), input-filtered devicechange (enumerateDevices fingerprint) and track-ended → reacquire, listeners torn down in teardownAudio. Exhaustion → degraded via new onCaptureHealth (NOT onConnectFailure: socket + voice lease stay live) with retryCapture() as the UI retry affordance. - onStats extended additively (VoiceStats); existing consumers keep reading byte totals. §D7.0b: frame paths do only O(1) preallocated-field writes + bounded arithmetic; heartbeat assembly/serialization rides the existing 500 ms stats timer; an A/B budget test asserts the instrumented handler adds ≤50 µs p99 over a pinned-replica baseline. Tests: 20 new cases (90 total in the transport suite; full repo suite 897 pass / 1 pre-existing skip). One existing R10 precondition updated: the captured stream now publishes to micStream only after the resume fence (strictly safer — no transient stale publication). * fix(voice): apply Codex round-1 review to P7 step 1 All seven findings were real; each is fixed with a regression test: 1. Gap erasure (reproduced): an outage that starts and ends between watchdog ticks — the frozen-thread case the design names — left no episode, because only the timer opened gaps and the resumed callback overwrote lastCapAt. The frame path now self-latches a stall-sized delta before overwriting (O(1) preallocated-slot writes). The gap clock also arms at graph-wire time (a capture whose first callback never fires now stalls visibly), and the watchdog stays armed while recovery is in flight (a reacquire outage tears the processor down). 2. Escalation drop (reproduced): a reacquire requested during an in-flight resume was discarded when the resume succeeded, leaving the dead track in place while reporting recovered. The loop now re-checks the escalate flag after success and continues into a reacquire. The loop is failure-counted (an escalated pass after a successful resume does not burn an attempt), with an iteration belt of 10. 3. Dishonest reacquire (reproduced): a reacquire onto a context stuck suspended wired the graph and reported recovered while capture stayed dead. The verdict now requires ctx.state === 'running'. 4. Single-flight not attempt-scoped: recoveryActive from a stale attempt's fenced-dead loop swallowed the new attempt's trigger. Replaced with recoveryOwner = owning attemptGen: the live attempt takes ownership from a stale owner; the finally only clears its own. 5. devicechange fence: stale enumeration completing after a recovery replaced the graph could reacquire again over the fresh capture — now dual-fenced (attemptGen + captureGen). 6. Heartbeat schema conformance: counters are now DELTAS since the last successfully-sent beat (D7.1 compact schema; lossless — a failed send's interval folds into the next delta because the baseline only advances on success), and the ≤300 B cap now holds even when the episode-free frame is too large: diagnostics drop in fixed order x → ba → sc, never the core evidence (n/q/c/cs/cap/mu/og/eo). 7. Lifecycle evidence exposed: ctxLastTransition, deviceEvents, recoveryEvents + explicit *Dropped truncation counters now ride VoiceStats so the engine/surface can actually consume D7.5 evidence. New tests: frozen-timer self-latch, wire-baseline stall, reacquire-outage watchdog, resume→reacquire escalation, suspended-ctx honest failure, cross-attempt recovery ownership, stale-enumeration fence, delta fold on send failure, lifecycle evidence via onStats, PCM-displacement bound (<1%), and a GC A/B allocation soak (auto-skips without --expose-gc). Transport suite: 101 tests, 100 pass + 1 conditional skip. * feat(voice): P7 step 2 — engine interim ledger, worker persistence, continuity guards Implements plan step 2 [A] of design-voice-p7-implementation.md (D7.1 engine side, D7.3 Tranche-A interim, §D7.0b): - src/voice-audio-health.ts — engine-side audio-progress ledger. Session-layer wraps (same monkey-patch pattern as the narration-tee): handleAudioFromClient counts delivered frames + runs the canonical ingress-RMS speech tracker (subsampled every 4th int16, allocation- free, pre-EchoGuard as stated in the design), handleJsonFromClient intercepts the client's audio_health heartbeats (delta counters accumulate into per-epoch totals; unknown fields tolerated; malformed frames can never break the message path), handleAudioOutput counts egress. Coverage is honestly 'session-only' (the ClientTransport buffering window stays blind until the bodhi pin). Tranche-A epoch scheme: the engine mints a monotonic epoch on first sight of each client nonce. getSpeechEvidence() {active, onsetAt, lastAboveFloorAt} is the ONE canonical speech signal (vision gate + matrix consumers); lazy hangover decay means a stalled client ends the evidence. inputHealth verdict (ok/stalled/no-client/unknown) for P4's ladder. Anomaly latches (open gap, new episode ids, heartbeat-stale, ingress stall while unmuted, sendSkipped, persistSkipped) clear per 30 s tick. - src/voice-audio-health-persist.ts — worker_threads sqlite writer (round-4 #3: DatabaseSync on the voice loop can block audio behind busy_timeout). One-slot mailbox: tryEnqueue() refuses while a row is in flight — the ledger counts samplesSkipped, nothing queues. Rows are written immediately on receipt (ack ⇒ committed), so crash evidence survives; row cap + 30-day retention pruned per insert. The worker spawns from an inline eval source because voice-agent ships as a single esbuild artifact where a sibling worker file cannot resolve. - src/voice-continuity.ts — D7.3 Tranche-A guards: shouldFireGoodbye (a reconnect replay repeating the same short farewell with no new real user turn must not re-fire session_end) and the centralized conversation clear (G-P7-8: items and the turn.end transcript cursor move together; the goodbye-detector path previously cleared items while leaving the cursor pointing past the emptied array). - src/voice-agent.ts wiring: ledger + persistence created at boot, wraps installed after session construction, onTurnLatency hook consumed (G-P7-12), client connect/disconnect wrappers reset the epoch and write a final row, the 30 s [Health] line carries the upgraded segments with anomaly-overrides-suppression (an anomalous tick logs even mid-ACTIVE-streak and persists an anomaly row), and a 60 s baseline persistence timer. All three conversation-clear sites route through the helper; the goodbye detector consults the guard. Tests: 27 new (parse tolerance, wrap transparency + idempotence + intercept, epoch minting/reset, speech tracker latch + lazy decay, inputHealth verdicts, anomaly latch/clear + mute suppression, persist row shape + mailbox skip, [Health] segments, crash-path durability via abrupt worker terminate, one-slot refusal, row-cap retention, failed-write visibility, goodbye guard, clear helper). Full suite: 934 pass / 2 conditional skips. * feat(voice): P7 step 3 — failure-localization matrix + hypothesis fixtures Implements plan step 3 [A] of design-voice-p7-implementation.md (D7.2): - src/voice-health-matrix.ts — a pure function over the engine ledger snapshot with HONEST outcomes. Row 0: RECONNECTING/CONNECTING is a reconnect-window; no heartbeat or no prior baseline is insufficient-evidence, never a guess. Row 1 (client capture dead) requires a CURRENT-epoch, unexpired gap — the open gap or a closed episode whose interval overlaps the ingress stall — while unmuted + connected (an old latched gap can never pair with unrelated later silence), and discriminates the S1/S2 shapes (ctx-suspended / post-suspension / device-recovery / main-thread-gap) from the fields that travel. Rows 2-4 (native server dispositions) are structurally unreachable at coverage session-only: their observable half surfaces as insufficient-evidence naming the unobserved hop. Row 5 (model-silent, the F4 residue) is likewise gated on full coverage — at session-only, speech-without-model-event names the blind upstream hop instead of blaming the model. Row 5' healthy-idle: a quietly listening user is not an incident. Row 6 fires on transmitted evidence (bufferedAmount growth or a sendSkipped delta); row 7a needs the server-side buffered input or stays silent; row 7b requires scheduling to advance with zero NATURAL completions and a frozen ctx clock (cancellations excluded by construction — barge-in is not a playback failure). - parseHeartbeat now decodes the full episode tuples (gap {startMs, durationMs}, speech {onsetSeq, offsetSeq, maxRmsPm, aboveFloorMs}) — the matrix needs intervals, not ids. The ledger tracks epochStartApproxMs (receipt-time approximation, labeled) so client-epoch-relative intervals can be placed on the engine clock, and carries lastMatrixVerdict into every persisted row. - voice-agent.ts: the 30 s tick evaluates the matrix (lastEgressAt as the Tranche-A model-event proxy, named), logs verdict transitions, and notes the verdict for persistence. - FE-1/FE-2 are encoded as HYPOTHESIS-labeled fixtures (round-1 #9: the incident counters never existed) to be validated against captured S1/S2 runs before promotion; FE-3 belongs to D7.6-S3, not the matrix. Tests: 20 matrix cases covering outcome totality, the partial-coverage-never-yields-a-layer-verdict rule, row-1 overlap + unmuted gates and shape discrimination, rows 6/7a/7b including the cancellation exclusion, healthy-idle, and both hypothesis fixtures. * fix(voice): apply Codex round-2 review to P7 steps 2-3 All ten findings were real; each fix carries a regression test: 1. Epoch now mints in onClientConnected (per the design) — a connection dying before its first heartbeat persists a real epoch; the client nonce BINDS on first sight instead of minting; a mid-connection nonce change re-mints defensively (never reuses an old epoch). Per-epoch reset now also covers samplesSkipped, latency samples, and the rate window baseline. 2. Unknown episode kinds are dropped, never misclassified as speech; the ep ingest caps at 8 entries (bounded telemetry, not trust). 3. Speech re-onset after a silent stall: the eager flag decays on the frame path BEFORE the floor check, so onsetAt re-latches instead of staying stuck at the first utterance (this fed the latency metric). 4. inputHealth gating: degraded (client recovery exhausted) surfaces even muted; mute is never a stall; client-reported evidence (open gap, suspended/closed ctx) counts only while the heartbeat is FRESH — a detached client's retained gap cannot report stalled forever; heartbeats crossing while NO PCM ever reached the session is now a stall, not silence-as-health. Anomaly reasons gain the same attached+unmuted+fresh gates. 5. voice-lifecycle.json now carries the additive inputHealth verdict (publishLifecycleSnapshot opts + schema; no-client folds to 'unknown'); the 30 s tick republishes on an inputHealth flip — P4's evidence ladder can actually consume 'stalled'. 6. Tick ordering: anomalies() is now a PEEK — the anomaly row persists before clearTickLatches() runs, so evidence can no longer be cleared before it serializes. Rate windows advance every tick (a suppressed streak no longer turns the next line into a long-window average). [Health] out.buffered is the SERVER egress socket (client uplink moved to clientHealth.upBuf); gaps counts gap episodes only. The worker's failedWrites surface in the line + as an anomaly. 7. Persistence contract: column renamed payload_json (design schema); the row cap is PER SESSION (a busy new session cannot evict another session's evidence); the 1-min timer only writes while a client is attached (idle rows were slowly evicting real evidence); an over-budget payload persists as a reduced-but-VALID JSON snapshot instead of a mid-document slice. 8. Goodbye guard rebases whenever the session gates reset (all three reset sites route through resetSessionGateState) — a legitimate goodbye in the next session, with the turn counter restarted below the old watermark, is no longer suppressed. 9. The D7.3 latency consumer exists: ingress speech onset → first model audio, computed on the server clock (labeled receipt-time approximation), exposed in the snapshot and [Health]. 10. The ingress RMS pass is bounded (8 KiB scan cap) — frame work can no longer scale with a client-controlled buffer length. Also new: a locked-health-DB test proving tryEnqueue never blocks the caller while the worker sits in busy_timeout, a 30-day retention test, and a lifecycle inputHealth publish test. (voice-agent.ts here also carries the 2-line D7.4 wiring that hands the ledger's getSpeechEvidence to vision-tools — the vision-side half lands in the next commit.) * feat(voice): P7 step 5 — vision egress controls (gate, token bucket, latest-frame slot, in-process downscale) Implements plan step 5 [A] of design-voice-p7-implementation.md (D7.4, the FE-1 controls): - ONE central egress path: every frame — pull tick, browser push (the previously ungated FE-1 hole), external sources — goes through sendFrameGated: gate → token bucket → send. Post-send hooks (screen-companion selection) fire only on real sends, including a later drain. - Tranche-A gate: ACTIVE ∧ ¬speechActive. The session-state check uses getSendFile's trust contract (gate only when sessionManager is exposed and non-ACTIVE); speech comes from the engine ledger's canonical getSpeechEvidence, injected by voice-agent (setVisionSpeechEvidence). Bodhi's buffering/replaying flags are not observable pre-pin — that blind spot stays NAMED until Tranche B. - Token bucket: 300 KiB/s refill, 600 KiB burst, ~1 fps send cap — application-level control on the shared upstream socket; socket depth gates nothing. - Deferral is a latest-frame-only slot, never a backlog (a queued burst draining after speech would re-create FE-1 under a different name): a new frame displaces the old one (counted), a 250 ms drain timer delivers the survivor when the gate opens and budget allows, the drain takes the frame OUT before retrying (no self-displacement), and stopStream clears the slot so stale vision never leaks into the next session. Drop/deferral counters ride getVisionState().egress. - Downscale/recompress executes in the CAPTURE SERVER's process (screen-capture-server.py maxdim/quality params → sips subprocess; bounded 320-3840 / 10-100): compression never competes with the voice event loop. The screen source requests the ~720p/q0.6 budget (maxdim=1280&quality=60). The residual voice-loop cost (base64 + the SDK's synchronous stringify) is named and covered by a budget test. Tests: gate deferral + drain, pause-during-speech, latest-frame-only burst semantics (newest wins, two displaced), fps-cap budget deferral, slot cleared on stop, egress diagnostics exposure, and the worst-case frame residual-cost budget guard. The screen-companion e2e now paces its real sends to the designed ~1 fps cadence. * fix(voice): apply Codex round-3 review to P7 steps 3+5 All ten blockers were real; each fix carries a regression test: 1. Epoch isolation: MatrixBaseline records its epoch — a boundary between ticks returns insufficient-evidence(epoch-boundary) instead of diffing cross-epoch garbage; an unexpected mid-connection nonce is now a FULL epoch boundary (server counters, heartbeats, latency samples, tick windows all reset), not just a client-baseline reset. 2. Episode placement: the heartbeat carries ea (ms since connect on the CLIENT clock at assembly) — epochStartApprox = receivedAt − ea, accurate to network latency, refreshed per beat. The first-beat −500 ms guess remains only as the pre-ea fallback. (Wire change: re-vendored into cinny in the companion commit.) 3. Honesty totality: episodeOverflow, a stale heartbeat, or a zero-progress unmuted window can no longer fall through to healthy-idle — each is insufficient-evidence with its reason named. Row-5 speech evidence is the CANONICAL ingress tracker only; a stale client speech episode idempotently re-sent in the window no longer counts as current speech. 4. D7.2 ordering: row 5 (model-silent) now evaluates BEFORE rows 6/7; 7b requires non-cancelled scheduling (an all-cancelled barge-in window proves nothing about the output path — chunksCancelled joins the baseline). 5. Model EVENTS, not model audio: the ledger gains noteModelEvent() (wired from turn.start and onToolCall); egress audio still counts. The speech→model latency sample consumes on the FIRST event, and the matrix receives lastModelEventAt from the snapshot. 6. Transition durability: a matrix verdict change persists an anomaly row (a transient verdict can no longer vanish before the 1-min baseline); failedWrites growth forces the [Health] line; episode anomalies require an attached client. 7. Legacy DB migration: the worker renames a pre-rename payload column to payload_json on open — an existing database is migrated, not silently bricked (reproduced by the new test). 8. Stream fence: stopStream/startStream bump a streamGen; an in-flight pull capture from a stopped stream drops its stale frame instead of sending after the stop (slow-source repro in tests). 9. §D7.0b activated paths: the screen-companion selection probe is now ASYNC end-to-end (execFile, never execSync on the voice loop; the frame hook is fire-and-forget with injection landing on resolve), and the webUI browser-push compresses in an OffscreenCanvas worker (main thread only grabs an ImageBitmap; feature-detected fallback to the old canvas path; one frame in flight — latest-frame discipline). 10. Hard caps: the capture server returns an error when sips fails AND the frame exceeds the 400 KiB fail budget (a native-res original can never silently pass); the vision bucket charges base64 wire bytes (4/3 of raw), closing the 25% under-charge. Full suite: 990 pass / 2 conditional skips. * fix(vision): bundled Watch never started — resolve .py sidecars from the repo root, make the lazy spawn observable Field root cause (bundled build, 2026-08-14): the lazy capture-server spawn resolved screen-capture-server.py NEXT TO THE RUNNING MODULE. True in dev (tsx src/voice-agent.ts); false in the bundle, where the running module is dist/voice-agent.js and build-bundle.mjs ships no .py files in dist/ — python3 exited 2 ('No such file') invisibly (detached, stdio ignore, no handlers), so the only symptom was the 8s port timeout, once per tick: '10 consecutive' frame errors, 0 frames in 107 s. The bundle DOES ship src/*.py as a sibling of dist/. - vision-tools: _captureServerScriptCandidates() resolves via findRepoRoot (marker sutando.config.json — the same walk python-binary.ts uses) → <root>/src/screen-capture-server.py, with the module-sibling kept as the dev/exotic fallback; a missing script now fails FAST naming every tried path instead of burning 8 s. - Spawn observability + process shape: NOT detached (the server stays a non-disclaimed child — the shape the supervisor deliberately uses for screen-capture so TCC attributes the grant to the app lineage; unref() still keeps the loop free), stdout/stderr to <workspace>/logs/screen-capture-server.lazy.log, pid + script + log path logged, and on('error')/on('exit') fail the port wait immediately with code/signal + log pointer instead of a ghost timeout. - task-delegation: telemetry.py had the identical module-relative defect (dist/telemetry.py cannot exist) — same root-first resolution. - vision ts() now logs UTC like voice-agent (a local-time subsystem next to UTC subsystems made incident timelines off-by-timezone: 09:48:04 [Vision] == 16:48:04 [VoiceSession]). Not touched: the frame path (this is spawn-time control plane only). The Screen Recording grant itself (supervisor gate: 'Screen Recording not granted') remains a user-side System Settings action. Tests: 4 new cases pin the candidate order against the bundle shape (dist/ + sibling src/), the dev shape (deduplicated), the no-marker fallback, and the real checkout from both src/ and a simulated dist/. Full suite: 994 pass / 2 conditional skips. * docs: regenerate src-map for P7 Tranche A modules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(voice): dead resumption handle poisoned every reconnect — clear it on the CLOSED fresh-connect path Field incident (2026-08-14 18:56Z): Gemini invalidated a ~9-minute session and closed 1008 'Requested entity was not found'. Every subsequent reconnect then died the same way, faster each time (9 min → 18 s → 0.9 s), including the health monitor's dead-session trigger. Root cause (bodhi pin 25cc6e8): handleTransportClose transitions ACTIVE→CLOSED but never clears transport.config.resumptionHandle; the CLOSED→fresh-connect path (handleClientConnected) then replays the dead session's handle in sessionResumption:{handle}. Gemini completes setup and asynchronously kills each new connection when the referenced session entity cannot be found. Bodhi's own clearResumptionHandle comment documents the intended recovery ('a dead handle routes recovery through the CLOSED → fresh-connect path') — but the only clearing trigger is a resumable:false update, which a dying connection never sends, and the sessionManager copy is not the one connect() reads. Engine-side fix at the current pin: clearStaleResumptionHandle() (voice-continuity) empties BOTH handle copies, wired into the existing handleClientConnected wrapper on the CLOSED path only — that path already rebuilds context via silent text injection, never via resume, so clearing costs nothing; mid-session GoAway resume (the legitimate handle use) does not run through it. The proper fix belongs in bodhi's handleTransportClose (Tranche B pin note). Tests: both copies cleared + reported; no-handle sync; missing seams never throw. * fix(voice): close P7 review findings * test(vision): execute capture downscale coverage --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
startup.sh already starts Claude Code, so no need for separate instruction.