Skip to content

fix(voice-agent): close idle Gemini transport to stop phantom turns - #602

Merged
sonichi merged 2 commits into
mainfrom
fix/voice-agent-idle-teardown
May 7, 2026
Merged

sonichi merged 2 commits into
mainfrom
fix/voice-agent-idle-teardown

Conversation

@liususan091219

Copy link
Copy Markdown
Collaborator

Summary

Voice-agent kept the Gemini Live session alive even with clientConnected=false. Every ~9-min Gemini reconnect (GoAway) produced a phantom assistant turn 2-3s later — sometimes a tool call (save_meeting_note, open_url, work) that polluted notes files, opened browser tabs, or wrote phantom task files. Reproduced 100% on Studio + verified on Maddy (latest 1c350fa / bodhi ffec0cd).

Adds a 60s idle teardown:

  • on handleClientDisconnected, schedule a timer
  • timer calls transport.disconnect() if no client returned in time
  • on handleClientConnected, cancel any pending timer
  • also armed at boot so a freshly-started voice-agent with no client doesn't keep an idle session open

CLOSED is a fixed point when clientConnected=false (existing health monitor only resumes CLOSED → CONNECTING when a client is present), so no phantoms can fire until a real client reconnects.

Pre-patch evidence

12:23 turn_1 [user] hello (injected via WS)
12:23 client disconnected
12:29 GoAway → 12:29:51 turn_2 [assistant] "ack"   ← PHANTOM
12:38 GoAway → 12:38:50 turn_3 [assistant] "ack"   ← PHANTOM
... (10 phantoms / 10 reconnect cycles in 90 min)

Cross-host (Maddy): same pattern, silent-failure surface — phantom [Tool] save_meeting_note polluting notes/meeting-2026-05-04.md (17 lines, all phantom — confirmed by Susan she didn't dictate any).

Post-patch verification (Studio, latest main + this patch)

15:09:54 voice-agent restart on patched code
15:10:50 [VoiceSession] Idle 60s — closing Gemini transport (no phantoms while CLOSED)   ← BOOT teardown
15:10:50 Transport closed (state=ACTIVE code=1000), Session Ended
15:11:20 → 15:13:51 [Health] state=CLOSED client=false   ← stayed CLOSED through 4 would-be reconnect windows, zero phantoms
15:13:54 Client connected → Gemini reopens automatically (existing health-monitor recovery)
15:13:59 Client disconnected
15:14:59 [VoiceSession] Idle 60s — closing Gemini transport   ← DISCONNECT teardown fired exactly 60s after client gone

Pre-patch baseline: 90 min idle = 10 phantoms.
Post-patch: 6 min spanning boot + connect + disconnect cycle = 0 phantoms + 2 confirmed teardown events.

Test plan

  • Boot voice-agent fresh, wait 60s, verify idle teardown log appears + state goes CLOSED
  • Connect a client (WS text-input), verify Gemini reopens (Client connected → Gemini inactive — resetting session and reconnecting)
  • Disconnect client, wait 60s, verify teardown fires again
  • Verify zero `[Tool]` calls or `Turn complete` events fire while state=CLOSED
  • (manual) Verify normal voice flow on a real client connection still works — voice metrics, transcripts, tool calls all behave as before

🤖 Generated with Claude Code

Voice-agent kept the Gemini Live session alive even with `clientConnected=false`. Every ~9-min Gemini reconnect ("GoAway") then produced a phantom assistant turn 2-3s later — sometimes a tool call (`save_meeting_note`, `open_url`, `work`) that polluted notes files, opened browser tabs, or wrote phantom task files. Reproduced 100% on Studio + verified on Maddy.

Adds a 60s idle teardown:
- on `handleClientDisconnected`, schedule a timer
- timer fires `transport.disconnect()` if no client returned in time
- on `handleClientConnected`, cancel any pending timer
- also armed at boot so a freshly-started voice-agent with no client doesn't keep an idle session open

CLOSED is a fixed point when `clientConnected=false` (existing health monitor only resumes CLOSED→CONNECTING when a client is present), so no phantoms can fire until a real client reconnects.

Verification (post-patch):
- 15:10:50 boot teardown fired, state CLOSED
- 15:11:20–15:13:51 stayed CLOSED through 4 would-be reconnect windows, zero phantoms
- 15:13:54 client connected → Gemini reopens automatically
- 15:14:59 disconnect teardown fired exactly 60s after client gone, state CLOSED

Pre-patch baseline: 90 min idle = 10 phantoms. Post-patch: 6 min idle = 0 phantoms + 2 confirmed teardown events.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@sonichi sonichi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — addresses today's lifecycle bug cleanly.

Verified architecture:

  • IDLE_TEARDOWN_MS = 60_000 (60s) timer wraps handleClientDisconnected to schedule teardown, wraps handleClientConnected to cancel it.
  • Initial arm at boot (line 1167) — voice-agent boots with no client; teardown fires if none connects in 60s. Closes the GoAway-without-client phantom-loop window before it can re-trigger.
  • Setting timeout callback re-checks clientConnected immediately before disconnect — prevents race where a client connects between schedule and fire. ✓
  • cancelIdleTeardown() clears + nulls the handle, idempotent.
  • The CLOSED-as-fixed-point reasoning is correct: existing health monitor only reconnects CLOSED→CONNECTING when a client is present; without a client, the session stays closed.

Empirical context: Mini's voice-agent had 3 GoAway events clustered in 62s today (logs/voice-agent.log) — exactly the bug-shape this fixes. After this lands + bridge restarts pick up the new code, those clusters should disappear from the logs across the fleet.

Gate check (per repo norm: import strategy / defaults / bind-address):

  • Import strategy: no new imports ✓
  • Defaults: 60s timer is conservative; first-connect adds at most ~1-2s reconnect latency if user takes >60s post-boot to connect ✓
  • Bind-address: N/A (pure timer logic) ✓

One non-blocker note: IDLE_TEARDOWN_MS is a hardcoded const. If we later observe the 60s default is too aggressive (e.g. some user pattern reconnects every ~70s), tuning means a recompile. Could expose as process.env.SUTANDO_VOICE_IDLE_TEARDOWN_MS ?? 60_000. Not in scope; defaults are sensible for the current evidence.

Approve. Recommend merging via gh pr merge --squash (no --delete-branch per repo norm).

… env

Per Mini's #602 review non-blocker note: hardcoded 60s default is fine for
current evidence, but tuning means a recompile. Read from
`process.env.SUTANDO_VOICE_IDLE_TEARDOWN_MS` with the same 60s default —
hosts with different connect/disconnect churn can adjust without code change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@sonichi

sonichi commented May 6, 2026

Copy link
Copy Markdown
Owner

Hi @liususan091219 — Chi wants to merge this. Anything you want to address first, or ready for merge?

@sonichi
sonichi merged commit 6ac9471 into main May 7, 2026
1 check passed
@sonichi
sonichi deleted the fix/voice-agent-idle-teardown branch May 7, 2026 17:28
sonichi pushed a commit that referenced this pull request May 7, 2026
…closes (#617)

Issue
  When the Gemini Live transport closes for a non-retryable reason
  (1011 credits depleted, 1011 quota exceeded, 401/403 auth, model not
  found), the voice-agent's health monitor keeps triggering reconnect
  every 60s. Each reconnect attempt fails after a 30s timeout, so the
  cycle is ~6 log lines per minute indefinitely while the upstream
  issue persists.

  Reproduced live this session (Studio): 17 instances of
  `code=1011 reason="prepayment credits depleted"` in a 5-min window
  while clientConnected=true. The voice-failure classifier already
  fires the OS notification + writes the actionable proactive-result
  file once per category, so the user has been told. The retry-loop
  just spams logs after that.

  PR #602's idle teardown only handles the *no-client* case (60s no
  client → close). Here a client is connected throughout, so #602
  doesn't apply.

Fix
  New `voiceFatalBackoffUntil` deadline (single closure-scoped variable,
  same scope as `lastReconnectAt`):

  - Bumped 5min into the future on every non-retryable close, including
    repeats of an already-notified category. Lives next to the existing
    notification dedup so the bump fires even when the per-category
    notification is suppressed.
  - Read by the 30s health monitor's reconnect-trigger condition — the
    monitor skips its `handleClientConnected()` call when the deadline
    is in the future. Reconnect cycle drops from 60s → 5min during a
    persistent outage.
  - Reset to 0 when the monitor observes state=ACTIVE — recovery from
    a transient close after the user fixed the upstream issue is not
    suppressed by a stale fatal window.

  No new env vars, no new constants exposed; one local `let` + four
  arithmetic uses. Defaults conservative (5min ≈ user-visible recovery
  delay after top-up).

Verification
  - `npx tsc --noEmit` — clean
  - `npm test` — 154/154 ts + all py green
  - Trace through the 1011-loop scenario:
    1. transport closes with 1011 → classifier → handleClose fires
    2. retryable=false → bumps voiceFatalBackoffUntil to now+5min,
       then notifies (first time only — dedup as before)
    3. health monitor 30s tick: state=CLOSED, client=true, 60s
       throttle elapsed, BUT `Date.now() > voiceFatalBackoffUntil` is
       false → skip reconnect
    4. log dedup at line 1209 keeps the [Health] line silent (status
       unchanged) — no spam
    5. ~5min later, voiceFatalBackoffUntil expires → reconnect fires
    6. on success: state=ACTIVE → reset voiceFatalBackoffUntil=0,
       normal 60s throttle restored
    7. on continued failure: classifier re-bumps voiceFatalBackoffUntil

Why not unconditional 5min throttle
  Today's 60s throttle handles transient network blips well. Slowing
  every reconnect to 5min would degrade UX during normal flaps. Better
  to slow only the cases the classifier already knows are fatal.

Why not stop reconnecting entirely on fatal
  The upstream issue (top-up, key rotation) often resolves without
  restarting voice-agent. A 5min auto-recovery window keeps the agent
  self-healing once credits/auth are back. Stopping permanently would
  require a manual restart.

Cc: stack interaction with PR #602 (idle teardown) — both touch
voice-agent.ts but in disjoint regions; merge order doesn't matter.

Co-authored-by: Vasiliy Radostev <vasiliyr@mac.mynetworksettings.com>
liususan091219 added a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
…onichi#602)

* fix(voice-agent): close idle Gemini transport to stop phantom turns

Voice-agent kept the Gemini Live session alive even with `clientConnected=false`. Every ~9-min Gemini reconnect ("GoAway") then produced a phantom assistant turn 2-3s later — sometimes a tool call (`save_meeting_note`, `open_url`, `work`) that polluted notes files, opened browser tabs, or wrote phantom task files. Reproduced 100% on Studio + verified on Maddy.

Adds a 60s idle teardown:
- on `handleClientDisconnected`, schedule a timer
- timer fires `transport.disconnect()` if no client returned in time
- on `handleClientConnected`, cancel any pending timer
- also armed at boot so a freshly-started voice-agent with no client doesn't keep an idle session open

CLOSED is a fixed point when `clientConnected=false` (existing health monitor only resumes CLOSED→CONNECTING when a client is present), so no phantoms can fire until a real client reconnects.

Verification (post-patch):
- 15:10:50 boot teardown fired, state CLOSED
- 15:11:20–15:13:51 stayed CLOSED through 4 would-be reconnect windows, zero phantoms
- 15:13:54 client connected → Gemini reopens automatically
- 15:14:59 disconnect teardown fired exactly 60s after client gone, state CLOSED

Pre-patch baseline: 90 min idle = 10 phantoms. Post-patch: 6 min idle = 0 phantoms + 2 confirmed teardown events.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sonichi#602): expose IDLE_TEARDOWN_MS via SUTANDO_VOICE_IDLE_TEARDOWN_MS env

Per Mini's sonichi#602 review non-blocker note: hardcoded 60s default is fine for
current evidence, but tuning means a recompile. Read from
`process.env.SUTANDO_VOICE_IDLE_TEARDOWN_MS` with the same 60s default —
hosts with different connect/disconnect churn can adjust without code change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Xueqing Liu <xueqingliu@Xueqings-Mac-Studio.local>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
liususan091219 pushed a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
…closes (sonichi#617)

Issue
  When the Gemini Live transport closes for a non-retryable reason
  (1011 credits depleted, 1011 quota exceeded, 401/403 auth, model not
  found), the voice-agent's health monitor keeps triggering reconnect
  every 60s. Each reconnect attempt fails after a 30s timeout, so the
  cycle is ~6 log lines per minute indefinitely while the upstream
  issue persists.

  Reproduced live this session (Studio): 17 instances of
  `code=1011 reason="prepayment credits depleted"` in a 5-min window
  while clientConnected=true. The voice-failure classifier already
  fires the OS notification + writes the actionable proactive-result
  file once per category, so the user has been told. The retry-loop
  just spams logs after that.

  PR sonichi#602's idle teardown only handles the *no-client* case (60s no
  client → close). Here a client is connected throughout, so sonichi#602
  doesn't apply.

Fix
  New `voiceFatalBackoffUntil` deadline (single closure-scoped variable,
  same scope as `lastReconnectAt`):

  - Bumped 5min into the future on every non-retryable close, including
    repeats of an already-notified category. Lives next to the existing
    notification dedup so the bump fires even when the per-category
    notification is suppressed.
  - Read by the 30s health monitor's reconnect-trigger condition — the
    monitor skips its `handleClientConnected()` call when the deadline
    is in the future. Reconnect cycle drops from 60s → 5min during a
    persistent outage.
  - Reset to 0 when the monitor observes state=ACTIVE — recovery from
    a transient close after the user fixed the upstream issue is not
    suppressed by a stale fatal window.

  No new env vars, no new constants exposed; one local `let` + four
  arithmetic uses. Defaults conservative (5min ≈ user-visible recovery
  delay after top-up).

Verification
  - `npx tsc --noEmit` — clean
  - `npm test` — 154/154 ts + all py green
  - Trace through the 1011-loop scenario:
    1. transport closes with 1011 → classifier → handleClose fires
    2. retryable=false → bumps voiceFatalBackoffUntil to now+5min,
       then notifies (first time only — dedup as before)
    3. health monitor 30s tick: state=CLOSED, client=true, 60s
       throttle elapsed, BUT `Date.now() > voiceFatalBackoffUntil` is
       false → skip reconnect
    4. log dedup at line 1209 keeps the [Health] line silent (status
       unchanged) — no spam
    5. ~5min later, voiceFatalBackoffUntil expires → reconnect fires
    6. on success: state=ACTIVE → reset voiceFatalBackoffUntil=0,
       normal 60s throttle restored
    7. on continued failure: classifier re-bumps voiceFatalBackoffUntil

Why not unconditional 5min throttle
  Today's 60s throttle handles transient network blips well. Slowing
  every reconnect to 5min would degrade UX during normal flaps. Better
  to slow only the cases the classifier already knows are fatal.

Why not stop reconnecting entirely on fatal
  The upstream issue (top-up, key rotation) often resolves without
  restarting voice-agent. A 5min auto-recovery window keeps the agent
  self-healing once credits/auth are back. Stopping permanently would
  require a manual restart.

Cc: stack interaction with PR sonichi#602 (idle teardown) — both touch
voice-agent.ts but in disjoint regions; merge order doesn't matter.

Co-authored-by: Vasiliy Radostev <vasiliyr@mac.mynetworksettings.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants