Skip to content

Fix phone agent duplicate responses on reconnect - #105

Merged
sonichi merged 1 commit into
mainfrom
fix/phone-reconnect-replay
Apr 3, 2026
Merged

Fix phone agent duplicate responses on reconnect#105
sonichi merged 1 commit into
mainfrom
fix/phone-reconnect-replay

Conversation

@sonichi

@sonichi sonichi commented Apr 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Replace fixed 6-second replay suppression with turn-count-based detection
  • Track conversation turn count before disconnect, unmute only after Gemini replays all previous turns
  • Fallback timeout scales with conversation length (max of 10s or 2s per turn)

Root cause

For longer calls (>3 turns), the fixed 6s timeout expired before Gemini finished replaying conversation history, causing duplicate audio output.

Call log scanner (PR #104) found 217 duplicate response instances across 57/158 calls — this was the #1 detected issue.

Test plan

  • Make a call, have 5+ turns, then trigger a Gemini reconnect — verify no duplicate audio
  • Short call (<3 turns) still works with the fallback timer
  • Verify [Phone] replay complete log message appears on reconnect

🤖 Generated with Claude Code

@sonichi @claude
Fix phone agent duplicate responses on reconnect
626b45b 
Replace fixed 6-second replay suppression timeout with turn-count-based
detection. Tracks conversation turn count before disconnect, unmutes when
Gemini replays all previous turns. Fallback timeout scales with conversation
length (max of 10s or 2s per turn).

Root cause: for longer calls (>3 turns), the 6s timeout expired before
Gemini finished replaying the conversation history, causing duplicate audio.
Call log scanner found 217 duplicate response instances across 57/158 calls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sonichi
sonichi merged commit 9f79021 into main Apr 3, 2026
@sonichi
sonichi deleted the fix/phone-reconnect-replay branch April 3, 2026 10:54
liususan091219 added a commit that referenced this pull request Apr 5, 2026
@liususan091219 @claude
Add --summary flag to call scanner for quality trend tracking
6fddc28 
Shows daily breakdown: total calls, clean vs issues, issue rate,
top patterns per day. Clear improvement trend visible:
- Mar 28-30: 80-100% issue rate (duplicates)
- Mar 31: 45% after PR #67
- Apr 2-3: 13-29% after PR #105
- Apr 4: 21% (remaining: confusion + tab switching)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 added a commit that referenced this pull request Apr 6, 2026
@liususan091219 @claude
Add --summary flag to call scanner for quality trend tracking
631a04c 
Shows daily breakdown: total calls, clean vs issues, issue rate,
top patterns per day. Clear improvement trend visible:
- Mar 28-30: 80-100% issue rate (duplicates)
- Mar 31: 45% after PR #67
- Apr 2-3: 13-29% after PR #105
- Apr 4: 21% (remaining: confusion + tab switching)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi pushed a commit that referenced this pull request Apr 7, 2026
@liususan091219 @claude
Add 6 new call log scanner detectors + quality trend (#185)
4b552b4 
* Add 3 new call log scanner detectors from transcript analysis

New patterns detected across 263 calls:
- reconnect_leak: 37 calls with "I'm back" after Gemini reconnect
- repeated_summon: 3 calls where user asked to summon 3+ times
- repeated_tab_switch: 7 calls where tab switching failed repeatedly

Added 6 tests (20 total, was 14). Scanner now has 8 detectors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add --summary flag to call scanner for quality trend tracking

Shows daily breakdown: total calls, clean vs issues, issue rate,
top patterns per day. Clear improvement trend visible:
- Mar 28-30: 80-100% issue rate (duplicates)
- Mar 31: 45% after PR #67
- Apr 2-3: 13-29% after PR #105
- Apr 4: 21% (remaining: confusion + tab switching)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add 3 new call detectors: recording confusion, scroll frustration, STT retry

Analyzed all 266 calls to find missing patterns:
- recording_confusion: 28 calls where user expected recording to start/stop
- scroll_frustration: 4 calls with 3+ scroll requests (tool unresponsive)
- stt_retry: 23 calls where caller rephrased (word overlap 30-90%)

Quality trend shows issue rate dropped from 80-100% (Mar 28) to 25-32% (Apr 5).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix recording_confusion false positives — exclude IVR lines

Tightened recording_confusion detector to exclude IVR "You have not
entered any numbers" and similar menu prompts. Also removed overly
broad "you have not" pattern that matched non-recording IVR messages.

Result: 28 → 8 detections (20 false positives removed).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove tests/ from tracked files — should be gitignored

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi pushed a commit that referenced this pull request Apr 7, 2026
@liususan091219 @claude
Infrastructure, security & tooling improvements (#178)
81f4af1 
* Pass previous description to Gemini for natural continuation

Screen descriptions now flow naturally instead of every sentence
starting with "The screen displays...". When a previous description
exists, the prompt says "Continue naturally — do NOT start with
The screen displays or repeat the previous description."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Revert "Pass previous description to Gemini for natural continuation"

This reverts commit 1105ed89e6cc41a0de777bc3b1dff9d157589d23.

* Make task watcher persistent (loop instead of one-shot)

watch-tasks.sh now loops continuously instead of exiting after
detecting one batch of tasks. This eliminates the recurring issue
where the watcher dies between task detections, causing delayed
task pickup (e.g. the Discord video send delay).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add proactive Discord DM support to bridge

The bridge now scans results/proactive-*.txt and sends them to the
owner's DM channel. Previously only task replies were sent back —
proactive messages (like sending a video unprompted) were silently
dropped because they had no matching pending_replies entry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix 3 CodeQL security alerts

- browser-tools.ts: Fix command injection in typeText — write
  AppleScript to temp file instead of inline shell interpolation
- conversation-server.ts: Sanitize DTMF digits in TwiML Play
  (strip non-digit chars to prevent XML injection)
- conversation-server.ts: Improve phone number masking in startup
  log (show only first 2 + last 2 digits)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Harden path traversal guards in dashboard and agent-api

- dashboard.py: Add resolve() + is_relative_to() for notes
  read and delete endpoints (supplements existing regex guard)
- agent-api.py: Remove unused mimetypes import (replaced by
  SAFE_TYPES allowlist in previous security fix)

Addresses CodeQL alerts #28-31 (dashboard) and cleans up #19-23
(agent-api already has guards, but adding belt-and-suspenders).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add 3 new call log scanner detectors from transcript analysis

New patterns detected across 263 calls:
- reconnect_leak: 37 calls with "I'm back" after Gemini reconnect
- repeated_summon: 3 calls where user asked to summon 3+ times
- repeated_tab_switch: 7 calls where tab switching failed repeatedly

Added 6 tests (20 total, was 14). Scanner now has 8 detectors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Expand security tests for CodeQL fixes (33 assertions, was 17)

New test coverage for:
- Dashboard note slug validation (7 tests: traversal, slash, null byte, injection)
- Dashboard note path resolution via is_relative_to (2 tests)
- DTMF digit sanitization for TwiML injection (5 tests)
- TypeText AppleScript injection prevention (2 tests)

Total test suite: 120 assertions across 4 suites, all passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add test runner script (4 suites, 120 assertions)

bash tests/run-all.sh runs all test suites:
- Phone access control (49 assertions)
- Browser tools / STT corrections (18 assertions)
- Security sanitization (33 assertions)
- Call log scanner (20 assertions)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix migration script: auto-detect memory dir + don't exit on optional paths

- Memory dir now auto-detected from repo path slug (falls back to
  default ~/Desktop/sutando path)
- Session dir also auto-detected
- Replaced set -e with set -uo pipefail — many migration paths
  legitimately don't exist and shouldn't abort the script
- Tested: successfully bundles .env, 22 memory files, claude config,
  session history, flywheel data into 3.2MB tarball

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add --summary flag to call scanner for quality trend tracking

Shows daily breakdown: total calls, clean vs issues, issue rate,
top patterns per day. Clear improvement trend visible:
- Mar 28-30: 80-100% issue rate (duplicates)
- Mar 31: 45% after PR #67
- Apr 2-3: 13-29% after PR #105
- Apr 4: 21% (remaining: confusion + tab switching)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add migration script tests (14 assertions, 5 suites total)

Tests cover:
- REPO_SLUG generation (no slashes, starts with dash)
- Memory dir auto-detection from repo path
- Session dir detection
- Bundle creation (env, memory, MEMORY.md)
- Script syntax validation (no set -e)

Total: 134 assertions across 5 suites, all passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add ngrok tunnel to health check (13 checks total)

Verifies the ngrok HTTPS tunnel is active — required for inbound
phone calls via Twilio. Shows tunnel URL in health output.
Only checked when Twilio is configured in .env.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add Twilio webhook URL verification script

Checks if Twilio's phone number webhook matches the current ngrok
tunnel. Stale webhooks are a common failure mode — inbound calls
silently fail. Supports --fix flag to auto-update Twilio.

Usage: bash src/check-webhook.sh [--fix]
Exit: 0=match, 1=mismatch, 2=error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add Twilio webhook verification to health check (15 checks)

Runs check-webhook.sh as part of the health check — warns if
Twilio's voice URL doesn't match the current ngrok tunnel.
Stale webhooks cause inbound calls to silently fail.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix Python 3.9 compat in github-webhook.py

Remove str | None return type annotation (requires Python 3.10+).
Susan runs Python 3.9 — this would crash on import.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Address review: fix 2 blockers + 2 yellow items (Wenjix review)

Blocker 1: check-webhook.sh Python injection — pass phone number
via env var instead of interpolating into Python source string.

Blocker 2: phone mask padEnd alignment — wrap entire masked string
in parens before padEnd. Show last 4 digits (standard compromise).

Yellow: discord-bridge proactive files — rename to .failed on send
failure instead of deleting (preserves message for inspection).

Yellow: typeText tmp file — add random suffix to prevent race,
use try/finally to ensure cleanup on execSync failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove test files from PR (tests/ is gitignored, should stay local)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove non-security changes from PR

Revert infra/tooling files to main — this PR should only contain
security fixes with CodeQL citations. Non-security changes (webhook
script, scanner detectors, health checks, migration, watcher,
Discord DM, Python 3.9 fix) moved to local infra branch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add CodeQL citation comments to all security fixes

Each fix now has an inline code comment citing the specific
CodeQL alert number and what vulnerability it addresses.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 pushed a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
@sonichi @claude
Fix phone agent duplicate responses on reconnect (sonichi#105)
1769c85 
Replace fixed 6-second replay suppression timeout with turn-count-based
detection. Tracks conversation turn count before disconnect, unmutes when
Gemini replays all previous turns. Fallback timeout scales with conversation
length (max of 10s or 2s per turn).

Root cause: for longer calls (>3 turns), the 6s timeout expired before
Gemini finished replaying the conversation history, causing duplicate audio.
Call log scanner found 217 duplicate response instances across 57/158 calls.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 added a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
@liususan091219 @claude
Add 6 new call log scanner detectors + quality trend (sonichi#185)
e21c32b 
* Add 3 new call log scanner detectors from transcript analysis

New patterns detected across 263 calls:
- reconnect_leak: 37 calls with "I'm back" after Gemini reconnect
- repeated_summon: 3 calls where user asked to summon 3+ times
- repeated_tab_switch: 7 calls where tab switching failed repeatedly

Added 6 tests (20 total, was 14). Scanner now has 8 detectors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add --summary flag to call scanner for quality trend tracking

Shows daily breakdown: total calls, clean vs issues, issue rate,
top patterns per day. Clear improvement trend visible:
- Mar 28-30: 80-100% issue rate (duplicates)
- Mar 31: 45% after PR sonichi#67
- Apr 2-3: 13-29% after PR sonichi#105
- Apr 4: 21% (remaining: confusion + tab switching)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add 3 new call detectors: recording confusion, scroll frustration, STT retry

Analyzed all 266 calls to find missing patterns:
- recording_confusion: 28 calls where user expected recording to start/stop
- scroll_frustration: 4 calls with 3+ scroll requests (tool unresponsive)
- stt_retry: 23 calls where caller rephrased (word overlap 30-90%)

Quality trend shows issue rate dropped from 80-100% (Mar 28) to 25-32% (Apr 5).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix recording_confusion false positives — exclude IVR lines

Tightened recording_confusion detector to exclude IVR "You have not
entered any numbers" and similar menu prompts. Also removed overly
broad "you have not" pattern that matched non-recording IVR messages.

Result: 28 → 8 detections (20 false positives removed).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove tests/ from tracked files — should be gitignored

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 added a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
@liususan091219 @claude
Infrastructure, security & tooling improvements (sonichi#178)
73dc194 
* Pass previous description to Gemini for natural continuation

Screen descriptions now flow naturally instead of every sentence
starting with "The screen displays...". When a previous description
exists, the prompt says "Continue naturally — do NOT start with
The screen displays or repeat the previous description."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Revert "Pass previous description to Gemini for natural continuation"

This reverts commit 1105ed89e6cc41a0de777bc3b1dff9d157589d23.

* Make task watcher persistent (loop instead of one-shot)

watch-tasks.sh now loops continuously instead of exiting after
detecting one batch of tasks. This eliminates the recurring issue
where the watcher dies between task detections, causing delayed
task pickup (e.g. the Discord video send delay).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add proactive Discord DM support to bridge

The bridge now scans results/proactive-*.txt and sends them to the
owner's DM channel. Previously only task replies were sent back —
proactive messages (like sending a video unprompted) were silently
dropped because they had no matching pending_replies entry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix 3 CodeQL security alerts

- browser-tools.ts: Fix command injection in typeText — write
  AppleScript to temp file instead of inline shell interpolation
- conversation-server.ts: Sanitize DTMF digits in TwiML Play
  (strip non-digit chars to prevent XML injection)
- conversation-server.ts: Improve phone number masking in startup
  log (show only first 2 + last 2 digits)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Harden path traversal guards in dashboard and agent-api

- dashboard.py: Add resolve() + is_relative_to() for notes
  read and delete endpoints (supplements existing regex guard)
- agent-api.py: Remove unused mimetypes import (replaced by
  SAFE_TYPES allowlist in previous security fix)

Addresses CodeQL alerts sonichi#28-31 (dashboard) and cleans up sonichi#19-23
(agent-api already has guards, but adding belt-and-suspenders).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add 3 new call log scanner detectors from transcript analysis

New patterns detected across 263 calls:
- reconnect_leak: 37 calls with "I'm back" after Gemini reconnect
- repeated_summon: 3 calls where user asked to summon 3+ times
- repeated_tab_switch: 7 calls where tab switching failed repeatedly

Added 6 tests (20 total, was 14). Scanner now has 8 detectors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Expand security tests for CodeQL fixes (33 assertions, was 17)

New test coverage for:
- Dashboard note slug validation (7 tests: traversal, slash, null byte, injection)
- Dashboard note path resolution via is_relative_to (2 tests)
- DTMF digit sanitization for TwiML injection (5 tests)
- TypeText AppleScript injection prevention (2 tests)

Total test suite: 120 assertions across 4 suites, all passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add test runner script (4 suites, 120 assertions)

bash tests/run-all.sh runs all test suites:
- Phone access control (49 assertions)
- Browser tools / STT corrections (18 assertions)
- Security sanitization (33 assertions)
- Call log scanner (20 assertions)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix migration script: auto-detect memory dir + don't exit on optional paths

- Memory dir now auto-detected from repo path slug (falls back to
  default ~/Desktop/sutando path)
- Session dir also auto-detected
- Replaced set -e with set -uo pipefail — many migration paths
  legitimately don't exist and shouldn't abort the script
- Tested: successfully bundles .env, 22 memory files, claude config,
  session history, flywheel data into 3.2MB tarball

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add --summary flag to call scanner for quality trend tracking

Shows daily breakdown: total calls, clean vs issues, issue rate,
top patterns per day. Clear improvement trend visible:
- Mar 28-30: 80-100% issue rate (duplicates)
- Mar 31: 45% after PR sonichi#67
- Apr 2-3: 13-29% after PR sonichi#105
- Apr 4: 21% (remaining: confusion + tab switching)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add migration script tests (14 assertions, 5 suites total)

Tests cover:
- REPO_SLUG generation (no slashes, starts with dash)
- Memory dir auto-detection from repo path
- Session dir detection
- Bundle creation (env, memory, MEMORY.md)
- Script syntax validation (no set -e)

Total: 134 assertions across 5 suites, all passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add ngrok tunnel to health check (13 checks total)

Verifies the ngrok HTTPS tunnel is active — required for inbound
phone calls via Twilio. Shows tunnel URL in health output.
Only checked when Twilio is configured in .env.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add Twilio webhook URL verification script

Checks if Twilio's phone number webhook matches the current ngrok
tunnel. Stale webhooks are a common failure mode — inbound calls
silently fail. Supports --fix flag to auto-update Twilio.

Usage: bash src/check-webhook.sh [--fix]
Exit: 0=match, 1=mismatch, 2=error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add Twilio webhook verification to health check (15 checks)

Runs check-webhook.sh as part of the health check — warns if
Twilio's voice URL doesn't match the current ngrok tunnel.
Stale webhooks cause inbound calls to silently fail.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix Python 3.9 compat in github-webhook.py

Remove str | None return type annotation (requires Python 3.10+).
Susan runs Python 3.9 — this would crash on import.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Address review: fix 2 blockers + 2 yellow items (Wenjix review)

Blocker 1: check-webhook.sh Python injection — pass phone number
via env var instead of interpolating into Python source string.

Blocker 2: phone mask padEnd alignment — wrap entire masked string
in parens before padEnd. Show last 4 digits (standard compromise).

Yellow: discord-bridge proactive files — rename to .failed on send
failure instead of deleting (preserves message for inspection).

Yellow: typeText tmp file — add random suffix to prevent race,
use try/finally to ensure cleanup on execSync failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove test files from PR (tests/ is gitignored, should stay local)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove non-security changes from PR

Revert infra/tooling files to main — this PR should only contain
security fixes with CodeQL citations. Non-security changes (webhook
script, scanner detectors, health checks, migration, watcher,
Discord DM, Python 3.9 fix) moved to local infra branch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add CodeQL citation comments to all security fixes

Each fix now has an inline code comment citing the specific
CodeQL alert number and what vulnerability it addresses.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bassilkhilo-ag2
bassilkhilo-ag2 approved these changes Jun 5, 2026
View reviewed changes

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — workspace-path fix looks correct.

@qingyun-wu qingyun-wu mentioned this pull request Jul 15, 2026
Merged
@qingyun-wu qingyun-wu mentioned this pull request Oct 19, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bassilkhilo-ag2 bassilkhilo-ag2 bassilkhilo-ag2 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sonichi @bassilkhilo-ag2