Stream repetition canceller — wayfinder map #53

Destination

The stream-repetition-canceller is merge-ready: every open design decision about the repetition guard prototype is resolved, so the prototype on feat/stream-repetition-canceller can be finalized, reviewed, and merged on the fork (weselben/GoModel; never upstream ENTERPILOT/GoModel without explicit user request).

Notes

• Domain: GoModel AI gateway; prototype lives in worktree /home/agent/workspaces/gomodel/.worktrees/stream-repetition-canceller, branch feat/stream-repetition-canceller.
• Prototype status: built and verified (unit tests + brown test). Commits 825d10e3..ae2c283a. Implementation follow-ups from decisions: remove holdback queue + straddling rewrite; per-chunk eager trigger; metric/log per Observability: log line only, or also a Prometheus metric on trigger? #61; kill switch per Kill-switch env + finish_reason on truncated SSE #63; tokenizer per Tokenizer library selection for the repetition guard #64.
• Every grilling session must invoke the grilling and domain-modeling skills.
• The user drives tickets from their phone; keep questions short and answerable in a reply.
• Standing rule: Postel's law + KISS; default behavior unchanged when STREAM_REPETITION_LIMIT is unset/0.
• Map hygiene: when ambiguity emerges, reopen closed tickets and record the why here. When a decision moves, update dependent tickets.

Decisions so far

• Detection unit — token-based detection chosen; blocked on tokenizer selection; byte-period stays as fallback until then.

• Holdback vs eager — eager per-chunk check, no holdback, no back-truncation; leak of up to one detection window accepted.

• On-trigger action — silent cut + server-side metric/log; client intact; end turn so a new request unblocks quickly; skip obviously-code repetition.

• Keep semantics — superseded: no rewrite/strip step under eager mode.

• Detection unit — B: token-based via tiktoken; own file; lazy-loaded only when guard active for a model. (Closed after Tokenizer library selection for the repetition guard #64 unblocked.)

• Tokenizer selection — tiktoken, latest/fastest pure-Go port; lazy-loaded; byte-period as fallback when vocabulary missing. (Closed.)

• Kill switch / per-model activation — opt-in per-model field in the Models view (next to slowdown); global default stays as off; no separate runtime kill switch. (Closed.)

• Prior art survey — reopened pending token lens (Tokenizer library selection for the repetition guard #64); byte-period defaults (max_unit 64 / min_repeats 3 / window 4096) retained as the fallback path's reference.

• Observability — B: slog + counter gomodel_stream_repetition_triggers_total{provider,model}; no histogram for v1.

• Config granularity — global default (env/YAML) + per-virtual-model override; per-provider rejected; threshold may vary per model.

• Virtual-model semantics + max-pattern knob — 0 beats global / unset inherits; repetition_max_pattern per-model field + STREAM_REPETITION_MAX_PATTERN global (default 8); env names unchanged. (Closed.)

Not yet specified

• Provider dialect coverage (reasoning_content / Responses / Anthropic) and passthrough-route wrapping — pending Dialect scope: chat completions only, or also reasoning_content / Responses API / Anthropic? #56.

Build plan

All decisions landed. The remaining work is implementation, not decisions. Each step lists the ticket(s) it derives from.

1. Token-based detection per chunk — task Replace byte-period detection with token-based per-chunk detection #66.
2. Lazy tiktoken loader — task Add lazy tiktoken loader with byte-period fallback #67.
3. Prometheus counter — task Add Prometheus counter for repetition triggers #68.
4. Per-virtual-model field — task Per-virtual-model repetition_limit override field #69.
5. Passthrough route coverage — task Extend repetition guard to passthrough routes #70.
6. Obvious-code skip heuristic — task Add obvious-code and tool-call skip heuristic #71.
7. Docs, YAML, smoke test — task Documentation, YAML example, and end-to-end smoke test #72.

Out of scope