Skip to content

server : reject a realtime session.update with another sample rate - #8

Draft
T0mSIlver wants to merge 4 commits into
realtime-testsfrom
realtime-rate
Draft

T0mSIlver wants to merge 4 commits into
realtime-testsfrom
realtime-rate

Conversation

@T0mSIlver

@T0mSIlver T0mSIlver commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Proposed body for review on the fork. Rewrite it in your own words before opening upstream (llama.cpp does not accept AI-written PR text). Stacked on the tests PR; upstream, open it after that one merges or base it on that branch. Needs an issue first (bug fix); the facts for it are in llama-rt-prs/issue-rate.md in the project files.

Overview

input_audio_buffer.append carries headerless PCM16, and the server always reads it at the model's sample rate (16 kHz for Voxtral). A client that sets session.audio.input.format.rate to something else, for example the 24 kHz that OpenAI Realtime clients default to, currently gets session.updated back and then a transcript of audio played at the wrong speed, with no error.

With this change, a session.update whose audio.input.format.rate is not an integer equal to the model rate gets an error (audio.input.format.rate must be 16000), and the session keeps its previous config. An update that does not set a rate is unchanged. The check reads the JSON value directly, because json_value() would fall back to the default for a string like "24000" and truncate a fraction.

The README also drops the stale note that held-back words are attributed to the next item, which ggml-org#30994 fixed.

Additional information

test_realtime_control_events gains three updates: 24000, "24000" (a string) and 16000.

without the C++ change:  FAILED (event 2 is 'session.updated', expected 'error')
with the change:         SLOW_TESTS=1 ./tests.sh unit/test_realtime.py -> 3 passed in 301.72s

For scale, on an RTX 3090, 24 kHz audio read as 16 kHz gave 41% WER unpaced against about 22% at 16 kHz.

Requirements


🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant