docs(stream): clarify VAD mode timestamps and CWD model path errors - #4019
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the whisper-stream example documentation to make its stdout output (especially in VAD / --step 0 mode) easier to parse reliably, and to clarify how model path resolution depends on the process working directory.
Changes:
- Documented the two distinct stdout output shapes for
--step > 0vs--step 0(VAD mode), including the VAD block markers and timestamped segment lines. - Clarified that
-m/--modelis resolved relative to the process CWD and that CWD mistakes can surface as the generic “failed to initialize whisper context” error.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@danbev - you wrote #3065, so you're the right person to ask about the part I couldn't settle while writing this up. When VAD is on, the timestamps that come back are relative to the filtered audio and get mapped back onto the original timeline. Reading the review thread on #3065, the mapping scales speech back across the original segment length rather than preserving where the silence actually sat. For a caller doing word-level alignment, is that difference meant to be treated as noise, or is preserving true silence positions something you'd want a follow-up for? I hit this building a local voice assistant with a "Delphi" wake word on top of whisper-stream, so my failure mode was practical rather than theoretical: I was discarding timestamped lines as log noise until I realised the prefix was the data. I'm a third-year CS student and I'm using this project mostly as an excuse to get better at reading code I didn't write. Thanks for #3065 either way. |
|
@apollo-2006 The stream example uses vad_simple and not the VAD implementation from #3065. For your question, please take a look at #3910 which may be relevant. |
|
Thanks, that's the part I had wrong. I was looking in the wrong place. Checked #3910 does answer what I was actually asking. Snapping a token that lands in removed silence to the nearer boundary, instead of interpolating across the gap, is the behaviour I meant. Merged, so nothing needed there. One thing I did find: in |
AI use: I found this while building against the library and used an AI assistant to help verify the relevant source (file/line refs above). The report and the documentation wording are mine, and I've checked every claim against the source myself.
|
@danbev this is out of draft and ready for review. Force-pushing does not notify, so flagging it here. One thing I noticed rereading the sample output: the block prints Worth a sentence distinguishing them here, or is that out of scope for a stream README? |
Co-authored-by: Daniel Bevenius <daniel.bevenius@gmail.com>
* ggerganov/master: (81 commits) fix(yt-wsp): Resolve script path without GNU realpath (ggml-org#4072) cli : load backends after validating input files (ggml-org#4069) ci : update android-actions to v4.0.4 (ggml-org#4074) docs : clarify VAD mode timestamps and CWD model path errors (ggml-org#4019) readme : document the ANEForge encoder backend (ggml-org#4073) whisper : optional ANEForge encoder backend (Apple Neural Engine) (ggml-org#3905) whisper : fix int overflow in whisper_full_parallel chunk offsets (ggml-org#4044) sync : ggml ggml : bump version to 0.24.0 (ggml/1627) tests(s390x): add non-vxe build to tests (llama/28776) sycl: rfc: Use radix select for top_k (llama/28670) ggml-cpu : disable PCH and fix CACHE_LINE_SIZE ambiguity to fix heap corruption (llama/28882) sycl : fix oneDNN scratchpad breaking the pool free order (llama/28704) ggml-cuda: fallback to F32 on device without BF16 hardware acceleration (llama/28846) ggml-cpu(s390x): guard VXE-only repack helpers (llama/28775) sycl : Fix get mem error (llama/28227) vulkan: workaround NV queuesubmit driver bug (llama/28830) opencl: apply the noshuffle row-alignment rule to q4_K, q5_K and q8_0, not just q6_K (llama/28575) ggml-cuda: hip add specific config table for AMD GCN (llama/27841) syscl : Handle (fail gracefully) unsupported tq1_0 quants (llama/28681) ...
Added documentation for undocumented timestamp behavior in VAD mode and clarified CWD errors to prevent parser breaks," and hit submit
AI use: I found this while building against the library and used an AI assistant to help verify the relevant source (file/line refs above). The report and the documentation wording are mine, and I've checked every claim against the source myself.