realtime WS: commit never yields .completed for utterances >= ~1s (0.5s works) #49

Summary

The realtime WebSocket transcription path in nemo-speech serve 0.1.0 never emits .completed (or input_audio_buffer.committed) after input_audio_buffer.commit once the streamed utterance reaches ~1 second or more. At 0.5 s it works (+0.19 s finalization); at 1.0 s, 1.5 s, and 2.0 s it hangs forever. Short-utterance success shows the protocol handling is correct, so this looks length-dependent in the streaming finish path.

A second, possibly related symptom: with --asr.endpointing.enable=true and 2 s of trailing zero-PCM silence, no endpoint .completed ever fires and audio_processed stalls partway through the stream.

Environment

• Binary: nemo-speech 0.1.0 release archive nemo-speech-0.1.0-linux-x86_64-cuda.tar.gz
• OS/GPU: NixOS 26.11, NVIDIA T1000 4 GB, driver 595.91, CUDA backend (backend=CUDA0)
• Model: nemotron-speech-streaming-en-0.6b.q8_0.gguf (665 MiB)
• Serve: nemo-speech serve --asr-model <gguf> --asr.batching.state_arena_slots 1 --host 127.0.0.1 --port 8080
• Note: default arena slots OOM (cudaMalloc failed: out of memory) with ~2 GB already resident on the 4 GB card; state_arena_slots 1 was required to run at all.

Repro

Raw WebSocket client (stdlib only, masked client frames), repeated per trial on a fresh connection:

1. GET /v1/realtime handshake, receive session.created.
2. Send {"type": "session.update", "session": {"sample_rate": 16000, "language": "en-US"}}, receive session.updated.
3. Stream 16 kHz mono PCM16 as binary frames in 100 ms (3200-byte) chunks, back-to-back.
4. Send {"type": "input_audio_buffer.commit"}.
5. Read events.

Audio: 16 kHz mono command utterances; trials used 0.5 s / 1.0 s / 1.5 s / 2.0 s prefixes of the same clip.

Observed

Mid-stream .delta events flow correctly in all cases (full transcript visible within ~0.3 s of arrival, ~6x realtime), so ingestion and decoding work; only finalization never surfaces. Endpointing variant (--asr.endpointing.enable=true --asr.endpointing.stop_history_eou_ms=800, utterance + 2 s trailing silence): audio_processed advances to ~2.0-2.2 s of a 2.66 s stream then stalls; no endpoint final ever arrives.

Additional observation: streaming tail words are unstable across runs on identical input ("Play some Kendrick Lam" vs "Play some Kendrick"), and trailing silence does not flush the remainder.

Notes

• Current main's server/http/http_server.cpp commit handler (ensure_stream(); emit(stream->finish()); ...committed) is identical to 0.1.0, so this likely reproduces on main (not built here; no toolchain on the test host).
• Possibly related to Streaming RNNT wedges after sustained zero-PCM silence; Vulkan aborts with GGML_ASSERT(ne3 == ne13) #48 (streaming RNNT stall) and Token-silence EOU misfires mid-sentence, hard reset corrupts transcript #40/fix(asr): make mid-stream EOU a soft reporting checkpoint, not a hard reset #41 (EOU handling), but this repro uses endpointing off, no silence mid-stream, and fails deterministically on utterance length alone.