You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Realtime ASR with language auto: return the detected language per utterance #90
Return the detected language for each utterance when Nemotron 3.5 ASR Streaming runs with language: auto:
in the realtime WebSocket (/v1/audio/transcriptions/realtime): a language field (for example "fr-FR") on each conversation.item.input_audio_transcription.completed event, and ideally on .delta events once known;
in nemo-speech transcribe --json with --stream/endpointing: the language per utterance or per word, not only a file-level list.
Why
docs/asr/models.md says that with auto, "Structured results include the selected or detected language". For live transcription of conversations that switch between languages (for example English and French), the client needs the language of each utterance, for display and to route translation. The model already conditions on a language-ID prompt, so the information exists inside the engine.
What I see (v0.2.0, nvidia/nemotron-3.5-asr-streaming-0.6b Q8_0, Vulkan)
Audio: 88 s, 60 s English (test_files/diar/ami_en2002d_2132.wav) followed by 28 s French.
Realtime WebSocket, session.update with "language": "auto": session.updated echoes language: auto. The 20 .completed events have only audio_processed, event_id, transcript, type and words (word, start, end, confidence, speaker). The French utterances are transcribed correctly in French, but no event says which language they are.
nemo-speech transcribe en_fr.wav --language auto --stream --endpointing --json: the result has a single "languages": ["fr-FR"] for the whole file, although most of the audio is English.
Activity
fix(asr): report the detected language per utterance and word