Repository navigation
Expose confidence estimation / token probabilities (currently always 1.0) #55
Description
Description:
I'm running nemotron-3.5-asr-streaming-0.6b via nemo-speech.cpp's /v1/audio/transcriptions endpoint. The confidence field in the response is always exactly 1.0, regardless of whether the transcription is actually correct.
Concrete example: audio saying "chi è Alfonso?" (Italian) is transcribed as "che è alfonso" (wrong word) — still returns confidence: 1.0. It also returns 1.0 on pure silence and on clearly hallucinated/garbled output.
I switched to running the same model through the full NeMo Python toolkit directly (not this C++ port), enabling decoding_cfg.confidence_cfg with the entropy-based method (tsallis entropy, entropy_norm: exp) documented in NeMo's own docs. That gives real, varying per-word confidence values (e.g., 0.02–0.9 depending on the utterance), which is genuinely useful for rejecting hallucinated output — something a flat 1.0 can't do.
Is there a plan to expose this confidence estimation in nemo-speech.cpp's API? The underlying decoder already computes per-token probabilities to pick output tokens, so it seems like this would be a matter of exposing an existing internal computation rather than a fundamental limitation — similar to a related open request on transcribe.cpp (#164) asking for per-token logprobs on Parakeet TDT/Qwen3-ASR.
Happy to share the exact confidence_cfg I used if useful!
Activity
pskrunner14 commentedon Sep 30, 2026
@NatakuZ agreed this is a useful feature we should expose. Will take this up soon and yes would be great if you can share your cfg and more failure cases!
NatakuZ commentedon Sep 30, 2026
cd "$(dirname "$0")/.." || exit 1
. services/lib.sh
start() {
if svc_up "$HEALTH"; then
pgrep -f -- 'nemo-speech serve' 2>/dev/null | head -1 > "$PIDFILE"
echo "$NAME: already running"; return 0
fi
if [ ! -x "$NEMO_BIN" ]; then
echo "$NAME: FAILED — $NEMO_BIN not found or not executable"; return 1
fi
echo "$NAME: starting..."
nohup "$NEMO_BIN" serve --host 127.0.0.1 --port 8090 --device cuda:0
--asr-model nemotron-3.5 \ (if possibile with p)
--tts-model magpie
--codec-model nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps
--tokenizer-dir nvidia/magpie_tts_multilingual_357m
--api-key "$(cat ~/.nemo_key)" --no-ui --access-log
</dev/null >"$LOG" 2>&1 &
echo $! >"$PIDFILE"
if wait_health "$HEALTH" 120; then
echo "$NAME: ready"
else
echo "$NAME: FAILED to come up — check $LOG"; return 1
fi
}
stop() { stop_pidfile "$NAME" "$PIDFILE"; }
status() { if svc_up "$HEALTH"; then echo "$NAME: up ($NEMO_URL)"; else echo "$NAME: down"; return 1; fi; }
dispatch start stop status "${1:-}"
for now i am using the python system with Parakeet , before i in the start thinking the issue is from the model, and i swap with nemotron-3.5, b