Windows: CUDA Backend Fails to Initialize RTX 4050 While Vulkan Backend Works #96

Version
• Build v0.3.0 (20999cc)
• Tested package: nemo-speech-0.3.0-windows-x86_64-vulkan.zip
• Tested package: nemo-speech-0.3.0-windows-x86_64-cuda.zip

Environment
• OS: Windows 11
• GPU: NVIDIA RTX 4050 Laptop GPU
• NVIDIA Driver: 555.97
• CUDA Version: 12.5

NVIDIA GPU Detection

nvidia-smi -L

GPU 0: NVIDIA GeForce RTX 4050 Laptop GPU
(UUID: GPU-1222eb33-44c3-5916-94dd-37982642f7fa)

Results
• ✅ Vulkan backend initializes successfully
• ✅ Vulkan transcription works
• ✅ Vulkan diarization works
• ✅ AMD Radeon 780M processes inference
• ✅ RTX 4050 is detected by nvidia-smi
• ❌ CUDA backend fails to initialize/run
• ❌ ggml_cuda_init returns (null)

Notes
The Vulkan backend works correctly using the same:
• Model
• Audio file
• Machine
• Build

This suggests the issue is specific to CUDA initialization or device detection rather than the model, audio file, or transcription pipeline.

Vulkan Output
nemo-speech --verbose transcribe audio.wav
--format srt
--model parakeet-tdt-0.6b-v3.q8_0.gguf
--output output.srt

[nemo-speech] transcribe session started
transcribe: model=parakeet-tdt-0.6b-v3.q8_0.gguf inputs=1 concurrency=1 device=0
[asr] model=parakeet-tdt-0.6b-v3.oss-align.q8_0 head=tdt backend=Vulkan0
[asr] mode=offline head=tdt attention=full-context

CUDA Error
transcribe: model=parakeet-tdt-0.6b-v3.q8_0.gguf inputs=1 concurrency=1 device=0
ggml_cuda_init: failed to initialize CUDA: (null)
nemo-speech transcribe: use_gpu=true but no matching GPU device found (gpu_device_idx=0)
[nemo-speech] transcribe session failed (exit code 1)

Expected Behavior
CUDA backend should initialize and run successfully on the RTX 4050, similar to how the Vulkan backend initializes and runs successfully on the AMD Radeon 780M.

Additional Observation

The --verbose flag produces some additional output, such as model and backend information for the Vulkan backend.

According to --help:

--verbose Emit additional diagnostics on stderr

However, during CUDA initialization the diagnostic output remains very limited.

The CUDA backend reports only:
ggml_cuda_init: failed to initialize CUDA: (null)
nemo-speech transcribe: use_gpu=true but no matching GPU device found (gpu_device_idx=0)
[nemo-speech] transcribe session failed (exit code 1)

No further CUDA initialization details or error information are emitted, making it difficult to diagnose the failure.

The NVIDIA GPU is visible to CUDA tooling via nvidia-smi, but the CUDA backend is unable to detect or initialize it.

This may be relevant on hybrid AMD/NVIDIA systems where Vulkan selects the integrated AMD Radeon 780M while CUDA targets the discrete NVIDIA RTX 4050.

Is there a way to enable more detailed CUDA logging or debug telemetry in the Windows builds to help diagnose CUDA initialization failures?

Summary
Vulkan (AMD Radeon 780M): PASS
CUDA (NVIDIA RTX 4050): FAIL

This appears to be a CUDA initialization and/or device detection issue specific to the Windows CUDA build.