CUDA backend ? #10

Activity

1. richiejp commented on Oct 29, 2026

   

   richiejp

   on Oct 29, 2026

   Contributor

   More actions

   Yes, I just use Vulkan, even on my NVIDIA card because the runtime is way smaller, compilation is faster, it works on any card with the right Vulkan extensions etc. It can probably be enabled quite easily in GGML though.
2. NicTanghe commented on Oct 29, 2026

   

   NicTanghe

   on Oct 29, 2026

   Author

   More actions

   I am a scrub. and i think it whould be a bit faster during inference though?

   you think i could add it, what whould i do where do i do it and whre do i get info ?