Skip to content

qsa_prompt_attn: Q4_0 KV on tensor cores (mode 4) - long prompts with --kv q4_0 ~19% faster - #452

Closed
architectds wants to merge 1 commit into
Niko1221:mainfrom
architectds:pr/q4-prompt-attn
Closed

qsa_prompt_attn: Q4_0 KV on tensor cores (mode 4) - long prompts with --kv q4_0 ~19% faster#452
architectds wants to merge 1 commit into
Niko1221:mainfrom
architectds:pr/q4-prompt-attn

qsa_prompt_attn: Q4_0 K and V on tensor cores (mode 4) - long prompts…

e71abf8
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs