cpu_x86: Q4_K reads the GGUF bytes below AVX-512, one copy of the weights (#577)

[geisten]

Summary

Before: below AVX-512, cpu_x86 repacked every Q4_K tensor into the x8 layout. That kept a second full copy of the weights next to the mapped GGUF and gave no speed advantage on that tier.

After: below AVX-512, Q4_K linears dot the GGUF blocks directly, so the weights exist once, in the mapped file. On AVX-512 the repack stays the default, because the raw kernel's prefill is about 2× slower there.

Part of #577.

Changes

• linear_q4k_raw.{c,h} (new) adds the raw Q4_K kernel:
  • Activations are quantized to int8 once per call, with one scale per 256 elements and an int32 sum per 32 elements.
  • The nibbles go straight into maddubs/madd, and the dmin offset is taken from the activation sums.
  • M>1 processes 4 tokens per weight-row pass, parallelised with OpenMP.
• backend.c picks the path automatically: raw unless q4kx8_avx512_usable(). GEIST_Q4K_RAW=1/0 overrides the choice.
• q4kx8_avx512_usable() is factored out of the AVX-512 kernel so the gate and the kernel agree.
• tests/test_x86_q4k_raw_unit.c compares against cpu_scalar with a derived activation-rounding bound. The worst ratio is 0.20, and a mutation check makes 47 cases fail.
• CHANGELOG entry.

Testing

• make format-check passes.

• make test-unit passes (84 passed, 31 skipped, 0 failed), both natively (AVX-512 VNNI) and with GEIST_FORCE_ISA=avx2.

• Peak RSS on a synthetic 1B Q4_K model (spm tokenizer, set_prompt plus a forward pass), AVX2 tier:

  Path	Peak RSS
  Repack	1308.7 MiB
  Raw	682.7 MiB (−48 %)

• bench_revision_ab.py comparing main against this branch with GEIST_FORCE_ISA=avx2 and 6 cycles: prefill −5.1 % / −3.6 % and decode −4.7 % / −2.3 %, all within noise.

API impact

None. There is one new optional env var, GEIST_Q4K_RAW.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AD86DiU1PpDqYprALdARAw

---

Generated by Claude Code