Repository navigation
third_party: bump llama.cpp to 391bad862 (3 fork commits: MUL_MAT_ID packet, attention-device pin, Zamba2-VL) - #1104
Conversation
…T_ID packet, #205 attention pin, #206 Zamba2-VL) Fork base `1bit/hrx-vulkan-patched` moved 3 commits past the pin engine main carries today (6e42864a6): - #204 (engine #991) hrx: element-wise expert weights use the 32-value packet in MUL_MAT_ID - #205 (engine #1078) graph: the fused attention op stays on its layer's device. With a device that claims host buffer types (HRX does), `-ngl 0` was silently losing Flash Attention: ggml_backend_sched skips FLASH_ATTN_EXT in pass 1 and its pass-3 tie-break then hands the weightless node to backend 0, so resolve_fused_ops saw "layer on CPU, FA on HRX0", warned, and set flash_attn = false for the whole context. Pinning the node to model.dev_layer(il) keeps FA enabled and running where the K/V cache is (measured on Qwen3-4B-IQ4_XS: `-dev HRX0 -fa auto` FA disabled -1.370574 -> FA enabled -1.369900, `-dev none` unchanged at -1.084810, `-ngl 8` keeps its offloaded FA nodes). - #206 (engine #1098) convert: Zamba2-VL (Zamba2 language model + Qwen2.5-VL tower), so the published GGUFs have an authoring site. registry/architectures.json regenerated, and it is not a pin-line-only change: the converter commit adds one architecture, so `hrx 297 -> 298`, `architectures 297 -> 298`, `mapped 297 -> 298`, with the new row `Zamba2_VLForConditionalGeneration -> zamba2 (hrx)`. A reader bumping the pin should expect that row. Checks: tools/registry_build.py --check-pins records the committed pins, tools/check_pins.py ok, tests/check_pins_test.py, and the fork was compiled and exercised at 391bad862 by its own PR gates (#204, #205, #206 each merged green). The engine build itself is unchanged - one gitlink and one registry file. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at ac1e67e9 — the bump is exactly the three commits, and I ran the registry tool against the pinned submodule rather than reading the diff.
$ git -C third_party/llama.cpp rev-parse --short HEAD -> 391bad862 (a worktree of the fork at the pin)
$ git -C <fork> rev-list --count 6e42864a6..391bad862 -> 3
391bad862 convert: Zamba2-VL … (#206)
69515bcef graph: keep the fused attention op on its layer's device (#205)
a4babdfd4 hrx: let element-wise expert weights use the 32-value packet in MUL_MAT_ID (#204)
$ python3 tools/registry_build.py --check-pins
/home/bcloud/wt/prs-1104/registry/architectures.json records the committed pins
$ python3 tools/registry_build.py --check-gaps
none of the 28 reviewed significant classes is mapped without a reason
$ python3 tests/check_pins_test.py
Ran 3 tests — OK
That is the check I failed myself on #1075 (I moved a gitlink without regenerating), so I ran it here rather than trusting the body. The registry delta is right in every count: hrx 297 → 298, architectures 297 → 298, mapped 297 → 298, the pin line 6e42864a6… → 391bad862…, and the new row Zamba2_VLForConditionalGeneration → gguf "zamba2", backends ["hrx"] — which is the consequence of #206 landing in the same bump and is why this one is not a pin-line-only change.
All three commits were reviewed by me at their own PRs (#204 predicate probe, #205 placement fix, #206 byte-identical restore + the published template sha verified by range fetch), so the bump carries reviewed content.
Not verified: a device run at the new pin. That needs the hw_gate (label it hw-gate and it will run on this box's self-hosted runner) — worth doing for a bump that now carries #204's matcher change and #205's scheduler change into the HRX build, since GGML_HRX is off in ci.yml and nothing else compiles that half (engine #1077).
What
third_party/llama.cpp6e42864a6→391bad862, the three commits the fork's1bit/hrx-vulkan-patchedhas gained since the pin enginemaincarries today, plus the regenerated registry line:MUL_MAT_ID-ngl 0when a device is presentWhat I verified
The registry change is not pin-line-only, which is worth knowing before the bump lands: the converter commit adds one architecture, so
tools/registry_build.pymoveshrx 297 → 298,architectures 297 → 298,mapped 297 → 298and addstools/registry_build.py --check-pins→ records the committed pins;tools/check_pins.py→ ok;tests/check_pins_test.py→ 3/3 OK;tools/copyright.py --check→ clean. Two files, +11/−5 (third_party/llama.cpp,registry/architectures.json).#205's own before/after, measured on this box at the fork head (Qwen3-4B-IQ4_XS,
-ngl 0,-c 512 -b 32 -ub 32,n_predict: 1,n_probs: 8, greedy, onegpu-holdper arm set):-dev none-dev HRX0-dev HRX0FLASH_ATTN_EXT x 36-dev HRX0 -ngl 8Honest scope, because it matters for what this bump does not buy: it does not make
-dev HRX0equal-dev none(0.285 nats remain, from ~2/3 of the graph running on HRX over host buffers at-ngl 0), and it does not change long-form quality on the HRX route —Qwen3-4B-IQ4_XSloops 15× on the sky prompt at every-nglwhileQwen3-4B-Q4_K_Mwrites prose. Those are engine #1078's remaining rows, not this pin's.What the reviewer should run
The HRX compile and the ops matrix at the new pin — the
hw-gatelabel — because that is the only check a pin bump cannot get fromci.yml(engine #1077).No merge from me.
Refs engine #1078, #991, #1098.