Skip to content

third_party: bump llama.cpp to 391bad862 (3 fork commits: MUL_MAT_ID packet, attention-device pin, Zamba2-VL) - #1104

Merged
bong-water-water-bong merged 2 commits into
mainfrom
1bit/pin-fa-layer-device
Nov 5, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
1bit/pin-fa-layer-device

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

What

third_party/llama.cpp 6e42864a6 → 391bad862, the three commits the fork's 1bit/hrx-vulkan-patched has gained since the pin engine main carries today, plus the regenerated registry line:

What I verified

The registry change is not pin-line-only, which is worth knowing before the bump lands: the converter commit adds one architecture, so tools/registry_build.py moves hrx 297 → 298, architectures 297 → 298, mapped 297 → 298 and adds

"Zamba2_VLForConditionalGeneration": { "gguf": "zamba2", "backends": ["hrx"] }

tools/registry_build.py --check-pins → records the committed pins; tools/check_pins.py → ok; tests/check_pins_test.py → 3/3 OK; tools/copyright.py --check → clean. Two files, +11/−5 (third_party/llama.cpp, registry/architectures.json).

#205's own before/after, measured on this box at the fork head (Qwen3-4B-IQ4_XS, -ngl 0, -c 512 -b 32 -ub 32, n_predict: 1, n_probs: 8, greedy, one gpu-hold per arm set):

arm before after
-dev none FA enabled, −1.084810 unchanged, −1.084810
-dev HRX0 FA disabled (placement mismatch), −1.370574 FA enabled, −1.369900
placement, -dev HRX0 1446 nodes, 288 off HRX 1266 nodes, 180 off HRX, now counting FLASH_ATTN_EXT x 36
-dev HRX0 -ngl 8 — FA enabled, offloaded layers keep their FA nodes, −1.562199

Honest scope, because it matters for what this bump does not buy: it does not make -dev HRX0 equal -dev none (0.285 nats remain, from ~2/3 of the graph running on HRX over host buffers at -ngl 0), and it does not change long-form quality on the HRX route — Qwen3-4B-IQ4_XS loops 15× on the sky prompt at every -ngl while Qwen3-4B-Q4_K_M writes prose. Those are engine #1078's remaining rows, not this pin's.

What the reviewer should run

The HRX compile and the ops matrix at the new pin — the hw-gate label — because that is the only check a pin bump cannot get from ci.yml (engine #1077).

No merge from me.

Refs engine #1078, #991, #1098.

…T_ID packet, #205 attention pin, #206 Zamba2-VL)

Fork base `1bit/hrx-vulkan-patched` moved 3 commits past the pin engine main carries today (6e42864a6):

- #204 (engine #991) hrx: element-wise expert weights use the 32-value packet in MUL_MAT_ID
- #205 (engine #1078) graph: the fused attention op stays on its layer's device. With a device that claims host
  buffer types (HRX does), `-ngl 0` was silently losing Flash Attention: ggml_backend_sched skips
  FLASH_ATTN_EXT in pass 1 and its pass-3 tie-break then hands the weightless node to backend 0, so
  resolve_fused_ops saw "layer on CPU, FA on HRX0", warned, and set flash_attn = false for the whole context.
  Pinning the node to model.dev_layer(il) keeps FA enabled and running where the K/V cache is (measured on
  Qwen3-4B-IQ4_XS: `-dev HRX0 -fa auto` FA disabled -1.370574 -> FA enabled -1.369900, `-dev none` unchanged
  at -1.084810, `-ngl 8` keeps its offloaded FA nodes).
- #206 (engine #1098) convert: Zamba2-VL (Zamba2 language model + Qwen2.5-VL tower), so the published GGUFs
  have an authoring site.

registry/architectures.json regenerated, and it is not a pin-line-only change: the converter commit adds one
architecture, so `hrx 297 -> 298`, `architectures 297 -> 298`, `mapped 297 -> 298`, with the new row
`Zamba2_VLForConditionalGeneration -> zamba2 (hrx)`. A reader bumping the pin should expect that row.

Checks: tools/registry_build.py --check-pins records the committed pins, tools/check_pins.py ok,
tests/check_pins_test.py, and the fork was compiled and exercised at 391bad862 by its own PR gates (#204,
#205, #206 each merged green). The engine build itself is unchanged - one gitlink and one registry file.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at ac1e67e9 — the bump is exactly the three commits, and I ran the registry tool against the pinned submodule rather than reading the diff.

$ git -C third_party/llama.cpp rev-parse --short HEAD           -> 391bad862     (a worktree of the fork at the pin)
$ git -C <fork> rev-list --count 6e42864a6..391bad862            -> 3
   391bad862 convert: Zamba2-VL … (#206)
   69515bcef graph: keep the fused attention op on its layer's device (#205)
   a4babdfd4 hrx: let element-wise expert weights use the 32-value packet in MUL_MAT_ID (#204)

$ python3 tools/registry_build.py --check-pins
   /home/bcloud/wt/prs-1104/registry/architectures.json records the committed pins
$ python3 tools/registry_build.py --check-gaps
   none of the 28 reviewed significant classes is mapped without a reason
$ python3 tests/check_pins_test.py
   Ran 3 tests — OK

That is the check I failed myself on #1075 (I moved a gitlink without regenerating), so I ran it here rather than trusting the body. The registry delta is right in every count: hrx 297 → 298, architectures 297 → 298, mapped 297 → 298, the pin line 6e42864a6… → 391bad862…, and the new row Zamba2_VLForConditionalGeneration → gguf "zamba2", backends ["hrx"] — which is the consequence of #206 landing in the same bump and is why this one is not a pin-line-only change.

All three commits were reviewed by me at their own PRs (#204 predicate probe, #205 placement fix, #206 byte-identical restore + the published template sha verified by range fetch), so the bump carries reviewed content.

Not verified: a device run at the new pin. That needs the hw_gate (label it hw-gate and it will run on this box's self-hosted runner) — worth doing for a bump that now carries #204's matcher change and #205's scheduler change into the HRX build, since GGML_HRX is off in ci.yml and nothing else compiles that half (engine #1077).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant