Skip to content

HRX: bump llama.cpp to 1a5dcba2 (engine#390 strict mode and placement report) - #424

Merged
bong-water-water-bong merged 1 commit into
mainfrom
1bit/390-engine-pin
Oct 16, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
1bit/390-engine-pin

Conversation

@bong-water-water-bong

@bong-water-water-bong bong-water-water-bong commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Moves third_party/llama.cpp from ad18320a to 1a5dcba2 — one commit, llama.cpp PR #102 — so engine#390's strict mode and placement report reach engine builds. registry/architectures.json is regenerated against the new pin by scripts/registry-regen.sh (--check-gaps and --check-pins pass).

What lands

Two env-gated instruments in the HRX backend, off by default, with no change to placement:

variable effect
GGML_HRX_STRICT=1 fatal, named error on the first graph node not placed on HRX
GGML_HRX_REPORT=1 one placement line for the first graph (2 = every graph)
GGML_SCHED_DEBUG>=1 also turns the line on
unset no output, no placement change

The line is placement_observer: placement: <N> nodes, <M> off HRX (<OP> x k, …), <S> graph splits. The hook is one observer call in the scheduler (ggml-backend.{h,cpp}) because only the scheduler knows every node's final backend — the HRX backend sees only its own splits, which is exactly why "what did we silently put on the CPU" had no answer before.

Why this is the right shape for the issue

The issue asks for three things. This PR delivers the first two to the engine:

  1. strict mode — demonstrated by an actual refusal on gfx1151 at -ub 4096: "model.input_embed" is GET_ROWS f32 ne=[5120,4096,1,1] and is not placed on HRX; 438 of 3655 nodes off HRX (MUL_MAT x 389, GET_ROWS x 1, SSM_CONV x 48), 806 splits, exit 134.
  2. the placement report — on Qwen3.8-27B dense it prints 3655 nodes, 0 off HRX (none), 1 graph splits; on Ornith-1.5-35B (qwen35moe, 256 experts) 3727 nodes, 0 off HRX (none), 1 graph splits. That second result is the substantive finding: a per-token expert gather runs fully on HRX with a single split, so the ~7.1x like-for-like decode gap is not explained by generic MoE splits. The only confirmed iq4_xs coverage gap is GET_ROWS be1=7 (test-backend-ops support -b HRX0).

Ask 3 (explain the Flash-Next gap) is not answered here, and the issue stays open for it: Flash-Next is ~88 GB against the Strix Halo box's currently available RAM, so the histogram run is pending a window with ~90 GB free.

Verification

Built from a fresh build directory at exactly this pair (engine 7d44c99, third_party/llama.cpp 1a5dcba2, third_party/hrx-system 3319e5361), -DONEBIT_HRX=ON, ONEBIT_HRX_TOOLCHAIN=/home/bcloud/sdk-gfx1201-devel, on gfx1151/Strix Halo:

check result
cmake --build --target llama_hrx onebit exit 0, 94 s, no reuse of any earlier build tree
instrument present in the engine build `nm -D libggml-base.so
op gate test-backend-ops -b HRX0 1141/1141 passed, Backend HRX0: OK, 0 failures
tooling, unset silent, exit 0
tooling, GGML_HRX_REPORT=1 -v placement_observer: placement: 986 nodes, 0 off HRX (none), 1 graph splits, exit 0
tooling, GGML_HRX_STRICT=1 no abort (nothing is off HRX — the correct non-fire), exit 0
llama-bench, Qwen3-0.6B Q4_K_M pp32 2952.58, tg8 309.13 tok/s (contended: load 10.6, Tctl 46→48 C)

That is asks 1 and 2 demonstrably working in an engine build, not only on the fork, and the instrument ships inert when the variables are unset. Evidence: team/evidence/390-engine-pin.md with the raw 390-engine-pin-{demo,tbo}.log.

… report)

One commit on top of the pin: llama.cpp PR #102, already verified on gfx1151.
The pair with the held-pin backport (hrx-system 3319e5361) is being built to
confirm before merge.
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

390 - Partially compliant

Compliant requirements:

  • Strict mode is implemented and tested, demonstrating a fatal error when an op cannot be placed on HRX
  • Placement report is implemented and logs node placement and graph splits
  • The implementation uses a scheduler hook in ggml-backend.{h,cpp} to observe final backend placement

Non-compliant requirements:

  • The PR does not include a mechanism to detect and report which ops ran on the CPU silently (this is a broader issue beyond the scope of this PR)

Requires further human verification:

  • Verification that the strict mode and placement report work correctly in all scenarios
  • Confirmation that the measurements and performance claims in the PR description are accurate
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Subproject Commit Update

The commit hash for the llama.cpp subproject has been updated from ad18320a to 1a5dcba2. While this is a standard update, it's important to verify that this change doesn't introduce any regressions or unexpected behavior in the HRX backend, especially in relation to the new strict mode and placement reporting features.

Subproject commit 1a5dcba2aa6d91dacec3c3c00a37dd77aa9c6084
Registry Update

The registry file has been updated to reflect the new commit hash for llama.cpp (hrx). This update is necessary to ensure that the registry accurately reflects the pinned version of the dependency. It's important to confirm that this change aligns with the intended behavior and that the registry is correctly updated.

"llama.cpp (hrx)": "1a5dcba2aa6d91dacec3c3c00a37dd77aa9c6084"

@bong-water-water-bong
bong-water-water-bong merged commit 2df1a9b into main Oct 16, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/390-engine-pin branch October 16, 2026 04:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant