Repository navigation
HRX: bump llama.cpp to 1a5dcba2 (engine#390 strict mode and placement report) - #424
Conversation
… report) One commit on top of the pin: llama.cpp PR #102, already verified on gfx1151. The pair with the held-pin backport (hrx-system 3319e5361) is being built to confirm before merge.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
Moves
third_party/llama.cppfromad18320ato1a5dcba2— one commit, llama.cpp PR #102 — so engine#390's strict mode and placement report reach engine builds.registry/architectures.jsonis regenerated against the new pin byscripts/registry-regen.sh(--check-gapsand--check-pinspass).What lands
Two env-gated instruments in the HRX backend, off by default, with no change to placement:
GGML_HRX_STRICT=1GGML_HRX_REPORT=12= every graph)GGML_SCHED_DEBUG>=1The line is
placement_observer: placement: <N> nodes, <M> off HRX (<OP> x k, …), <S> graph splits. The hook is one observer call in the scheduler (ggml-backend.{h,cpp}) because only the scheduler knows every node's final backend — the HRX backend sees only its own splits, which is exactly why "what did we silently put on the CPU" had no answer before.Why this is the right shape for the issue
The issue asks for three things. This PR delivers the first two to the engine:
-ub 4096:"model.input_embed" is GET_ROWS f32 ne=[5120,4096,1,1] and is not placed on HRX; 438 of 3655 nodes off HRX (MUL_MAT x 389, GET_ROWS x 1, SSM_CONV x 48), 806 splits, exit 134.3655 nodes, 0 off HRX (none), 1 graph splits; on Ornith-1.5-35B (qwen35moe, 256 experts)3727 nodes, 0 off HRX (none), 1 graph splits. That second result is the substantive finding: a per-token expert gather runs fully on HRX with a single split, so the ~7.1x like-for-like decode gap is not explained by generic MoE splits. The only confirmediq4_xscoverage gap isGET_ROWS be1=7(test-backend-ops support -b HRX0).Ask 3 (explain the Flash-Next gap) is not answered here, and the issue stays open for it: Flash-Next is ~88 GB against the Strix Halo box's currently available RAM, so the histogram run is pending a window with ~90 GB free.
Verification
Built from a fresh build directory at exactly this pair (engine
7d44c99,third_party/llama.cpp1a5dcba2,third_party/hrx-system3319e5361),-DONEBIT_HRX=ON,ONEBIT_HRX_TOOLCHAIN=/home/bcloud/sdk-gfx1201-devel, on gfx1151/Strix Halo:cmake --build --target llama_hrx onebittest-backend-ops -b HRX0Backend HRX0: OK, 0 failuresGGML_HRX_REPORT=1 -vplacement_observer: placement: 986 nodes, 0 off HRX (none), 1 graph splits, exit 0GGML_HRX_STRICT=1That is asks 1 and 2 demonstrably working in an engine build, not only on the fork, and the instrument ships inert when the variables are unset. Evidence:
team/evidence/390-engine-pin.mdwith the raw390-engine-pin-{demo,tbo}.log.