Skip to content

hw-gate: build the private HIP kernels, as the release does - #478

Merged
bong-water-water-bong merged 5 commits into
mainfrom
1bit/hw-gate-private-kernels
Oct 20, 2026
Merged

bong-water-water-bong merged 5 commits into
mainfrom
1bit/hw-gate-private-kernels

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Stacked on #477 (base 1bit/pin-set-rows-q8). GitHub retargets it to main once #477 merges.

The gate built llama.cpp without the gpu-kernels add-on, so it passed main 1834ceb, whose llama.cpp and gpu-kernels pins do not compile together (dispatch-hip-set-rows-q8.cpp:39, HipSetRowsArgs). Its build phase now does what cmake/hrx.cmake (ONEBIT_GPU_PRIVATE) and scripts/weekly.sh do:

  • It fetches the add-on at the commit config/gpu-kernels.config pins, with scripts/fetch-private.sh and the runner box's gh login, and passes GGML_HRX_HIP_ADDON_DIR. It also checks that llama.cpp has the hook.
  • The build tree is keyed by the add-on choice, and the gate reconfigures when the gpu-kernels pin moves, so new kernel files are seen.
  • The build fails when libggml-hrx.so does not export ggml_hrx_hip_addon_register.
  • On a failed build, the job log shows only the compiler's error lines. The log is public and the add-on's sources are not.
  • A box that cannot read gpu-kernels stops with exit 2. HW_GATE_PRIVATE=0 gates the public plumbing only.

docs/hw-gate.md is updated, and its "Private kernels" limitation is removed.

Proof on ryzen (--phases build, ROCR_VISIBLE_DEVICES=-1, nice 19, -j8, TheRock copy):

Not verified: that the self-hosted runner's gh login can read 1bit-MONSTER/gpu-kernels. The workflow sets no GH_TOKEN for this step, so fetch-private.sh uses the runner user's own login. If the gate run on this PR ends with "cannot fetch the private gpu-kernels add-on", that login needs read access.

🤖 Generated with Claude Code

bong-water-water-bong and others added 3 commits October 20, 2026 18:08
…need (census G1)

gpu-kernels be68256d (#476) compiles addons/hip-hrx/dispatch-hip-set-rows-q8.cpp against
HipSetRowsArgs, which only the fork branch 1bit/hrx-set-rows-q8 had: with the private
kernels the release build failed at dispatch-hip-set-rows-q8.cpp:39. Fork #116 squash-merged
that branch onto 1bit/hrx-vulkan-patched (8792923f8, the same tree as the head proven here).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gate built llama.cpp without the gpu-kernels add-on, so it passed engine main 1834ceb,
whose pins do not compile together with the private kernels (dispatch-hip-set-rows-q8.cpp:39,
HipSetRowsArgs). The build phase now fetches the add-on at the pinned commit
(scripts/fetch-private.sh, the runner's gh login) and passes GGML_HRX_HIP_ADDON_DIR like
cmake/hrx.cmake; it reconfigures when the gpu-kernels pin moves, fails when libggml-hrx.so
lacks ggml_hrx_hip_addon_register, and prints only the compiler's error lines (the job log is
public). A box without read access stops with exit 2; HW_GATE_PRIVATE=0 gates the public part.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong bong-water-water-bong added the hw-gate Run the HRX hardware gate on Strix Halo (docs/hw-gate.md) label Oct 20, 2026
Base automatically changed from 1bit/pin-set-rows-q8 to main October 20, 2026 21:16
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 63b7727)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

477 - Partially compliant

Compliant requirements:

  • Pin llama.cpp to commit 8792923f8
  • Fetch and compile the private HIP kernels add-on
  • Fail build when private kernels are missing from libggml-hrx.so
  • Update docs/hw-gate.md to reflect new behavior

Non-compliant requirements:

  • None

Requires further human verification:

  • None
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Build tree keying logic

The build tree keying logic uses a SHA1 hash of the source path and PRIVATE flag to determine if a build should be reused. This approach may not be robust enough to detect changes in the private kernel add-on pin, potentially leading to stale builds if the add-on changes but the key remains the same. The key should also include the add-on pin to ensure proper rebuilds when the add-on changes.

# one persistent build tree per source path (and per add-on choice), so the runner's checkout
# rebuilds incrementally
key=$(printf '%s|private=%s' "$src" "$PRIVATE" | sha1sum | cut -c1-10)
Error message clarity

The error message when fetching the private gpu-kernels add-on fails does not provide enough context about why the fetch might have failed. It only shows the last line of the log, which may not be sufficient to diagnose issues with authentication or network problems. The error message should be more informative to help with debugging.

if ! addon=$("$ROOT/scripts/fetch-private.sh" "$ROOT/config/gpu-kernels.config" "$STORE/gpu-kernels" 2> "$OUT/private.log"); then
  note "::error::cannot fetch the private gpu-kernels add-on at ${addon_pin:0:12} (gh login with read access on this box?): $(tail -n 1 "$OUT/private.log")"
  note "  (HW_GATE_PRIVATE=0 gates the public plumbing only)"
  exit 2

⚠️ Review coverage: The following files were not included in this review because of the token budget:

  • tests/golden/hw-gate/kat-qwen3-0.6b.json
  • tests/golden/hw-gate/kat-qwen3-coder-30b-a3b.json
  • tests/golden/hw-gate/kat-qwen3-4b-iq4xs.json
  • tests/golden/hw-gate/manifest.json

…nels

The gate now builds llama.cpp with the gpu-kernels add-on (be68256d), as the release does, so the
kat goldens and the KLD base logits are recorded from that build (llama.cpp 8792923f, hrx-system
82804197). Recorded with scripts/hw-gate.sh --record --phases kat,kld,needle on strixhalo
(performance mode), from the binaries of hw-gate run 39498508232.

Why the old goldens no longer fit (ZAYA1-8B Q4_K_M, wiki.test 8 x 512, performance mode):

  KLD vs the CPU (--device none):  public build 0.0381 (p99 0.431, same top 92.45%)
                                   add-on build 0.0357 (p99 0.384, same top 92.60%)
  KLD vs the F16 model on the CPU: public 0.5798, add-on 0.5816 (+-0.019), CPU Q4_K_M 0.5986
  add-on vs the old (public) base: 0.0069; hip.mul_mat_id_prefill alone 0.0056, the
                                   dense prefill tile (ragged lm_head) alone 0.0003

The kat failure was prefix-reuse token 0, a near-tie:
  CPU:    green -0.942, The -0.964
  public: The -0.931, green -1.068  (old golden)
  add-on: green -0.860, The -1.060  (new golden)
The add-on now picks the same token the CPU picks.

Kernel checks: hrx_prefill_mmid q4_K/q6_K on the ZAYA expert tensors (T 512 and 135, top-1)
match the f16-activation reference to 7e-7; the ragged lm_head (q6_K 2048 x 262147, T 1..512)
has the same NMSE vs the CPU in its last 3 columns as in the rest (~5e-5), 2 repeats each;
test-backend-ops MUL_MAT_ID 132/132, 3 runs.

Base sha256 changes: zaya1-8b, qwen3.8-27b, qwen3-4b-iq4xs. Qwen3-0.6B and Qwen3-Coder-30B-A3B
record the same bytes as before. Only one kat text changed (zaya1-8b prefix-reuse).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) October 20, 2026 22:41
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 63b7727

@bong-water-water-bong
bong-water-water-bong merged commit 77c2ef0 into main Oct 20, 2026
8 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/hw-gate-private-kernels branch October 20, 2026 22:45
bong-water-water-bong added a commit that referenced this pull request Oct 21, 2026
…nside Lemonade) (#488)

- README: lead with the census (43/43 on the GPU, #476/#480/#481) and Qwen3.8-27B MTP
  36.0/26.8/29.9 tok/s (#487); one hop (#475), strict placement (#482), hw-gate kernels and
  placement (#478/#484), allocation safety (#486), Ubuntu 26.04 PPA lemond (#483), ONNX on the
  NPU via the user's Ryzen AI install (#386); Vulkan/ROCm/CUDA/ZINC/ComfyUI/MLX/Windows cut
  (#306); no release date; NPU in progress with no speed numbers.
- npu.md: a status page with the public parts only (model format, serving, add-on hook, build);
  no runtime method, no NPU speeds.
- releases.md: "Releases", first release when the checklist is done; neutral package checks.
- lemonade.md, serve.md, hrx.md, laya.md, PORTING.md, registry.md: one hop, MTP on HRX (#487),
  census lead, add-on wording without private repository names or NPU speeds.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hw-gate Run the HRX hardware gate on Strix Halo (docs/hw-gate.md) Review effort 3/5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant