Repository navigation
tools/npu-arena: the kernel arena, its DMA-ceiling probe, and a test for the layout contract - #668
Merged
Merged
Conversation
added 3 commits
October 30, 2026 15:06
…for the layout contract The arena that produced the last two PRs was a private directory. It belongs in the repo: it is the harness every future kernel change is scored with, and the numbers it produces only mean something if anyone can rebuild it. tools/npu-arena/ is that harness, self-contained and box-agnostic (paths resolve from the environment, or from a git-ignored TOOLCHAIN.lock; TOOLCHAIN.lock.example documents the pins): * `harness/gen.py` freezes a per-phase harness: A (activations) int8, B int8 in the engine's packed layout, the expected int32 = the exact integer matmul, per-column scales for provenance, and SHA256SUMS. Three seeds, ternary with the container's real per-32-row-group magnitude structure, so a lossless 2-bit packing stays a legitimate entry. * `runner/runner.cpp` is the on-device scorer: one full ELF, the frozen harness, a byte-exact int32 check (two consecutive clean reads, because the runtime's readback is occasionally short -- engine issue #666), median of N launches. It can also score a submission's own weight layout (`--w`) and skip the check for a probe (`--no-check`). * `make_task.sh` + `oracle.sh` are the competition: a task bundle per phase, and an oracle that rebuilds a submission from source with the pinned toolchain, refuses a shared device, replays the leader, and prints the leaderboard row on ACCEPT. * `make_probe.sh` + `probe/mm_dmabw.cc` + `ceiling.sh` are phase 1 of the 2026-10-30 14:58 plan: the phase's own objectfifos, taps, blocked layout, tile geometry and column-group loop with the matrix unit replaced by a pass-through microkernel, so a launch measures the data path alone. Two variants per phase (8-core M=16 / 32-core M=64 with 4 core rows) and a CEILING.md report, so every entry can be judged as a percentage of what this box can actually stream. * `LEADERBOARD.md` and `HARVEST.md` carry the records: the blocked-B layout is entry #2 on every phase (2.4-2.6x, since merged as #667), and what the numbers still owe (the 32-core N-partition, the 2-bit ternary stream). Also moves the blocked layout's index math out of the runtime header into npu/forward/include/q4nx_weight_layout.h, and tests it: tests/npu_blocked_layout_test.cpp proves the mapping is a bijection over six geometries (including the 27B's 5120x17408 and 17408x5120), that each 64x128 tile occupies one contiguous 8192-byte window with the tiles k-fastest, that element (i,j) inside a tile lands where the int8 microkernel reads it (b_in[((kb*16+nb)*8+r)*8+c], an independent restatement of its "B LAYOUT silicon-verified" note), that a geometry which is not a whole number of tiles is refused, and that the row-major layout every earlier container uses has not moved. Pure index math, no device and no runtime, so it runs in CI. No kernel binaries: the ELFs stay private and are emitted from source with the pinned toolchain (the same rule as the rest of the NPU artifacts).
…tree Owner decision on this PR: the probe microkernel and the perf records are kernel material, so they live with the rest of the ELF tree (npu-kernels) rather than in the public engine. Moved to npu-kernels (PR #19, branch arena/leaderboard-and-probe): * `npu/probe/mm_dmabw.cc` -- the pass-through AIE microkernel the ceiling probe links. * `docs/arena/LEADERBOARD.md`, `docs/arena/HARVEST.md` -- the per-phase records and the techniques/findings behind them. What stays here is host tooling and documentation only: * the arena's harness generator, runner, oracle, task/probe builders and ceiling driver, which reference the private tree by path (`NPU_KERNELS`/`PROBE_SRC`/`GENERATORS`) and carry no kernel source; * the README, now describing the method instead of our measured tables; * `q4nx_weight_layout.h` (the layout contract, mechanism only -- the numbers moved to the records) and its CI test. Also applies tools/copyright.py --fix to the new files (the build's copyright check was failing on them).
CI builds an explicit target list and then runs `ctest`, so a registered test whose binary is not on that list reports "Not Run" and never executes -- which is exactly what happened to the layout test (test 41, 46 tests, 1 failed, exit 8). Editing that workflow list needs a token scope this lane does not have, and a test of pure index math does not need a build target at all: `tests/npu_blocked_layout_test.sh` compiles tests/npu_blocked_layout_test.cpp with the build's own compiler at test time and runs it. add_test() therefore passes the compiler, the include dir and the source. It uses nothing else: no runtime, no device, no NPU toolchain, so it passes on the hosted runner. Verified with the runner's own configure (cmake -G Ninja, gcc-16/g++-16, no NPU options): `ctest -R npu_blocked_layout` -> Passed 0.21s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on main
d51d151(both NPU PRs in: #661 m16/head/step-skip, #667 blocked-B layout). Adds the arenatooling that produced those two, phase 1 of the 14:58 addendum's plan, and a CI test for the layout contract
that #667 introduced.
What is in this PR, after the owner's split
Host tooling and documentation only — no kernel source, no kernel-perf tables:
harness/gen.pyrunner/runner.cpp--wscores a submission's own weight layout,--no-checkfor a probemake_task.sh,oracle.shmake_probe.sh,ceiling.shCEILING.mdREADME.md,TOOLCHAIN.lock.example,lib/env.shNPU_KERNELS/PROBE_SRC/GENERATORS)q4nx_weight_layout.h,tests/npu_blocked_layout_test.cpp,CMakeLists.txtMoved out of this PR (kernel material, npu-kernels PR #19)
npu/probe/mm_dmabw.cc— the pass-through AIE microkernel the ceiling probe links (it exports the real microkernel's symbols andDIM_M/K/Ncontract, so it is kernel source by the same rule as the rest of the ELF tree).docs/arena/LEADERBOARD.md,docs/arena/HARVEST.md— the per-phase records and the techniques/findings.The scripts here reference that tree by path (
PROBE_SRC,NPU_KERNELS,GENERATORS); nothing in this PR embeds kernel source or a GB/s table.Verification
tools/copyright.py --checkclean.npu_blocked_layout_test: OK (bijection over six geometries incl. the 27B's, tile windows, microkernel element order, refusals, row-major unchanged) — registered as thenpu_blocked_layoutCTest.make onebitgreen with the pinned toolchain on this head.ceiling.shrun is queued behind the box handover and its output goes to the private tree.