[bong-water-water-bong]

hrx: densify a row-padded activation once instead of falling back to the CPU (engine ggml-org#991)

Stacked on #208 (8ac62a3f7); rebased onto 62c0282fb. The K % 4 predicate is what lets a K=92 contraction be
considered at all, so this lands after it; it also carries the n=512 test-case commit that missed #208's merge (the
merge took 9972afa1b, the n=512 rows were pushed after it).

The gap

The SSM ports hand their small projections a column slice of a wider tensor. BlackMamba's dt_proj contracts 92 of the
264 columns of x_proj, so its activation is a view whose rows are dense but gapped (nb[1] != 4 * 92). The plain dense
matcher refused it on !input->contiguous, which put all 18 dt_proj nodes of the prefill ubatch on the CPU:

GGML_HRX_STRICT=1: "node_31" is MUL_MAT f32 ne=[2944,512,1,1] and is not placed on HRX;
18 of 1179 nodes off HRX (MUL_MAT x 18), 37 splits

The same contraction with a dense activation already placed (test_mul_mat(bf16, f32, m=2944, n=512, k=92) is OK), and the
node's own shape probe reads in(t=0 f32 ne=[92,512,1,1] c=0) — a row-padded view, not a size problem.

What changes

The kernels bind a packed 2-D tile, so such a view cannot be bound directly — but it can be copied once. That copy path already
exists for GGML_OP_CPY (ggml_copy_f32, and ggml_copy_strided_source_f32, which takes the source extents and per-dimension
strides as compile-time config), so:

• append_f32_densify_dispatch (new, exported from the copy dispatch unit) fills a Dispatch that copies a strided F32 source
  into a packed destination, reusing exactly the strided-source/kernel selection make_copy_f32_dispatch already does.
• common_match_mul_mat_any_format takes allow_row_padded_activation: the input may then be a row-padded F32 view
  (nb[0] == sizeof(float)), and the match records strided_activation.
• build_mul_mat_dispatch allocates a packed transient for that node, emits the densify dispatch, and binds the transient as
  the activation. Cost is one copy of K * tokens floats per node per ubatch (188 KiB here) instead of a per-layer CPU
  fallback; q8 packing is ruled out in that case rather than combined with it.
• The plain and unary dense matchers opt in. Nothing else changes: the weight must still be dense, and every other matcher
  keeps requiring a contiguous activation.

Evidence

One gpu-hold ticket (pi-zarch-994-densify-take2: 1334 s queued behind other lanes, 169 s on the token), every item an
assertion that had to hold — a printed line next to a dead run does not count (~/work/zarch/proof/994-densify.sh):

• Library sanity: no undefined densify symbol in libggml-hrx.so, append_f32_densify_dispatch exported. (Take 0 failed
  precisely here — see the reviewer notes.)
• test-backend-ops (-b HRX0 -o MUL_MAT) — the three new row-padded cases are OK against the CPU reference:
  bf16 m=2944 k=92 row_stride=264 at n=512 and n=8, and m=3712 k=232 row_stride=264 at n=512.
  Earlier classes do not regress: m=2944 k=92 and m=3712 k=232 dense at n=8/n=512 still OK; k=90 still
  not supported (the % 4 boundary from hrx: admit element-wise dense MUL_MAT contractions at input_size % 4 (engine #991) #208). Zero Loom compilation failed in the arm.
• BlackMamba-2.8B, GGML_HRX_STRICT=1 exits 0 — the check that cannot be misread, since it aborts on any off-HRX node.
  Placement on both the prefill and the decode graph: 1179 nodes, 0 off HRX (none), 1 graph split (was 18 off MUL_MAT,
  37 splits). It is fully on HRX end to end.
• Numerics unchanged: PPL (wikitext-2, -c 512 --chunks 4) 12.5406 — identical to the pre-change value, i.e. the copy
  transient is a placement mechanism, not a different computation.
• Zamba-7B-v1 unchanged, as expected: 13 of 5048 off (FLASH_ATTN_EXT x 13), 27 splits, GGML_HRX_STRICT=1 still aborts on
  fattn-2 (ne=[464,16,512,1], value_head_size = 464 against the kernels' multiple_of(64), no partial-tile path in the
  prefill op). PPL 20.0704, same as before.
• No speed claim. The bench rows in this ticket (pp512 3653.48 ± 39.35, tg64 109.34 ± 2.65 for BlackMamba;
  194.68 ± 1.52 / 7.19 ± 0.05 for v1) were taken while other lanes were saturating the box and are quoted only as
  "the run happened", not as a result. The claim this PR makes is placement, and placement is asserted.

Residual, stated rather than smoothed over: 1 graph split remains with 0 nodes off HRX, and I have not explained it.

What this does not do

• v1's FLASH_ATTN_EXT residue is untouched. That is the next unit, not this one.
• A genuinely non-dense row (nb[0] != sizeof(float), e.g. a transpose) is still refused: the matmul kernels read a packed
  row, and the densify here is a row-gap copy, not a general transpose.

Notes for the reviewer

This is the fourth round on the same class, and take 0 of this PR is the round worth reading:

• The exported helper was first defined inside the file's namespace { … }. Internal linkage, dead in that translation
  unit, so clang dropped it; the extern declaration in dispatch-copy.h stayed unsatisfied and a shared library links
  happily with an undefined reference until the first call. ninja exited 0, the tree was clean, and the library was
  broken — the op suite died with symbol lookup error, and both model arms exited 127. The harness refused to read those as
  results (no placement line, no PPL → arm fails), which is why the ticket came back red instead of "0 off HRX" with no
  numbers behind it. Fixed by moving the definition below the anonymous-namespace close (6dc15d36c), and the arm now asserts
  the symbol with nm -D before it runs anything.
• The first version of the harness also grepped k=92,row_stride=264 adjacently. The case name carries
  bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=0 between them, so that assertion could never match and would have failed a
  passing run. All six patterns were re-validated offline against lines in the real output format; the assertion was the
  wrong part, not the device.
• The pattern across the four rounds: each round looked finished because something printed. Placement lines next to PPLs are
  not evidence. What is evidence is a GGML_HRX_STRICT=1 exit code, a test-backend-ops verdict against the CPU reference,
  and a symbol that is actually in the library.

[bong-water-water-bong]

VERDICT: READY

Device evidence from my own HRX build at this head (GGML_HRX=ON, amdclang, gpu-hold -l "pi prs 210+211", token granted after 3870 s at die 67 C, held 46 s).

The three new cases pass against the CPU reference, exactly as the body says:

MUL_MAT(type_a=bf16,type_b=f32,m=2944,n=512,k=92,row_stride=264): OK
MUL_MAT(type_a=bf16,type_b=f32,m=2944,n=8,k=92,row_stride=264): OK
MUL_MAT(type_a=bf16,type_b=f32,m=3712,n=512,k=232,row_stride=264): OK

Full test-backend-ops test -b HRX0 -o MUL_MAT: rc 0, 540 OK, 0 FAIL, 0 Loom compilation failed, and the % 4 boundary is preserved — m=512,n=8,k=90 still reports not supported [HRX0].

The placement claim holds, and the residual is my build's configuration, not this PR. GGML_HRX_STRICT=1 on BlackMamba-2.8B (prefill 512) at this head:

placement: 1179 nodes, 90 off HRX (MUL_MAT_ID x 54, SSM_SCAN x 18, GLU x 18), 73 graph splits

MUL_MAT is not in that list at all — the 18 dt_proj nodes the PR is about are on HRX, which is the change working. The 90 that remain are ops whose dispatchers live in the gpu-kernels add-on: my configure has GGML_HRX_HIP_ADDON_DIR:PATH= (empty), while the binary the body measured has GGML_HRX_HIP_ADDON_DIR:PATH=/home/bcloud/work/zarch/gk/addons/hip-hrx, and that tree contains exactly hip_ssm_scan_f32.hip, moe_decode_mmid.hip and glm_moe_decode.hip. So the body's "1179 nodes, 0 off HRX, 1 graph split" is the same measurement with the add-on, and my 90-off is what the same tree looks like without it. I could not reproduce "0 off / 1 split" without a rebuild against that add-on, and I would rather say that than imply it.

One instrument error of mine, since it is the fourth of the same kind today: I counted not supported with ': not supported$', which matches nothing because the line ends not supported [HRX0] , so my summary printed 0 while the k=90 line is right there. The line is what counts, not my count.