Repository navigation
tests: build the HRX tests against the ggml-hrx we ship (engine#422) - #104
Merged
bong-water-water-bong merged 2 commits intoOct 18, 2026
Merged
bong-water-water-bong merged 2 commits into
bong-water-water-bong merged 2 commits into
Conversation
f2099e9 put ggml/src/ggml-hrx back to our backend as of 522dab4 but left AMD's newer tests/hrx-backend-test.cpp, test-hrx-ops.cpp and test-hrx-loom-jit.cpp in place. They target AMD's ggml-hrx (activation publication, WorkloadSpecialization, symmetric-i4), which we do not ship, so none of the three compiled. Errors before, at 8d9ce2c with hrx-system d3c731e1 (clang -ferror-limit=0, the missing include removed so the rest of each file is checked): hrx-backend-test.cpp fatal include dispatch-symmetric-i4.h + 21 errors (workload_specialization/WorkloadSpecialization, CommandPlanMetadata::activation_publication_diagnostics, common_symmetric_i4_lowrow_mul_mat_eligible, one struct initializer) test-hrx-ops.cpp fatal include dispatch-activation-publication.h + 128 errors (CommonActivationPublication*, DispatchActivation*, add_activation_consumer, WorkloadSpecialization, find_generated_resource) test-hrx-loom-jit.cpp 26 errors (WorkloadSpecialization and changed argument counts of the JIT entry points) test-hrx-buffer.cpp and testing_suite.h compile unchanged. The three files are taken whole from 522dab4, the version matching our ggml-hrx. No commit of ours touched them after 522dab4, so nothing is re-applied. After this, hrx-backend-test, test-hrx-ops, test-hrx-loom-jit, test-hrx-buffer and test-backend-ops all build (-j2, 0 errors). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (engine#422) The 522dab4 copies restored in the previous commit predate two changes of ours: - a4963bb lets GET_ROWS of IQ3_S and IQ4_NL rows run on HRX, so the token embedding graph for both types is now schedulable. hrx-backend-test still required it to be refused and aborted at line 3494; it now requires support. - 39588ef points ggml_flash_attention_f32_f16_wmma at a kernel that declares @ggml.flash_attention.value_stride. The dispatch passes value->nb[1] in f16 elements; the test's prefill config did not set it, so the async-flash-prefill compiles failed. It now sets key_value_head_count * head_size, the stride of a contiguous value row. Measured on strixhalo (performance mode), two runs each: test-hrx-loom-jit aborted at line 369 -> exit 0 (28 kernels compiled) hrx-backend-test aborts at line 3731 instead of 3494 (gather-add support expectation, not changed here) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 18, 2026
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 18, 2026
…rds (engine ggml-org#445) Owner approved turning the accepted regressions into guards that cannot be quietly accepted. Adds a KNOWN-FAIL status to the case runner and restores the cases the ruling was about. Runner: a case can carry known_fail("ggml-org#449") or known_fail("ggml-org#450"). It must fail while the regression stands: the runner prints [KNOWN-FAIL ggml-org#449]/[KNOWN-FAIL ggml-org#450] and counts those apart from the suite's own failures, so the exit code is driven only by real FAILs (and by an UNEXPECTED-PASS, which prints [UNEXPECTED-PASS #nnn] and exits non-zero: someone fixed or changed it, so the marker has to be retired on purpose). Restored with their original pre-#104 expectations (the plan each regression net was built to protect), adapted only to compile against the current API - activation_publication_diagnostics() is alternate_values() now, and common_symmetric_i4_lowrow_mul_mat_eligible() lived in the header f2099e9 deleted: ggml-org#449 (publication dropped whole in f2099e9) generic_swiglu_k16_publish, tiled_matmul_alternate_publish, rmsnorm_binary_f16_output, qwen_decode_publication_consumer_qualification, qwen_weighted_reduce_next_rmsnorm_q8_publication - re-added, #104 had deleted them packed_f16_producer_consumer, rmsnorm_binary_k16_output, gdn_rmsnorm_gate_dispatch - the three #104 had rewritten to pass, put back to their original expectations ggml-org#450 (symmetric-i4 consumer route re-enabled in aadd602) symmetric_i4_consumer_qualification - re-added, #104 had deleted it Host suite here: 66 case(s): 44 passed, 13 failed, 9 known-fail, 0 unexpected-pass. The 13 failures are the same rows #105 reports (class A plan-shape, class B kernel-name, and the NEEDS-DEVICE qwen_router_top8_dispatch), so the accepted regressions are visible without being failures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
added a commit
that referenced
this pull request
Oct 18, 2026
* tests: run every HRX case and take the ggml-org#445 ACCEPT rows (engine ggml-org#445) The suite is one fail-fast main(), so one red case hides every case after it - and engine ggml-org#445 keeps 9 REGRESSION rows red on purpose. Replace main() with a table of cases and a runner that reports [PASS]/[FAIL] per case plus totals and exits non-zero if any case failed. REQUIRE throws a RequirementFailure that the runner catches instead of calling std::abort(); each case keeps its own expectations, so a red row is a row and not the end of the run. Then take the ggml-org#445 ACCEPT rows this tree still had stale (#104 had already made some): - gather_add_dispatch: the (2048, 13, 1, false) manual graph, the 96-hidden variant and the partial graph are claimed through the generic path now (get_rows x2 + binary_strided, three dispatches) where they were refused; the canonical shape still fuses to hrx:ggml_gather_add_f32, and the strided-coalesced variant is still refused. - swiglu_q8_output: the q8-output publish kernel went with f2099e9; the shape is the kquant swiglu decode feeding the kquant matmul decode, which reads f32, so there is no producer or alternate to bind. - qwen_flash_attention_dispatch: a QK-layout (MLA) shape routes to ggml_attention_rows_f32_f16 (strided extents, bindings 5, no gate) rather than the flash kernel; the gated transposed F16 variant is claimed now, the F32-value one is still refused. Not re-added: generic_swiglu_k16_publication_edges, whose case #104 dropped. Left as they are: the REGRESSION rows and qwen_router_top8_dispatch (NEEDS-DEVICE). Host suite here: 47 passed, 13 failed, 60 total; device cases skipped (no HRX device). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tests: bring back the ggml-org#445 REGRESSION cases as KNOWN-FAIL guards (engine ggml-org#445) Owner approved turning the accepted regressions into guards that cannot be quietly accepted. Adds a KNOWN-FAIL status to the case runner and restores the cases the ruling was about. Runner: a case can carry known_fail("ggml-org#449") or known_fail("ggml-org#450"). It must fail while the regression stands: the runner prints [KNOWN-FAIL ggml-org#449]/[KNOWN-FAIL ggml-org#450] and counts those apart from the suite's own failures, so the exit code is driven only by real FAILs (and by an UNEXPECTED-PASS, which prints [UNEXPECTED-PASS #nnn] and exits non-zero: someone fixed or changed it, so the marker has to be retired on purpose). Restored with their original pre-#104 expectations (the plan each regression net was built to protect), adapted only to compile against the current API - activation_publication_diagnostics() is alternate_values() now, and common_symmetric_i4_lowrow_mul_mat_eligible() lived in the header f2099e9 deleted: ggml-org#449 (publication dropped whole in f2099e9) generic_swiglu_k16_publish, tiled_matmul_alternate_publish, rmsnorm_binary_f16_output, qwen_decode_publication_consumer_qualification, qwen_weighted_reduce_next_rmsnorm_q8_publication - re-added, #104 had deleted them packed_f16_producer_consumer, rmsnorm_binary_k16_output, gdn_rmsnorm_gate_dispatch - the three #104 had rewritten to pass, put back to their original expectations ggml-org#450 (symmetric-i4 consumer route re-enabled in aadd602) symmetric_i4_consumer_qualification - re-added, #104 had deleted it Host suite here: 66 case(s): 44 passed, 13 failed, 9 known-fail, 0 unexpected-pass. The 13 failures are the same rows #105 reports (class A plan-shape, class B kernel-name, and the NEEDS-DEVICE qwen_router_top8_dispatch), so the accepted regressions are visible without being failures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 18, 2026
…te is not a regression (engine ggml-org#450) Measured on strixhalo (gfx1151), Qwen3-4B-Q5_K_M, route on vs off (GGML_HRX_DISABLE_DISPATCH=common.rmsnorm_binary_symmetric_i4_k32), everything else identical, -dev HRX0 -ngl 99 -t 4: pp512 3033.38 +- 23.60 vs 2991.08 +- 39.41 (route on is faster) pp2048 2647.52 +- 95.31 vs 2613.31 +- 117.61 tg64 65.65 +- 0.36 vs 63.61 +- 0.11 (+3.2 %, far outside the error bars) Numerics, held-out Gutenberg text (Pride and Prejudice + Moby Dick, not the repetitive text the earlier run used): PPL = 16.3451 +- 0.74733 over 20 chunks of -c 512, and the cross-arm llama-perplexity KLD over 5 chunks is 0.000000 +- 0.000000 (maximum 0.000056, max delta-p 0.005 %). Three 64-token greedy completions at temperature 0 are byte-identical between the arms (931 bytes each). f2099e9's "substantial numerical performance regression" is not observable on AMD's current kernels. So symmetric_i4_consumer_qualification states what ships instead of what was hoped: its qualifying graphs (a Q5_K/IQ4_XS weight consuming the normalised value, directly or through a reshape) select loom_libs:ggml_rmsnorm_binary_symmetric_i4_k32, and a graph with no such consumer does not. The q8 publication assertions that came with the pre-#104 case are ggml-org#449's territory (ggml_rmsnorm_binary_q8_1_x4_publish went with f2099e9) and are left to the guards there rather than failing this case for a publication reason. Host suite: 66 cases - 45 passed, 13 failed, 8 known-fail, 0 unexpected-pass (was 44/13/9). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the build half of 1bit-MONSTER/engine#422. Based on
8d9ce2c1a, the engine pin since engine ggml-org#441.Cause. f2099e9 (2026-10-06) put
ggml/src/ggml-hrxback to our backend as of 522dab4, but left AMD's newer tests outside that directory. Those tests are written against AMD's ggml-hrx: activation publication,WorkloadSpecialization, symmetric-i4. At this base, three of them fail to compile:tests/hrx-backend-test.cppdispatch-symmetric-i4.h, plus 21 errorstests/test-hrx-ops.cppdispatch-activation-publication.h, plus 128 errorstests/test-hrx-loom-jit.cppThe raw logs are on strixhalo in
~/wt/loom-422-errs.Fix.
testing_suite.handtest-hrx-buffer.cppcompile as they are and are unchanged.flash_attention_config()passesggml.flash_attention.value_stride, which the prefill flash kernel from 39588ef needs.Results on strixhalo (gfx1151), on this branch with hrx-system
d3c731e1:Not in this PR. hrx-backend-test and test-hrx-ops still abort on old routing expectations:
ggml_mul_mat_f32_f32_decode_wave64for Q4_K at one token, which now routes to ourggml_kquant_mul_mat_decode_f32.Rewriting those expectations to whatever runs now could hide a real routing regression, so I'll track them in a separate engine issue.
🤖 Generated with Claude Code