Repository navigation
tests : update Metal fusion baseline for beam - #30764
Merged
Merged
Conversation
Regenerate tests/fusion/MTL.csv with test-fusion --record. The newly added beam architecture has no baseline rows, so test-fusion fails with 2 failing rows (MUL+ADD x6, RMS_NORM+MUL x25). All other arch counts are unchanged. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
Add a skill for regenerating a per-device test-fusion baseline. It keeps the workflow from tests/fusion/README.md and the parts that are easy to get wrong: diff the recorded file before applying it, the distinct meanings of a FAIL row, and never recording a partial run. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
github-actions
Bot
added
documentation
ggerganov
marked this pull request as ready for review
October 19, 2026 06:11
Member
|
Interesting, this revealed that |
Member
Author
|
It's fixed in #30795. It was failing before this change too. |
edwardyoon
pushed a commit
to edwardyoon/focus-llama
that referenced
this pull request
Oct 24, 2026
* tests: add beam fusion rows to MTL baseline Regenerate tests/fusion/MTL.csv with test-fusion --record. The newly added beam architecture has no baseline rows, so test-fusion fails with 2 failing rows (MUL+ADD x6, RMS_NORM+MUL x25). All other arch counts are unchanged. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * skills: add update-fusion-table skill Add a skill for regenerating a per-device test-fusion baseline. It keeps the workflow from tests/fusion/README.md and the parts that are easy to get wrong: diff the recorded file before applying it, the distinct meanings of a FAIL row, and never recording a partial run. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
auRose94
pushed a commit
to auRose94/llama.cpp
that referenced
this pull request
Oct 28, 2026
* tests: add beam fusion rows to MTL baseline Regenerate tests/fusion/MTL.csv with test-fusion --record. The newly added beam architecture has no baseline rows, so test-fusion fails with 2 failing rows (MUL+ADD x6, RMS_NORM+MUL x25). All other arch counts are unchanged. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * skills: add update-fusion-table skill Add a skill for regenerating a per-device test-fusion baseline. It keeps the workflow from tests/fusion/README.md and the parts that are easy to get wrong: diff the recorded file before applying it, the distinct meanings of a FAIL row, and never recording a partial run. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
test-fusion --check tests/fusion/MTL.csvcurrently fails with 2 rows: thebeamarchitecture has no rows in the Metal baseline, so both of its fusions are compared againstexpected 0:Regenerate the baseline with
--recordper tests/fusion/README.md. I recorded to a temp file and diffed against the committed CSV first: the only difference is the two newbeamrows, no other architecture's counts moved. After the update the run issummary: 284 ok, 0 failed(exit 0).Also adds a small
update-fusion-tableskill recording the baseline-refresh workflow and the ways it goes wrong (recording a partial run wipes the rows of the architectures that were not run; annmse_fusfailure is a correctness bug and must not be recorded away).Additional information
ref: #30618 (vanished-row detection), #29812 (previous baseline refresh of this kind)
Requirements
pi:llama.cppMacBook, M5 Max, 128GB