Repository navigation
Conversation
Contributor
CI performance trends
Krackan - OperatorsOperators added: Benchmarks that moved by at least 5% and by more than 2x the spread the runs measured:
Phoenix - OperatorsOperators added: Benchmarks that moved by at least 5% and by more than 2x the spread the runs measured:
|
a0162c7 moved eighteen applications out of #222 to their own pull requests, each with its checkpoints, goldens and the library code only it exercises. Granite goes first: this ports 46a5d87's model, profiles, README and device-free tests back in, unchanged, against today's iron.lm. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Granite's GraniteConfig covers test_config.py's key-mapping check alongside LlamaConfig's, and long_context's APPLICATIONS holds it to the same one-compile, any-context guarantee as Llama and Gemma 4; its deep_kl figures are 46a5d87's own, measured at 4095 positions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
8c25885 gave every model test fixture a model.release() after the module's (and TestEachStep's) tests, so the next model's CREATE_BO doesn't meet a full heap behind a reference cycle still awaiting GC. Granite's fixtures, restored from before that commit, didn't have it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
754e3ae cut iron/lm's and the three #222 applications' docstrings to what a caller relies on, the rationale left for the commits that made it. Granite's, restored from before that commit, kept the longer form; this brings its module and Granite class docstrings to the same register. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
74f7004's table (cut from b1e224c6 to #222's models) left out Granite's row when it moved out in a0162c7. Restored at its measured figure (86.5 s on this host today, against b1e224c6's 86.5 s): its deepest prompt chunk, at its native 131072 context, as one command. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
bd6c053, d23627e and fdaaab1 widened long_context's five context sizes to the per-step form (an xclbin a step, the host starting each on the token the last one drew) on npu1 and npu2; Granite's entry, restored from before that work, was still full-ELF only. Granite has an npu1 profile (empty, every operator's device defaults), so it joins at each_step=("npu1", "npu2") like Llama's and Gemma 4's. Both forms measured on this host's npu2, 4096 to the whole 131072 context: decode deep in the cache KL 0.0010 to 0.0022, its deepest prompt chunk 1.7e-8 to 1.4e-4, top-1 agreeing throughout. npu1 is still unmeasured on this host. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Granite 4.0-H 1B joins Granite 4.2 3B as a follow-on to #222 (a0162c7, 0f0a540): the dense 1B this ports, its README, profile and device-free tests, unchanged, against today's iron.lm. Its library dependencies (selective_scan, CausalConv1d, Checkpoints) and the two later fixes follow in their own commits. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Granite's tokenizer adds no BOS, so since prompts begin as its post-processor begins them, the accuracy checks see 292 and 3261 tokens where they saw 293 and 3262, and the golden rows, KL figures, README and bounds were for the old prompts. The golden is re-recorded device-free (transformers in float32) and the figures re-measured on xcoradaie220. Measured on Strix Halo (npu2), one compile at the native 131072 positions, projections at HIGH as before: - KL over 40 steps after 292 tokens: mean 0.0024, p90 0.0048, max 0.0063 (2 top-1 swaps); two chunks and two decode steps: mean 0.0027, max 0.0049. The chat turn is bit for bit the run from scratch; determinism 0 differing runs. - TTFT 2.9 s after 13 characters, 3.6 s after 292 tokens, 8.0 s after 2048; 6.7 tokens/s. MAX_KL was about twice the old figures; it is again about twice the new ones (mean 0.006, p90 0.012, max 0.015). The README's Precision table is left to the precision pass, which has these numbers. Net: tests +4/-3, docs +9/-8. Checked: pytest tests/applications/granite_hybrid/test.py on the NPU (8 passed, at the old bounds; the new bounds hold the same deterministic figures). collect-only, black, reuse. Co-Authored-By: Claude <noreply@anthropic.com>
granite_hybrid was listed as unheld because Experts streams every expert's weights for each row. That is true of the MoE sizes, but the dense 1B the package ships runs no Experts. The 1B now joins APPLICATIONS, and long_context holds it to its native 131072 positions like the other decoders: nothing grows with the context but a row per position, and the deepest decode step and prompt chunk match the graph's reference. The MoE sizes have no checkpoint or test here. Measured on Strix Halo (npu2), full ELF at 131072 positions: - decode at 131071: KL 0.0035 against the graph's reference, same top-1. - the deepest prompt chunk: KL 4.1e-5, 14.8 s. - TTFT 32.8 s at 8192 tokens, 732 s at the whole 131072. Net: tests +7/-2. Checked: pytest tests/applications/long_context/test.py -k granite_4.0_h_1b: test_nothing_grows_with_the_context, decode and prompt, passes device-free; the 4 extensive device tests pass. test_every_decoder_application_is_held_to_its_context fails before and after this change, because qwen3_embedding is a CausalLM missing from both lists. collect-only, black, reuse. Co-Authored-By: Claude <noreply@anthropic.com>
0f0a540 left SelectiveScan, its shared state_slices, iron.lm.Checkpoints and (a0162c7) CausalConv1d with the applications that exercised them, to come back with whichever follow-on pull request needed them again. Granite 4.0-H 1B's Mamba2 mixer needs all four: the scan over its state, the short causal convolution ahead of it (F.causal_conv1d, ConvState), and the per-chunk checkpointed state both carry between calls. iron.lm.testing's _fill_caches, used by long_context's deep checks, regains the Checkpoints branch it had before 0f0a540 trimmed it to KVCache alone, so a deep check sees the chunk-boundary state a prompt left, not an empty one. selective_scan.py drops one case, "granite_small_prefill": the Case it marked (lower=True, run=False) relied on a run field Case removed, and the case was for a different checkpoint (Granite 4.0-H Small) out of scope here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The package 46a5d87 shipped built GraniteMoE for any config with num_local_experts: Router, then Experts over the checkpoint's stacked expert weights. a0162c7 and 0f0a540 moved Experts and TopKRouter out with Qwen3-MoE, untested at any shape that runs at a usable rate (an unrelated follow-up); restoring them here only to exercise an MoE size this package stages no checkpoint or test for would be restoring code nothing checks. The shipped 1B is dense: GraniteHybridLayer no longer builds a GraniteMoE, and GraniteHybridConfig refuses a checkpoint with experts outright, rather than silently ignoring num_local_experts. The device-free reference test held an MoE toy alongside the dense one (goldens/toy.npz), checking the graph's reference against Hugging Face's GraniteMoeHybrid at a toy MoE shape. It depends on the same GraniteMoE this drops, so it goes with it, replaced by a test that the config refuses experts; the dense toy's own class and golden are what the shipped 1B is checked against. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
8c25885 gave every model test fixture a model.release() after the module's tests, so the next model's CREATE_BO doesn't meet a full heap behind a reference cycle still awaiting GC. Granite 4.0-H 1B's fixture, restored from before that commit, didn't have it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Its row names the real checkpoint (granite-4.0-h-1b) now staged, in place of the "reference only"/"none yet" row from before it existed. Its own README drops the mixture-of-experts section, which measured Experts at an MoE size this package no longer builds (27870ec0), and its intro and reference/ mentions follow the dense-only package. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
generate reads a device loop's draws once the last is drawn: the loop always draws max_tokens, and generate stops at the first eos among them. Its rate counted only the tokens through eos, over the time the loop took to draw all of them. Granite 4.0-H 1B draws eos 8th after the benched 13-character prompt, so its TPS read 1.21 where the device draws 6.7 tokens a second. The nightly trend check would have compared that figure. A Model now says whether generate's draws come at once (draws_at_once: a CausalLM's device loop, a Multimodal's text model's; False otherwise, OnHost too). Where they do, generate counts every token drawn in the time it measures, and keeps returning only those through eos. stream, which reads each token as it is drawn, and draws stepped from the host still stop at eos, so no step runs that otherwise wouldn't. Net: library +31/-5, tests +21. Checked: a new device test, test_granite_4_0_h_1b_rate_counts_the_draws_past_eos, compares generate's rate with stream's over the same early-eos draw: 6.68 vs 6.67 tokens/s (generate read 1.21 before the change, measured with a per-token probe and in the module run). The benched 13-character case reads 6.69 TPS. tests/applications/granite_hybrid/test.py: all 8 other tests pass. Device-free pytest -m "not supported_devices and not extensive" iron/tests/common/test_llama_reference.py iron/tests/common/test_multimodal.py iron/tests/lm iron/tests/infrastructure/test_lm_host.py: 155 passed. collect-only, black, reuse. Co-Authored-By: Claude <noreply@anthropic.com>
Measured on npu2 (xcoradaie220), full ELF, five context sizes 4096 to the whole 131072: decode deep in the cache KL 0.0007 to 0.0069, its deepest prompt chunk 4.1e-5 to 0.0052, top-1 agreeing throughout; npu1 still unmeasured. The deepest prompt chunk at the native context, one command: 14.8 s, matching cbf21694's recorded figure, added to the driver-timeout table between Llama 3.2 1B and EmbeddingGemma 2. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Erika's decision: the dense 1B must run on npu1 (Phoenix), not npu2 alone. 7c34b69 brought Granite 4.2 3B to npu1 the same way: an empty profile (every operator's device defaults), a TestEachStep class (npu1, npu2) holding the per-step generation cases plus a shallower accuracy and determinism check, and long_context's each_step=("npu1", "npu2") so its five-size sweep covers the per-step form too. CI's Phoenix runner covers the real npu1 run; this host is npu2 only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…test Co-authored-by: hunhoffe <54562339+hunhoffe@users.noreply.github.com>
Co-authored-by: hunhoffe <54562339+hunhoffe@users.noreply.github.com>
Co-authored-by: hunhoffe <54562339+hunhoffe@users.noreply.github.com>
a0162c7 kept #222 to the library, Llama, EmbeddingGemma 2 and Gemma 4, moving eighteen applications to follow-up pull requests. This is the Qwen3 family's: qwen3_embedding and qwen3_reranker build directly on qwen3's Qwen3 class, and qwen3_moe on its Qwen3Layer, so the four travel together. - Qwen3 1.7B/0.6B, Qwen3-Embedding 0.6B and Qwen3-Reranker 0.6B restored unchanged from before a0162c7, with their tests and toy goldens. - Qwen3-MoE (30B-A3B, reference only) needs two operators a later cleanup commit (0f0a540) also moved out: restored iron/operators/router.py (TopKRouter) and experts.py (Experts), re-exported from iron/operators/__init__.py. - Restored TopKRouter's two rejected-shape tests in test_rejected_shapes.py. Recreated test_core_streams.py scoped to Experts alone; 0f0a540's version also covered SelectiveScan and GatedDeltaRule, which belong to lfm2/granite_hybrid/qwen3_5's own follow-up. - README's model table gets its four rows back. Checked: black; device-free tests/applications/{qwen3,qwen3_embedding, qwen3_reranker,qwen3_moe}/reference and test_core_streams.py (20 passed); iron/tests/operators/test_rejected_shapes.py collects clean (134 tests). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Restores the decoders and long_context coverage Qwen3, Qwen3-Embedding, Qwen3-Reranker and Qwen3-MoE already have their own device tests for: every other model's checkpoint is held to its whole context, and every full-elf decoder but Gemma 4 is run in the decoders suite's accuracy and determinism checks. - Qwen3-MoE's 30B joins decoders (nightly, its toy test already runs per PR) and long_context, staged=False as its checkpoint isn't on every runner. Its p90 bound is 0.1 (bf16 transformers itself reaches 0.065 on its steps, near-tie experts); the figure is carried from before #222's numeric fixes and not yet re-measured here, so it's not asserted as current. test_determinism skips it: test_host_steps already holds a host step's calls exact, and a full redo is 15 minutes at this scale. - long_context's loaded fixture checked a form by device_loop, which a full ELF with a host step (Qwen3-MoE's paged experts) doesn't have; it now checks the decode step's packaging instead, what the form actually is. - Qwen3 0.6B/1.7B, Qwen3-Embedding and Qwen3-Reranker join long_context. Qwen3-Embedding's row is its final hidden state, held by its cosine to the reference, not KL (check_deep_decode/check_deep_prompt gain cosine=); Application gains deep_cosine and staged. Measured on npu2 (Strix Halo), native context (32768), teacher-forced: Qwen3 0.6B decode KL 0.0020, prompt chunk 0.0021 (39.9s); Qwen3 1.7B (full ELF and each-step alike) decode KL 0.0385, prompt chunk 0.0565 (40.2-40.9s); Qwen3-Embedding decode cosine 0.98611, prompt chunk cosine 0.99859 (31.8s), against its 0.97 bound (twice the decode's distance from 1); Qwen3-Reranker decode KL 0.0212, prompt chunk KL 0.0038 (39.9s), well under DEEP_KL. All ten cases pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
hunhoffe
force-pushed
the
qwen3-family
branch
from
October 29, 2026 18:43
8ae0479 to
51e4833
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #249. a0162c7 kept #222 to the library, Llama, EmbeddingGemma 2 and Gemma 4, moving eighteen applications to follow-up pull requests. This is the Qwen3 family's:
qwen3_embeddingandqwen3_rerankerbuild directly onqwen3'sQwen3class, andqwen3_moeon itsQwen3Layer, so the four travel together.iron/operators/router.py(TopKRouter) andexperts.py(Experts), re-exported fromiron/operators/__init__.py.TopKRouter's two rejected-shape tests intest_rejected_shapes.py. Recreatedtest_core_streams.pyscoped toExpertsalone; 0f0a540's version also coveredSelectiveScanandGatedDeltaRule, which belong to lfm2/qwen3_5's own follow-up.PR Merge Checklist
develcommit and pointing todevel.