Repository navigation
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
SmolLM2 and SmolLM3 tests omit explicit model release, allowing reference cycles to retain multi-GB device buffers and overlap subsequent model allocations.
3 open issues
What changed in this PR
Restores SmolLM2 and SmolLM3 on the shared iron.lm decoder infrastructure, with device profiles, reference checks, and application tests.
Changes:
- Adds Llama-based SmolLM2 and SmolLM3 with selective RoPE disabling and high-precision projections.
- Adds model-specific tests and extends shared decoder and long-context coverage.
- Documents both models, their checkpoint requirements, and device profiles.
| File | Description |
|---|---|
tests/applications/smollm3/test.py |
Adds per-step generation, accuracy, and determinism checks. |
tests/applications/smollm3/reference/test.py |
Adds toy reference and NoPE configuration validation tests. |
tests/applications/smollm2/test.py |
Adds generation, accuracy, sampling, chat, and per-step checks. |
tests/applications/smollm2/reference/test.py |
Adds toy reference checks using the shared decoder suite. |
tests/applications/long_context/test.py |
Registers both models for long-context checks. |
tests/applications/decoders/test.py |
Registers SmolLM3 for shared decoder tests. |
README.md |
Adds both models to the application catalog. |
iron/applications/smollm3/README.md |
Documents architecture, profiles, tests, and checkpoint files. |
iron/applications/smollm3/profiles/npu2.json |
Sets four attention pipelines for decode and prefill. |
iron/applications/smollm3/profiles/npu1.json |
Adds an empty profile using operator defaults. |
iron/applications/smollm3/model.py |
Defines SmolLM3 configuration and selective RoPE disabling. |
iron/applications/smollm3/__init__.py |
Exposes SmolLM3 through lazy exports. |
iron/applications/smollm2/README.md |
Documents architecture, context costs, and checkpoint files. |
iron/applications/smollm2/profiles/npu2.json |
Adds an empty profile using operator defaults. |
iron/applications/smollm2/profiles/npu1.json |
Adds an empty profile using operator defaults. |
iron/applications/smollm2/model.py |
Specializes Llama with SmolLM2-specific profiles. |
iron/applications/smollm2/__init__.py |
Exposes SmolLM2 through lazy exports. |
Review effort: Balanced
| @pytest.fixture(scope="module") | ||
| def model(load): | ||
| """The model, compiled and loaded once.""" | ||
| yield load() |
Comment on lines
+81
to
+82
| if aie_utils.DefaultNPURuntime is not None: | ||
| aie_utils.DefaultNPURuntime.cleanup() |
Comment on lines
+43
to
+44
| if aie_utils.DefaultNPURuntime is not None: | ||
| aie_utils.DefaultNPURuntime.cleanup() |
a0162c7 kept #222 to the library, Llama, EmbeddingGemma 2 and Gemma 4, moving eighteen applications to follow-up pull requests. This is the SmolLM family's: SmolLM2 is Llama's architecture at its own shape; SmolLM3 is Llama's layer with RoPE skipped on every fourth (NoPE). - SmolLM2 1.7B Instruct (iron/applications/smollm2) restored unchanged, with its test.py (TestEachStep included) and toy golden. - SmolLM3 3B (iron/applications/smollm3) restored, and brought to the same npu1-compile standing granite_hybrid got in #249: an empty npu1.json profile, and a new test.py holding only its per-step TestEachStep (its full-ELF generation, accuracy and determinism already run through the decoders and long_context suites, beside Gemma 3, Gemma 4, Qwen3.5 and Qwen3-MoE). Its config gains precision=Precision.HIGH, as 3B's other dense decoder (Llama 3.2 3B) and Granite 4.2 3B are. - Wired SmolLM3 into tests/applications/decoders and both SmolLM2 and SmolLM3 into tests/applications/long_context, the way Qwen3's family was in 8ae0479. - README's model table gets both rows. Checked: black, reuse lint clean; device-free reference suites (smollm2 6, smollm3 7) and long_context's test_every_decoder_application_is_held_to_its_context pass. On npu2 (Strix Halo): smollm2's shortest call and smollm3's shortest call (in decoders) pass; smollm3's per-step form (TestEachStep) passes generation, determinism (0 differing runs), and accuracy (KL mean 0.0045, p90 0.012, max 0.019 over 20 steps, 2 top-1 swaps), its missing each-step golden row recorded device-free for this. SmolLM3 compiles for npu1 with no device bound; its real npu1 run is CI's Phoenix runner. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
hunhoffe
force-pushed
the
smollm-family
branch
from
October 29, 2026 18:43
e5da0f4 to
18e5811
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #253. a0162c7 kept #222 to the library, Llama, EmbeddingGemma 2 and Gemma 4, moving eighteen applications to follow-up pull requests. This is the SmolLM family's: SmolLM2 is Llama's architecture at its own shape; SmolLM3 is Llama's layer with RoPE skipped on every fourth layer (NoPE).
iron/applications/smollm2/) restored unchanged, with its owntest.py(generation, accuracy, determinism, chat turn, and aTestEachStepclass for npu1) and toy golden.iron/applications/smollm3/) restored, and brought to the same npu1-compile standing Granite 4.0-H 1B got in Granite 4.2 3B and Granite 4.0-H 1B #249: an emptynpu1.jsonprofile, and a newtest.pyholding only its per-stepTestEachStepclass. Its full-ELF generation, accuracy and determinism already run throughtests/applications/decodersandlong_context, beside Gemma 3, Gemma 4, Qwen3.5 and Qwen3-MoE. Its config gainsprecision=Precision.HIGH, as 3B's other dense decoder (Llama 3.2 3B) and Granite 4.2 3B are.tests/applications/decodersand both SmolLM2 and SmolLM3 intotests/applications/long_context, the way Qwen3's family was in Add Qwen3, Qwen3-Embedding, Qwen3-Reranker and Qwen3-MoE on iron.lm #251.Measured (Strix Halo, npu2)
SmolLM3's per-step form (
TestEachStep): generation passes; determinism passes (0 differing runs over 3 draws); accuracy KL(fp32 CPU || NPU) over 20 steps mean 0.0045, p90 0.012, max 0.019, 2 top-1 swaps, well under the 0.02/0.04/0.08 bound. Its missing each-step golden row (20 steps, a 256-character prompt) is recorded, device-free.SmolLM2 and SmolLM3's shortest call (13-token prompt, 1 token) both pass on npu2.
npu1
SmolLM3 compiles for npu1 with no device bound (
TestEachStep, device-free). Its real npu1 run is CI's Phoenix runner, as Granite 4.0-H 1B's and both models'long_contextdeep checks are.CI
Checked: black, reuse lint clean. Device-free: reference suites (smollm2 6, smollm3 7) and
long_context'stest_every_decoder_application_is_held_to_its_contextpass.Each runner needs
smollm2-1.7b-instruct/andsmollm3-3b/under/srv; staging zip not yet built for this PR.