Repository navigation
Conversation
Implements support for NVIDIA's NemotronHPuzzleForCausalLM architecture, a hybrid mamba2/attention/MoE model with per-layer heterogeneous MoE configuration and a 2-block MTP draft head (attention + MoE sub-blocks). Key changes: - Add per-layer n_ff_exp_arr[] and n_expert_used_arr[] to llama_hparams; rename scalar fields to n_ff_exp_impl/n_expert_used_impl with accessors n_ff_exp(il) and n_expert_used(il) for per-layer lookup with scalar fallback - Add nextn_predict_per_head for multi-block MTP steps (Puzzle uses 2) - Add LLM_TYPE_75B_A9B for Puzzle 75B; discriminate from 120B Super via per-layer top-k heterogeneity check (both have 88 trunk layers) - Add llama_model_n_nextn_heads() API so speculative decoding counts invokable draft heads rather than raw nextn block count - Add graph_mtp for NEMOTRON_H_MOE: runs the 2 MTP sub-blocks serially (attention blk.n_trunk + MoE blk.n_layer-1) as a single draft step; fix MTP context to use plain KV cache (not hybrid) for Puzzle - Convert: add NemotronHPuzzleModel with block_configs per-layer parsing, FP8-dequant support, mtp.safetensors mapping onto blk.88/89, and fix _experts AttributeError in NemotronHModel base class - gguf_writer: make add_expert_feed_forward_length and add_expert_used_count accept arrays for per-layer Puzzle encoding - Add MTP tensor allowlist (NEXTN_EH_PROJ/ENORM/HNORM/SHARED_HEAD_NORM) to NEMOTRON_H_MOE in gguf constants Co-authored-by: rcmorano <55876+rcmorano@users.noreply.github.com>
- Remove redundant `const auto & hp = hparams` alias in load_arch_tensors - Fix lambda capture in mtp_on_hybrid_nemotron_h filter: capture n_trunk by value (not this), filter to il == n_trunk (attention sub-block only) - Clarify graph_mtp GGML_ASSERT messages to say nemotron_h (not NEMOTRON_H_MOE) - Add comment explaining num_heads usage for d_inner in NemotronHPuzzleModel Co-authored-by: rcmorano <55876+rcmorano@users.noreply.github.com>
Co-authored-by: rcmorano <55876+rcmorano@users.noreply.github.com>
Co-authored-by: rcmorano <55876+rcmorano@users.noreply.github.com>
Co-authored-by: rcmorano <55876+rcmorano@users.noreply.github.com>
Copilot
AI
changed the title
feat: add NemotronHPuzzleForCausalLM (Nemotron-3-Puzzle-75B-A9B) support
convert_hf_to_gguf: harden NemotronHPuzzle MTP tensor remapping
Jul 9, 2026
rcmorano
pushed a commit
that referenced
this pull request
Jul 18, 2026
* docs: add UltraQuality rankleave32 recipe * docs: map released models to qualified recipes --------- Co-authored-by: ciru-ai <290232805+ciru-ai@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Nemotron Labs 3 Puzzle conversion was failing in
convert_hf_to_gguf.pyafter weight indexing because the Puzzle path assumed onlymtp.layers.{0,1}.*tensors and did not robustly handle top-levelmtp.*tensors.This change makes MTP remapping resilient to actual Puzzle checkpoint naming and to variable MTP layer counts from config.
MTP name handling
mtp.fc→model.layers.{bid}.eh_projmtp.pre_fc_norm_embedding→model.layers.{bid}.enormmtp.pre_fc_norm_hidden→model.layers.{bid}.hnormmtp.norm→model.layers.{bid}.shared_head.normDynamic MTP block indexing
bid in (0, 1)assumption with bounds fromlen(self.mtp_block_configs).eh_proj/enorm/hnorm/final_layernormand routes remainingmtp.layers.{bid}.*tensors through the existing backbone mapping path.Failure semantics
Additional information
The warning about
trust_remote_code=Trueduring config loading is expected for this model family and not the conversion failure root cause; the crash path was in Puzzle MTP tensor remapping during export.Requirements
AGENTS.mdandCONTRIBUTING.mdbefore submission.