Repository navigation
Conversation
|
Follow-up validation: the metadata repair was used successfully with the unmodified signed DeepSeek engine 0.1.49 after converting this artifact’s separately unsupported dense Q2/Q3 tensors to BF16 and Engram Q2_K tables to Q6_K through stock GGML reference functions. Routed expert bytes were unchanged. The engine loaded and produced coherent, correct arithmetic ( The unconverted Q2_K artifact still cannot load as published: the dense loader falls through to the generic “not an EXL3 matrix” error, and the Engram reader does not accept Q2_K. This PR fixes metadata compatibility; it does not add those tensor formats or claim general model-quality validation. |
Older converters write the Engram scalars as engram.key_length and engram.pad_id, and omit the sparse-attention geometry. The packer now reads the old names only when the canonical engram.head_dim and engram.pad_token_id are absent, and a new --config-json option fills the omitted geometry from the source model's config.json. A value that disagrees with metadata already in the GGUF stops the pack with a clear message. The weights are never rewritten and a set that already has the canonical names packs exactly as before. Tests: LegacyModelMetadata (2 new, 5 in the file) in tests/terminal/test_pack_gguf_engram.py. Not covered: the author reports the older Q2_K set still needs dense Q2/Q3 and Engram Q2_K tensor support in the engine; this does not add it.
|
Thank you. Fixed in 0.2.7: Accept older DeepSeek GGUF metadata names. Your name is on the commit. Release: https://github.com/Niko1221/Strata/releases/tag/v0.2.7 |
Older DeepSeek-V4.1 GGUF conversions can have supported tensors but fail during packing. The pinned mrfakename Q2_K uses
engram.key_lengthandengram.pad_id; current packing expectshead_dimandpad_token_id, sowrite_engramraisesTypeError: list indices must be integers or slices, not NoneType. Those files also omit the sparse KV/index source lists and candidate geometry entirely.The packer now accepts those two older Engram names. An explicit
--config-jsonoption fills omitted sparse geometry from the exact source model configuration. It checks existing core/sparse geometry for disagreement instead of replacing it. This does not rewrite GGUF metadata or tensors and does not change engine code or inference defaults.Example:
The pinned source config agrees with the official 1017 config: Engram dimension256/pad2; KV source layers2,8,14,20; candidate layer20/top-k2048/block8. This is the source of the missing values, not a guess based on architecture size.
Validation on Linux/Python3.14, base
7dd5cf18e68c639a8773e096c5c79f971ea2a9f5:python3 tests/terminal/test_pack_gguf_engram.py(5 tests). The new legacy-Engram case first reproduced the TypeError on the base implementation. It now writes the expected DSE1 header and identical bytes to current names. Canonical values retain precedence. The source-config case checks restored geometry and rejects a conflict.--helpexposes the option; syntax checks andgit diff --check.