Skip to content

[Engine] pack older DeepSeek GGUF metadata with exact source geometry - #3955

Closed
Tsury wants to merge 1 commit into
Niko1221:mainfrom
Tsury:fix/dsv41-legacy-gguf-metadata
Closed

Tsury wants to merge 1 commit into
Niko1221:mainfrom
Tsury:fix/dsv41-legacy-gguf-metadata

Conversation

@Tsury

@Tsury Tsury commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

Older DeepSeek-V4.1 GGUF conversions can have supported tensors but fail during packing. The pinned mrfakename Q2_K uses engram.key_length and engram.pad_id; current packing expects head_dim and pad_token_id, so write_engram raises TypeError: list indices must be integers or slices, not NoneType. Those files also omit the sparse KV/index source lists and candidate geometry entirely.

The packer now accepts those two older Engram names. An explicit --config-json option fills omitted sparse geometry from the exact source model configuration. It checks existing core/sparse geometry for disagreement instead of replacing it. This does not rewrite GGUF metadata or tensors and does not change engine code or inference defaults.

Example:

python tools/dsv41/pack_gguf.py --model /path/to/Q2_K --out /path/to/pack --config-json /path/to/source-model/config.json

The pinned source config agrees with the official 1017 config: Engram dimension256/pad2; KV source layers2,8,14,20; candidate layer20/top-k2048/block8. This is the source of the missing values, not a guess based on architecture size.

Validation on Linux/Python3.14, base 7dd5cf18e68c639a8773e096c5c79f971ea2a9f5:

  • Passed: python3 tests/terminal/test_pack_gguf_engram.py (5 tests). The new legacy-Engram case first reproduced the TypeError on the base implementation. It now writes the expected DSE1 header and identical bytes to current names. Canonical values retain precedence. The source-config case checks restored geometry and rejects a conflict.
  • Passed: full pack CLI with all ten real Q2_K tensor directories/metadata retained from bounded HTTP Range reads. Temporary sparse files had the published logical sizes but contained no weight data. Packing produced47,286 tensor entries, correct source/candidate geometry, and two Engram layers with24 hash columns. The resulting Engram hash-table binary was byte-identical to the current-format Orca1017 header's table.
  • Passed: target Python3.15.1 packer --help exposes the option; syntax checks and git diff --check.
  • Not run: live model loading or inference for this repair. Weights are still downloading. Header/packing checks do not establish numerical correctness or performance. No backend code changed, so CUDA/HIP/SYCL builds are outside this Python metadata fix.

@Tsury

Tsury commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up validation: the metadata repair was used successfully with the unmodified signed DeepSeek engine 0.1.49 after converting this artifact’s separately unsupported dense Q2/Q3 tensors to BF16 and Engram Q2_K tables to Q6_K through stock GGML reference functions. Routed expert bytes were unchanged. The engine loaded and produced coherent, correct arithmetic (17 × 19 = 323).

The unconverted Q2_K artifact still cannot load as published: the dense loader falls through to the generic “not an EXL3 matrix” error, and the Engram reader does not accept Q2_K. This PR fixes metadata compatibility; it does not add those tensor formats or claim general model-quality validation.

Niko1221 pushed a commit that referenced this pull request Oct 27, 2026
Older converters write the Engram scalars as engram.key_length and
engram.pad_id, and omit the sparse-attention geometry. The packer now
reads the old names only when the canonical engram.head_dim and
engram.pad_token_id are absent, and a new --config-json option fills the
omitted geometry from the source model's config.json. A value that
disagrees with metadata already in the GGUF stops the pack with a clear
message. The weights are never rewritten and a set that already has the
canonical names packs exactly as before.

Tests: LegacyModelMetadata (2 new, 5 in the file) in
tests/terminal/test_pack_gguf_engram.py.

Not covered: the author reports the older Q2_K set still needs dense
Q2/Q3 and Engram Q2_K tensor support in the engine; this does not add it.
Niko1221 added a commit that referenced this pull request Oct 27, 2026
@Niko1221

Copy link
Copy Markdown
Owner

Thank you. Fixed in 0.2.7: Accept older DeepSeek GGUF metadata names. Your name is on the commit.

Release: https://github.com/Niko1221/Strata/releases/tag/v0.2.7

@Niko1221 Niko1221 closed this Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants