[yangzhuxinyzx]

Purpose

Load standalone GGUF configuration, Qwen BPE tokenizer and embedded chat templates without constructing a Transformers model. Select a Qwen3.5 dense adapter to map checkpoint tensors directly to vLLM parameters. Explicit HF configuration and tokenizer remain authoritative.

Restore A_log in FP32, conv dimensions, norm offsets and GDN head tiling. Preserve mixed projection ordering and dtypes, validate TP boundaries, reconstruct unaligned dense rows before TP and replicate KV heads when needed. Layout transforms compose with the packaged native fallback policy and report preparation reasons through the SM70 framework. Plugin attribution and license headers are retained.

Test Plan

Check metadata, shard discovery, tokenizer parity, tensor mapping and TP/mixed-storage contracts. Compare full-vocabulary first logits and output quality against pinned llama.cpp, then verify the ordinary installed wheel and embedded chat templates.

Test Result

• 19 metadata/tokenizer/shard tests and 13 adapter tests pass from source and a fresh wheel installation. Six tokenizer samples exactly match HF token IDs and decoding.
• Installed Qwen3.5-0.8B Q4_K_M inference completes with V100 32GB x4, TP4, CUDA 12.8, Torch 2.10.0+cu128, FP16, maxlen 2048, maxbatch 256, eager and no MTP. English/arithmetic each match 64/64 llama.cpp greedy tokens. First-logit top-1 matches for all three prompts; RMSE is 0.128859/0.148466/0.165342 and relative L2 is 0.030586/0.049015/0.063375.
• Chinese matches 23 tokens in the installed run before diverging. Earlier controlled comparisons localize the legacy batched divergence to the linear numerical path: a single request and an FP16-dequantized batch match 20/20 CPU tokens. Finer rounding/activation/accumulation separation and broader quality comparisons remain pending.
• Embedded chat-template examples produce Paris, 4 and 你好 with thinking disabled.
• All 210 installed dependencies pass compatibility checking. Normal packaged and installed operator hashes match; no private extension override is required.

docs/design/gguf_v100_control.md records reproducible settings and remaining quality work. This change does not claim TurboMind model throughput; original-block TurboMind operators are developed separately. Flash-Next remains a separate adapter with TP4.

The adapter also maps Qwen3.8-27B UD-Q4_K_M in the separate TurboMind dense integration (#869). TP4 generation completes with four reference-matching greedy bodies (Paris, arithmetic, translation and a Chinese explanation); model throughput and broader quality are recorded in that integration scope.

Greedy token agreement is diagnostic evidence rather than a correctness gate. Numerical distributions and task quality determine model correctness; small logit-margin changes can alter greedy output.