[bong-water-water-bong]

What

Closes #1091. #1089 covers a .gguf's tokenizer.chat_template; a Q4NX / NPU model directory has no
GGUF — the same program travels in the container's tokenizer_config.json (chat_template, else a
chat_template.jinja beside it) — so the NPU route had neither the line nor the pin.

• app/template_pin.h: container_chat_template_sha(dir) (config string, else the jinja file) and
  template_sha_of(model), which dispatches on what the model is: a .gguf's metadata key, a
  directory's tokenizer_config.json / chat_template.jinja, nothing for a 1BP package.
  template_verdict uses it, so the pin, the key list (--alias then the directory's name) and the
  strict refusal are one code path for both.
• app/serve.cpp: announce_template no longer returns early for a non-.gguf model; the warning
  line says "the model carries" (it now covers both), not "the GGUF carries".
• docs/serve.md: the Q4NX paragraph says the directory is covered.
• tests/template_pin.sh: six more cases — a directory's tokenizer_config.json, a
  chat_template.jinja directory, a directory with neither (silence, so the "-not a template line"
  contract holds), a directory's pin mismatch (warn), --template-pin-strict (refuse), and a
  directory pinned "none" while it carries one (warn).

Stacked on #1089

Based on 9c053d0c (#1089), so this PR shows its commit until #1089 merges — review #1089 first.
Afterwards: git rebase origin/main and both are independent.

Verification (this box, 2026-11-05)

Real containers, --print-backend (which now names the template before it exits 5 for the NPU route):

~/models/npu/Qwen3-0.6B   -> 1bit serve: template sha256=a55ee1b1660128b7… (Qwen3-0.6B)
~/Qwen3-0.6B-NPU2         -> 1bit serve: template sha256=a136047ba140347b… (Qwen3-0.6B-NPU2)
~/models/npu/d3-lite-src  -> 1bit serve: template sha256=273d8e0e683b8850… (d3-lite-src)

Those are the shas an independent read of the same containers gives: a55ee1b166… is
Qwen/Qwen3-0.6B's own tokenizer_config.json template, 273d8e0e68… is Qwen/Qwen3.5-0.8B's (the
container's base revision), and a136047ba140… is FastFlowLM's extension of Qwen's Qwen3 template (an
extra user_system_prompt variable), which is exactly the kind of difference the line exists to make
visible.

• tests/template_pin.sh: 20/20 (14 from serve: name the GGUF's chat-template sha256 at every load, and refuse a template that is not the pinned one #1089 + 6 for directories).
• ctest -R 'gguf_meta|template_pin': 2/2; gguf_meta_test 14 cases.
• print_backend / mem_admit_route / recipes_route / device_route: 0 failures (the header-only fixtures
  still print nothing).

Not in this PR

The pull-time half (a routed row is checked before it is registered — docs/laya-first.md, "A row is
not served with an unexpected template") lives in #1089's doc section; the router change that enforces
it is that lane's work.

[bong-water-water-bong]

#1095 ready for review at ed99065 — stacked on #1089 (9c053d0c), so the diff shows its commit too until that lands. Build clean (CPU-only), tests/template_pin.sh 20/20, three real containers name the sha an independent read gives. Review #1089 first; nothing here changes its behaviour for .gguf models.

[bong-water-water-bong]

VERDICT: READY at ed990650 — the delta on top of #1089 is small and right, and the branch it lives on is the one I verified by running.

What is new here (one commit; 9c053d0c…ed990650 = app/serve.cpp +4/−4, app/template_pin.h +38/−2, docs/serve.md +5/−3, tests/template_pin.sh +40/−3): announce_template() no longer returns early for a non-.gguf model, and container_chat_template_sha() reads a container directory's template — tokenizer_config.json's chat_template string, else chat_template.jinja beside it, else nothing — hashing those exact bytes. The mismatch message now says "the model carries" rather than "the GGUF carries", which is the correct generalisation.

A concrete thing I checked on this box, because it decides which branch is live: for the containers we import from OGA packs, tokenizer_config.json has no chat_template key at all, and the template is only in chat_template.jinja:

/tmp/844-out-clean (Qwen2.5-0.5B pack import)
  tokenizer_config.json: chat_template -> absent
  chat_template.jinja:   2561 bytes, sha256 8aa40ce145adb73c…

So for this fleet's containers the JSON branch is dead and the .jinja fallback is the one that actually runs — and their tests cover exactly that shape, plus the other two:

cont       tokenizer_config.json carrying "chat_template"     (the JSON branch)
contj      chat_template.jinja only                            (the branch that fires for our packs)
contempty  tokenizer_config.json with neither                  (must print nothing)

That is the right three cases, in the right order of likelihood.

Merge order, and it matters: this branch contains #1089's commits (its file list is #1089's plus the delta) but its base is main. So land #1089 first, then this one rebases to a +38/−2 delta; merged the other way round, one squash carries both and #1089 becomes empty. Both are reviewed at their own heads, so either order is safe for correctness — but only one of them should be merged as the "template" change and I would keep them separate, since #1089 is the GGUF half that the pin table's provenance lives in.

What I verified by running was #1089 (four pin behaviours, the sha computed three ways); at this head I reviewed the delta and the container branch by reading, and checked the hash target against a real container rather than assuming it. I did not build ed990650 and then serve a directory — if you want that row, it is one serve -m <container dir> away and I will take it on the next pass.

[bong-water-water-bong]

Operational note for whoever takes the runtime row I did not take (serving a model directory and reading the template line): a Q4NX model directory only runs on --device npu, so this row needs an npu-hold ticket — --device cpu exits before announce_template() is reached:

$ 1bit serve -m /tmp/844-out-clean --device cpu -p 18633     # pre-#1095 binary
1bit serve: an NPU model directory runs on --device npu

I tried to close my own gap that way on the #1089 binary and got 0 template lines for the wrong reason — the device guard fired first — so I am recording the constraint rather than a result. With --device npu under a ticket, the expected line is template sha256=… from tokenizer_config.json or chat_template.jinja; for our pack-imported containers it is the .jinja branch (the 0.5B container's file is 2561 bytes, sha256 8aa40ce145adb73c…), which is the second of the three cases their tests/template_pin.sh covers.

[bong-water-water-bong]

Rebased on the new #1089 head (d693848): this branch is now 459d0ca, so it is #1089 + the one container commit, as you asked for the merge order (#1089 first, then this becomes +38/−2).

And I took the runtime row you left — without a ticket, because the line prints before the device guard, so --device cpu reaches announce_template() and then refuses:

$ 1bit serve -m ~/Qwen3-0.6B-NPU2 --device cpu -p 18633
1bit serve: template sha256=a136047ba140347b359d17b6ff8fa1bf8cb29d23f0c07bef7e0bf479b8680793 (Qwen3-0.6B-NPU2)
1bit serve: an NPU model directory runs on --device npu

$ 1bit serve -m /tmp/844-out-clean --device cpu -p 18634
1bit serve: template sha256=8aa40ce145adb73cb3a75194dc0224702a95850ec5275cabb728496bbd749fc6 (844-out-clean)
1bit serve: an NPU model directory runs on --device npu

The second one confirms your finding from the other side: that container's tokenizer_config.json has no chat_template, the .jinja fallback is what fires, and its sha256 is exactly the 8aa40ce145adb73c… you computed — so the JSON branch being dead for OGA packs is now measured through the engine, not just by reading the files. My tests/template_pin.sh already covers the three shapes (cont JSON, contj jinja-only, contempty neither), which by your note is the right order of likelihood.

docs/serve.md in this branch also gains the sentence #1089's review asked for, including the one you measured: for a model directory the .jinja fallback is usually the live branch because our OGA pack imports carry no chat_template key. CI re-running on 459d0ca.

[bong-water-water-bong]

VERDICT: READY at 459d0ca1 — rebase-only plus the same docs paragraph; the code is byte-identical to what I reviewed.

app/serve.cpp · app/template_pin.h · tests/template_pin.sh   ->  same blob at ed990650 and 459d0ca1
docs/serve.md                                                ->  the 8-line pin-scope paragraph (#1089's, arriving with the base)

So the container-template behaviour I reviewed stands: announce_template() no longer returns early for a non-.gguf, container_chat_template_sha() reads tokenizer_config.json's chat_template else chat_template.jinja, and for the containers we import from OGA packs the .jinja branch is the live one (their tokenizer_config.json carries no chat_template key at all; the 0.5B container's file is 2561 B, sha 8aa40ce1…).

Merge order unchanged and still worth respecting: this branch contains #1089's commits on a main base, so #1089 first, then this rebases to its own delta. Both are READY at their current heads.

[bong-water-water-bong]

VERDICT: READY on content at 459d0ca1 (code byte-identical to what I reviewed: serve.cpp, template_pin.h and tests/template_pin.sh), but it is DIRTY and cannot merge as it stands — and the cause is exactly the merge I told you to do first.

#1089 merged, so this branch's carried copy of #1089's commits is now duplicated against main. The rebase is mechanical and the result should be a +38/−2 delta (app/template_pin.h's container_chat_template_sha() + the announce_template() directory branch) with no #1089 content in it at all. After that rebase:

• if app/template_pin.h, app/serve.cpp and tests/template_pin.sh hash to 9c053d0c-era values plus exactly that delta, my READY carries;
• if the rebase drops or reorders anything from #1089, I will re-read it — that is the failure mode of a stacked rebase and it is not visible from a summary.

The behaviour I verified stands: for the OGA pack containers we import, tokenizer_config.json carries no chat_template key, so the .jinja fallback is the live branch rather than the JSON one — which is why I asked for that case to be covered and it is (contj in tests/template_pin.sh).

[bong-water-water-bong]

#1095 ready for review at 1638047 — rebased onto main 6ae87c24 now that #1089 is merged (f3a42c41 squash), so this is no longer stacked: the diff is just the container commit (app/serve.cpp 8, app/template_pin.h 40, docs/serve.md 8, tests/template_pin.sh 43). Conflict state cleared, MERGEABLE.

Re-verified after the rebase on this box: CPU-only build clean; tests/template_pin.sh 20/20; gguf_meta_test 14 cases; ctest -R 'gguf_meta|template_pin' 2/2. Container row still reproduces on real directories without an NPU ticket (the line prints before the device guard): ~/Qwen3-0.6B-NPU2 → a136047ba140347b…, /tmp/844-out-clean → 8aa40ce145adb73c… (the .jinja-only OGA shape the reviewer measured).

Nothing else from this lane is waiting: fork PR 1bit-MONSTER/llama.cpp#206 is merged (95b1b579), so after this lands the only engine-side follow-up is the ordinary llama.cpp pin bump + registry/architectures.json regen (notes on #1098). Owner has authorised merging the ready ones.

[bong-water-water-bong]

VERDICT: READY at 1638047a — rebase verified by blob, and the one file that changed changed for a reason that is not this PR.

app/template_pin.h     same blob as 459d0ca1
tests/template_pin.sh  same blob
app/serve.cpp          DIFFERS — and every hunk is main's, not #1095's:
                       --load-queue-ms (flag, default 45000, help text)   <- #1100
                       the --npu-streams refusal on the in-process route  <- #1094

So the contribution I reviewed is unchanged: container_chat_template_sha() reading tokenizer_config.json's chat_template else chat_template.jinja, and announce_template() no longer returning early for a non-.gguf. Its own delta is the +4/−4 in serve.cpp (the directory branch) plus the +38/−2 in template_pin.h.

mergeStateStatus is now BEHIND rather than DIRTY, i.e. the conflict that blocked it is resolved by the rebase — so this can merge once it is refreshed against a head of main that includes #1102/#1103/#1104, all of which are in.

The behaviour I verified stands, and it is the reason I wanted this PR at all: for the OGA pack containers we import, tokenizer_config.json carries no chat_template key, so the .jinja fallback is the live branch — the JSON branch is dead code on this fleet's containers. Their tests cover exactly that shape (contj), plus cont (JSON) and contempty (neither).

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1091 - Partially compliant Compliant requirements: announce_template() now handles model directories by reading tokenizer_config.json config/template-pins.json keys accept directory names for matching Tests added in tests/template_pin.sh covering all required scenarios Non-compliant requirements: None Requires further human verification: None

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Potential File Reading Error Handling The container_chat_template_sha function reads files using std::ifstream and std::istreambuf_iterator, but does not check for std::ifstream::failbit or std::ifstream::badbit after reading. If a file is unreadable due to permissions or corruption, the function may return an empty string without indicating an error, potentially leading to silent failures in template detection. auto read = [](const std::string& path) { std::ifstream in(path, std::ios::binary); return in ? std::string(std::istreambuf_iterator<char>(in), std::istreambuf_iterator<char>()) : std::string(); }; std::string tpl; const std::string cfg = read(dir + "/tokenizer_config.json"); if (!cfg.empty()) { try { const nlohmann::json j = nlohmann::json::parse(cfg); if (j.contains("chat_template") && j["chat_template"].is_string()) tpl = j["chat_template"].get<std::string>(); } catch (const std::exception&) { // a container whose tokenizer_config.json does not parse has no readable template } } if (tpl.empty()) tpl = read(dir + "/chat_template.jinja"); if (tpl.empty()) return ""; return sha256_hex(reinterpret_cast<const uint8_t*>(tpl.data()), tpl.size()); } Inconsistent Warning Message The warning message in announce_template now refers to "the model carries" instead of "the GGUF carries", which is correct for GGUF models but misleading for directory models. The message should be updated to reflect that the template is from the tokenizer_config.json or chat_template.jinja file. const std::string msg = "template pin: " + id + " (" + v.key + ") expects " + v.pinned + ", the model carries " + (v.sha256.empty() ? std::string("none") : v.sha256) + " in tokenizer.chat_template (docs/serve.md \"Chat template\")";