Skip to content

npu-pack: a pack with no text artifacts imports clean and only fails later, at 1bit serve #1029

Closed

Description

@bong-water-water-bong

npu-pack: a pack with no text artifacts imports clean and only fails later, at 1bit serve

What happens

1bit npu-pack import copies the pack's tokenizer/config files when it finds them and says nothing when it
finds none. The result is a model directory with model.q4nx + config.json and no tokenizer.json, and the
command that then fails is 1bit serve, two commands later:

$ 1bit serve -m <imported dir>
<dir> is not an NPU model directory (model.q4nx, config.json, tokenizer.json)

That message reads like an engine bug, not an incomplete pack, and there is nothing in the import's own output to
point at the pack.

Why it matters beyond a missing file

Two of the four AMD OGA packs on this box were exactly that case — Qwen3-1.7B_rai_1.8.0_npu_4K and
Qwen3-4B_rai_1.8.0_npu_4K held 5 files each (weights + config.json + cache/) because they were fetched weight
by weight. AMD's repos ship all of it: 71 files each, including tokenizer.json (11,422,654 B),
tokenizer_config.json, vocab.json, merges.txt, chat_template.jinja, genai_config.json. The silence at
import time is what let a previous session conclude "the Qwen3 packs ship no tokenizer files" — a false statement
about the vendor's packs recorded as a fact (amd-oss/npu-re FACTS §7.26).

Which text files a pack carries also varies by pack, so a fixed expectation is wrong in both directions: the
Llama-3.2-1B pack ships a 17.2 MB tokenizer.json and no vocab.json/merges.txt at all.

Measured (before)

1bit npu-pack info on a 5-file pack: no mention of the tokenizer, chat template or generation config anywhere in
the 200-line JSON.

Fix

Report the split instead of assuming it:

command now says
npu-pack info text_artifacts: {present: [...], missing: [...], servable: bool} (info writes nothing, so this is visible before an import)
npu-pack import text: N of 8 tokenizer/chat files copied[, absent from the pack: ...], or a WARNING: naming tokenizer.json and the files to put beside model.q4nx
the container config.json records q4nx_source.text_artifacts, so a container read later states whether it ever had the text

Same shape as the existing tools/gguf_to_q4nx.py behaviour ("warns loudly when tokenizer.json is missing") that
the pack path never had. servable is exactly "tokenizer.json is present", which is what docs/npu.md requires
of a model directory.

Verification

  • npu_pack_import_test gains test_text_artifacts: a pack with none reports 8 missing / not servable, and the
    same fact through manifest(); with two files present it reports those two, six missing, servable.
  • Real packs, both directions: a 5-file pack (warns, container records "none in the pack") and a complete one
    (8 of 8 ... copied, container records "copied from the pack" and 1bit serve starts).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions