Skip to content

Status of the official gguf → q4nx conversion tool announced in v0.9.26 — and supported path for fine-tuned Qwen3/Qwen3.5? #828

Open

Description

@ScottyVenable

Hi — in the v0.9.26 release notes (Jan 2026), the "Runtime Restructure for Fine-Tuned Models" section said a gguf → q4nx conversion tool was "almost ready." We're now on v1.0.8+ and I can't find the official tool or docs. Could you clarify:

  1. Did the official conversion tool ship (under what name/docs URL), or was the plan superseded?
  2. Our use case: fine-tunes (merged LoRA/full SFT) of catalog architectures — text Qwen3 and Qwen3.5 at 0.6B–9B. If self-serve conversion is supported, which families/sizes, GGUF quant formats, and context lengths are in scope?
  3. The community Q4NX_Converter + flm-add path (convert GGUF to .q4nx, register via model_list.json) currently works with the restructured runtime. Is that a configuration you consider supported/stable across releases, or best-effort?
  4. Relatedly: can Lemonade register a user-converted q4nx package under the flm recipe, or is the flm recipe limited to catalog packages?
  5. If self-serve isn't the intended route: is the FastFlow SDK "Customize" tier (IRON/HRX) meant for this, or is there a partner conversion program? Any timeline you can share?

Context: we run a small local fleet on Ryzen AI (XDNA2) and want our fine-tuned variants on the NPU seat; the stock catalog models already serve there via FLM. Happy to test pre-release tooling and report results. Thanks!

Activity

  1. jtuyls commented on Oct 28, 2026

    @jtuyls
    Collaborator

    Hi @ScottyVenable, you might be able to use the public https://github.com/ROCm/FLM_Q4NX_Converter. Does that work for your use case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions