[bong-water-water-bong]

1bit serve --device onnx serves ONNX Runtime GenAI models, the format of Lemonade's ONNX models. A directory with a genai_config.json goes there on its own. See docs/onnx.md.

How it works

• The server. third_party/ryzenai-server (lemonade-sdk/ryzenai-server, MIT) is pinned at 69765cc. It is an OpenAI-compatible server around ONNX Runtime GenAI, and the engine runs it as a child process, exactly as it runs llama-server.
• The build. scripts/build-onnx.sh builds it against Microsoft's ONNX Runtime GenAI 0.11.2 and ONNX Runtime 1.23.2 (MIT), fetched pinned and sha256-checked, which gives CPU execution. -DONEBIT_ONNX=ON runs the script during the engine build.
• NPU and hybrid. These need AMD's Ryzen AI Software (the Vitis AI execution provider, under its own licence). The engine does not ship it: with RYZENAI_INSTALL_PATH set, the same script builds against the user's install.
• The binary carries DT_RPATH $ORIGIN, so ONNX Runtime GenAI's run-time dlopen of libonnxruntime.so finds the copy beside it.
• NOTICE names ryzenai-server, what its build fetches (cpp-httplib, nlohmann/json, the GenAI headers), and ONNX Runtime / GenAI.

Tested on Strix Halo, from a clean clone with -DONEBIT_ONNX=ON, on amd/Qwen2.5-0.5B-Instruct-quantized_int4-float16-cpu-onnx (CPU int4), with no --device:

• 1bit serve -m <dir> chose onnx. It answered "The capital of France is Paris." and a list of three colors at 125–129 tok/s, and streamed SSE.
• /v1/models reports device onnx, and stopping 1bit serve stopped the backend.
• ctest 6/6.

The next step is ryzenai-server.exe in scripts/build-windows.sh (ONNX Runtime GenAI's Windows builds add DirectML).

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit 084b3a4

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Incorrect device validation for ONNX models The validation logic for ONNX model directories in serve_child function (lines 967-973) only checks if the device is "auto" or "onnx", but does not ensure that the model directory actually contains a valid ONNX Runtime GenAI model structure. This could lead to runtime errors if the directory structure is incorrect or missing required files like genai_config.json. if (is_onnx_model_dir(o.model)) { if (o.device != "auto" && o.device != "onnx") throw std::runtime_error("an ONNX Runtime GenAI model directory (genai_config.json) runs on --device onnx"); Options onnx = o; onnx.device = "onnx"; return serve_child(onnx); } Potential library path issues The script uses ln -sfn to link libraries, which may not work correctly in all environments or with all filesystem types. If the symbolic link creation fails or is not properly resolved, it could lead to runtime errors when ryzenai-server tries to load required libraries. ln -sfn "$work/oga/lib" "$oga_root/deployment/lib"