[bong-water-water-bong]

Engine half of item 4 on the Lemonade checklist. Lemonade loads embedding and reranking models as models of their own, but 1bit serve only served them as companions of a chat model (--embed, --rerank).

The change: --embedding and --reranking serve the model itself in that role. llama-server runs on the device asked for (vulkan, hrx, rocm) with that flag, and 1bit serve exposes /v1/embeddings (+ /embeddings) or /v1/rerank (+ /rerank, /v1/reranking, /reranking).

Measured on Strix Halo:

• Direct: embeddings work on Vulkan and HRX (768-dimension vectors). Rerank works on Vulkan: "Paris is the capital of France." scores 0.20 against 0.0016 for the other document.
• Rerank on HRX fails: HRX's JIT can't link the fp32 matmul-with-bias kernel jina-reranker-v1-tiny needs. docs/serve.md documents this.
• With the fork's matching recipe change (1bit-MONSTER/lemonade, branch onebit-embed), Lemonade's LLM suite through onebit:
  • Vulkan: 29 of 29 run tests pass, with 2 skipped (echo and generation parameters, which Lemonade's llamacpp recipe doesn't declare either);
  • HRX: 26 of 29 pass; the 3 reranking tests fail on the HRX JIT bug.

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit 92c522a

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Incorrect Role Handling for Reranking The code introduces a new --reranking flag that sets o.role to "reranking", but the logic in serve_child and the HTTP handler only checks for o.role == "embedding" to set up the appropriate endpoints. This means that when --reranking is used, the HTTP server will not correctly register the /v1/rerank and related endpoints, as the condition o.role == "embedding" will be false. This could lead to the reranking model not being accessible via the expected API endpoints. if (o.role == "embedding") for (const char* path : {"/v1/embeddings", "/embeddings"}) srv.Post(path, h); else for (const char* path : {"/v1/rerank", "/rerank", "/v1/reranking", "/reranking"}) srv.Post(path, h); Potential Incorrect Device Validation The validation logic for the device in the --role branch checks if dev is one of "vulkan", "hrx", or "rocm". However, it does not ensure that the device matches the role being served. For example, if a user specifies --reranking --device rocm, it should be valid, but the current check might not properly enforce that the device is appropriate for the specific role being served. This could lead to incorrect behavior or runtime errors. if (dev != "vulkan" && dev != "hrx" && dev != "rocm") throw std::runtime_error("--" + o.role + " runs on llama-server devices (vulkan, hrx, rocm), not " + dev); if (o.adaptive || !o.mtp.empty() || !o.prefill_device.empty()) throw std::runtime_error("--" + o.role + " does not combine with --adaptive, --mtp or --prefill-device");