Skip to content

release-check: a laya-auto row, so the first request is the one Laya decided - #1071

Merged
bong-water-water-bong merged 4 commits into
mainfrom
feat/laya-release-row
Nov 5, 2026
Merged

bong-water-water-bong merged 4 commits into
mainfrom
feat/laya-release-row

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

What

docs/laya-first.md §7 makes this the gate for the whole feature: on a cold Lemonade the first request must
go through Laya's router, which classifies it and picks one of our models. This adds that as a
release-check row, laya-auto.

Why it is a verdict and not a chat test

laya_verdict() requires all four of:

  1. the class's own Laya rule matched (matched_rule: laya-<class>),
  2. the request went to that class's model,
  3. it did not fall through to default_model ("default_used": false),
  4. Laya's score is in the trace (classifier:laya).

The case it exists for is a request that lands on the right model by accident — a deterministic rule, or
the default — which any status-only row calls healthy. That is exactly the failure a first-request gate has
to catch: the request answered 200, so nothing looks wrong, and the whole premise of the feature is silently
absent.

Self-test (device-free, CI)

./os/mini/release-check.sh --self-test → 65 checks pass, 10 new, and ctest's release_check_row passes:

ok   PASS                                                         # code class, right model, no fall-through
ok   PASS                                                         # class/model not constrained
ok   FAIL fell open to default_model, so Laya did not decide
ok   FAIL no Laya rule matched the request                        # a deterministic rule decided
ok   FAIL matched the code rule, not the prose one
ok   FAIL routed to another model than Qwen3-0.6B-1bit
ok   FAIL no Laya score in the trace
ok   FAIL no Laya rule matched the request                        # empty body
ok   PASS                                                         # answer text in front of the decision
ok   short -> Qwen3-0.6B-1bit / prose -> ZAYA1-8B-1bit / code -> Qwen3-Coder-30B-A3B-Instruct-1bit /
     long_doc -> Qwen3.8-27B-1bit / not-a-class -> (empty)

Skip rather than fail

On a stock stick there is no Laya checkpoint, so no 1bit.Laya-First router is registered and the row
skips with the fix named (scripts/fetch-laya.sh, then 1bit laya-first --policy …) instead of failing a
release it cannot judge. A row that cannot pass on the artifact must say so — silently passing would be worse,
and failing would block on something that is not a regression.

The class → model mapping the row asserts mirrors config/laya-first.json (engine #1069), and is itself
self-tested, so changing the table turns into a red self-test rather than a row that checks yesterday's
promise. 1bit laya-first --check validates the table itself.

Not run on a device

The row's own arm needs a board with a Laya checkpoint and the router registered — the release lane's window,
not this one. What is proven here is the verdict logic, the mapping, and the skip path; the board arm is
written and will run when the next stick carries a checkpoint.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at c928ce4-equivalent head — self-test passes here and the verdict helper is tested in both directions, which is the thing that makes a gate worth having.

Ran it: bash os/mini/release-check.sh --self-test → release-check self-test PASS, including the class→model mapping (prose -> ZAYA1-8B-1bit, code -> Qwen3-Coder-30B-A3B-Instruct-1bit, long_doc -> Qwen3.8-27B-1bit, not-a-class -> (empty)).

The part that matters — laya_verdict() is exercised as a gate, not as a happy path. Nine cases, six of them failures, and the failures are the ones that would otherwise pass a status-only row:

PASS  the good decision
FAIL  default_used:true                      -> fell open to default_model, so Laya did not decide
FAIL  matched_rule:"long-input"              -> a deterministic rule decided, not Laya
FAIL  matched laya-prose when expecting code -> the wrong class's rule
FAIL  routed to Qwen3-0.6B when code needs the Coder -> the wrong model
FAIL  no "classifier:laya" in the trace      -> no Laya score
FAIL  empty body                             -> nothing answered

That is the "answered 200 so nothing looks wrong" case caught by construction: a request that lands on the right model by accident fails three of those six.

The row itself only fires when it can mean something. It skips (visibly) unless a 1bit.Laya-First router is registered in MODELS, and it probes one prompt per class with the strict laya_verdict "$body" "$class" "$model" form, so the strict checks are the ones in force on an appliance.

One nit, and it is the fail-open shape that has bitten this repo before: laya_verdict silently checks less when it cannot map the class. _cls/_want empty skips both "which class" and "which model", so laya_verdict "$body" '' '' returns PASS for any body that matched some laya-* rule, did not fall open, and carried a Laya score — the self-test's own line 540 asserts that behaviour. Today the row always passes a known class so _want is never empty, and laya_class_model() returns '' for an unknown one, which is exactly how an empty pair could reach the helper if a future row adds a class without touching the mapping. One line closes it: if $2 is non-empty and laya_class_model came back empty, return FAIL no model is named for class <x> in this script rather than checking only the shape. A gate should fail closed when it does not understand its own input.

Not verified: the live row on an appliance (it needs a Laya checkpoint and a running Lemonade) — everything above is the device-free half, which is the half CI runs.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Two defects in my own row, both fixed at 28c2c27e — found by exercising it off the stick instead of reading it.

  1. The name is not a constant. A live lemond refuses to register anything outside user.*: "Registered model definitions must use a non-empty user.* name". My row looked for 1bit.Laya-First, which the server cannot even hold. Because that is the name laya-first: ship the routing table and desugar it to a collection.router policy #1069's desugar emitted, laya-first: ship the routing table and desugar it to a collection.router policy #1069 is fixed too (fb30a9c0: model_name is now user.Laya-First). The row now reads the name from the engine's own 1bit laya-first output and falls back to user.Laya-First, so a table rename cannot break it.
  2. auth_post is wget-shaped. It spells the body as --post-data, which is wget's flag; on an image with curl but no wget it returns nothing and every post row reads as "no answer". This row uses auth_post_watch, which handles both fetchers and returns the body. The other post rows have the same shape — flagging rather than changing them in this PR.

Also fixed in the same exercise: the row's class floor. min_score: 0.0 matched every class (0.0 >= 0.0), so short and prose routed to the code model on a live server. The row now ships and asserts 0.01, via #1069.

Measured after all fixes, against a live lemond with a stand-in engine: short, prose, code each PASS through the row's own verdict, and an engine that cannot answer FAILs with "no Laya rule matched the request" — the "answered 200 on the default" case the row exists to catch.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 28c2c27e — the router name now comes from the engine instead of the script, and that is the fix that makes this row survive the name change #1069 just made.

The change is right and I checked it against the binary it parses. #1069 corrected the desugared policy's model_name from 1bit.Laya-First to user.Laya-First (the server only registers user.* definitions), and this head stops hardcoding it:

LAYA_ROUTER=user.Laya-First        # the documented default, not the authority
for _b in $(command -v 1bit) /opt/1bit-engine/1bit /opt/1bit/1bit /opt/1bit-package/bin/1bit; do …
    _n=$("$LAYA_BIN" laya-first | sed -n 's/.*"model_name": *"\([^"]*\)".*/\1/p' | head -1)
    [ -n "$_n" ] && LAYA_ROUTER=$_n

On the rebuilt engine binary that parse yields user.Laya-First, which is exactly what the row then looks for in MODELS — so the row reads the name from the thing that produces it rather than from a copy that would drift the day the table changes. The fallback chain is sensible too: a documented default if no binary is found, and the row still skips visibly rather than passing when no such router is registered.

--self-test passes at this head (release-check self-test PASS), and auth_post → auth_post_watch is the right instrument for a row that must not block a release-check run on a slow first request.

My earlier nit stands unchanged and is still a nit: laya_verdict() checks less when it cannot map a class (laya_verdict "$b" '' '' → PASS on the shape alone, asserted by its own self-test line). Today the row always passes a known class, so nothing is wrong — but a gate that fails closed on an input it does not understand is one line better, whenever this file is next open.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

The auth_post half of my "found while exercising this row" note now has its own PR: #1079. Measured on a curl-only PATH against a live server: the old path returned 0 bytes (the row reports "did not answer" for a server that answered) and the new one returns the real 627-byte reply; with wget present the reply is unchanged (640 bytes), so the stick's behaviour does not move.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

The fail-closed nit is fixed at 4995fc3. laya_verdict() now returns FAIL no model is named for class <x> in this script when it is handed a non-empty class that laya_class_model() cannot map, before any of the shape checks, so an unmapped class can no longer skip both the class and the model check and PASS on shape alone. --self-test asserts that exact message (one new line) and stays green: 66 ok, 0 FAIL, self-test PASS. bash -n clean. Nothing else in the script moved.

bong-water-water-bong and others added 3 commits November 5, 2026 06:14
…decided

docs/laya-first.md §7 makes this the gate: on a cold Lemonade the first request must go through Laya's
router, which classifies it and picks one of our models. The row sends one prompt per class through the
Laya-first router and reads the router's own decision back.

What makes it a row and not a chat test is laya_verdict(), which requires all four of: the class's Laya
rule matched, the request went to that class's model, it did NOT fall through to default_model, and
Laya's score is in the trace. The case it exists for is a request that lands on the right model by
accident - a deterministic rule, or the default - which any status-only row calls healthy. That case is
in the self-test, along with the wrong class, the wrong model, a missing Laya score, and an empty body.

The row skips, with the fix named, when no 1bit.Laya-First router is registered: on a stock stick there
is no Laya checkpoint, and a row that cannot pass must say so rather than fail the release.

Self-test: 65 checks pass (10 new), and ctest's release_check_row still passes. The class -> model
mapping the row asserts is mirrored from config/laya-first.json and is itself self-tested, so a table
change turns into a red self-test instead of a row that checks yesterday's promise.

Not run on a device yet: the row needs a board with a Laya checkpoint and the router registered, which
is the release lane's window, not this one. What is proven here is the verdict logic and the skip path.
…her that works on curl-only images

Two defects found by exercising the row off the stick rather than by reading it:

1. The name is not a constant. A live lemond refuses to register anything outside `user.*`
   ("Registered model definitions must use a non-empty `user.*` name") - so the row now reads the name from
   the engine's own desugar (`1bit laya-first` prints `model_name`) and falls back to `user.Laya-First`. A
   hardcoded name would also break the row the day the table's name changes.
2. `auth_post` spells its body as wget's `--post-data`, so on an image with curl but no wget it returns
   nothing and every post row reads as "no answer". This row uses `auth_post_watch`, which handles both
   fetchers and hands back the body. (The other post rows have the same shape; flagging it here rather
   than changing them in a release-check PR.)

Measured against a live lemond with a stand-in engine: short, prose and code each PASS through the row's own
verdict, and an engine that cannot answer FAILs with "no Laya rule matched the request" - which is the
"answered 200 on the default" case the row exists to catch.
…annot name

laya_verdict() skips both the class check and the model check when it is handed an
empty expectation, so a row for a class that is not in laya_class_model() would PASS
on shape alone. Fail closed instead, naming the class, and assert it in --self-test.

  bash os/mini/release-check.sh --self-test   ->  66 ok, 0 FAIL, self-test PASS

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 4995fc31 — the author closed the fail-open I raised, and I proved the guard is load-bearing rather than decorative.

The delta since my READY (28c2c27e) is one commit, os/mini/release-check.sh +7/−0: a guard at the top of laya_verdict() that fails closed when laya_class_model has no model for the class, plus a self-test case for it. That is exactly the nit — "the helper checks less when it cannot map a class" — and it is fixed in the right place, before the rule check, so an unmapped class can no longer reach a check that is skipped by default.

Tested both versions of the function with the same body (a laya-ghost rule and a trace score, i.e. every other check satisfiable, _want empty so the model comparison would be skipped):

OLD (28c2c27e)   laya_verdict <body> ghost ''  ->  PASS
NEW (4995fc31)   laya_verdict <body> ghost ''  ->  FAIL no model is named for class ghost in this script

So the old code passed a row whose class has no model — the vacuous pass — and the new one refuses it by name. --self-test passes at the new head, and the added case is the one that would have failed before.

Everything from my original READY stands: --self-test PASS, and laya_verdict() exercised in both directions, the six of nine cases that must fail still failing (fell-open, deterministic rule, wrong class, wrong model, no Laya score, empty body).

Nothing else in os/mini/release-check.sh moved, and this row remains documentation-or-gate only — no engine behaviour changes.

@bong-water-water-bong
bong-water-water-bong merged commit 05da4d8 into main Nov 5, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the feat/laya-release-row branch November 5, 2026 11:17
bong-water-water-bong pushed a commit that referenced this pull request Nov 5, 2026
… of them inside the split's file, and the moved block survives it

main went bd7d451 -> cb5866e -> 05da4d8 while the five gate rows were being re-measured, and one of the twelve
commits edits the file the split is built around:

  npu/forward/src/q4nx_forward.cpp   +38/-5   (#1087 rope-scaling refusal by name, #1072 the batch phase counters
                                               zeroed BEFORE BatchPhases snapshots them)
  npu/forward/include/q4nx_rope.h    +72      (new: q4nx_rope_declared_type / q4nx_rope_type_supported)
  npu/pack_import.cpp/.h             +97      (#844 final-norm import from the .fconst tail envelope)
  app/, config/, docs/, tests/, tools/, os/   (laya-first #1069, determinism tool #1068, serve logging #1074,
                                               onnx pricing #1076, puppy vm-check #1080, release-check #1079/#1071)
  third_party/llama.cpp              pin bump 6e42864a6 (#1075)

Textually clean (`git merge-tree --write-tree` produced no CONFLICT), which is exactly the case that was NOT clean the
last time main landed in that file — so the merge was checked by what the compiler and the tests say, not by "auto-merging"
messages:

- #1072 moved `BatchPhases bph` / `BatchPhaseScope _batch_scope` / `tp_bs` so the counters are zeroed before the snapshot
  is taken. Those symbols live in npu/forward/include/q4nx_lh.h in this branch (the move this PR exists for); the
  statement order is main's and is carried over verbatim, so the reordering lands on the header's definitions unchanged.
- The moved set is still defined exactly once: `inline StepPhases g_ph` (q4nx_lh.h:116), `phprof_on`/`ph_ms`
  (q4nx_lh.h, one each), `inline bool g_in_batch` (:131), `struct BatchPhaseScope` (:132), `lh_dump_layer_host` — and
  q4nx_forward.cpp no longer defines any of them.
- `BatchPhases` remains a local struct in q4nx_forward.cpp (one definition), which is where main keeps it.

Build: `cmake --build build-p4 --target onebit -j4` under ~/lb/thermal-run.sh -> rc 0, 16 objects rebuilt (warm tree; the
pin bump and the two touched TUs). Tests: `ctest` **87/87, 0 failed** (85 passed, 2 skipped: mtp_batch_all,
gdn_batch_order — the same two that skip on main).

Consequence, stated rather than implied: the five gate rows measured on A=bd7d451a are VOID and are being re-run against
A built at this main; this commit carries no numeric claim of its own.
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

1069 - Partially compliant

Compliant requirements:

  • Add config/laya-first.json with class → {model, hf_repo, file, device, ctx} mappings
  • Compile the table into the engine like config/recipes.json and config/pm-experts.json
  • Support --laya-first FILE and ${XDG_DATA_HOME:-~/.local/share}/1bit/laya-first.json as overrides
  • Add 1bit laya-first [--check] [--policy FILE] command to validate the table and print the policy
  • Ensure min_score: 0.01 on every class rule
  • Ensure labels on the classifier entry are declared to prevent silent rule mismatches
  • Validate the table strictly with version "1", exact four Laya class names, valid model/device/ctx fields
  • Ensure a bad table is refused with the offending value named
  • The emitted policy must be byte-identical to tests/laya_first_policy.json
  • The emitted policy must be structurally valid against Lemonade's route_policy.schema.json
  • The laya-auto row must be added to ensure the first request goes through Laya's router
  • The laya-auto row must verify that Laya decided, not a deterministic rule or default model
  • The laya-auto row must validate that the request went to the correct model for the class
  • The laya-auto row must ensure no fall-through to default_model
  • The laya-auto row must ensure Laya's score is in the trace

Non-compliant requirements:

  • Packaging the file to share/1bit/laya-first.json in a release (this is noted as out of scope)

Requires further human verification:

  • CI does not exercise schema validation unless $LEMONADE_ROUTE_POLICY_SCHEMA is set
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Incorrect Model Mapping in Self-Test

The self-test in laya_verdict uses hardcoded model names that may not match the actual models specified in config/laya-first.json. This could lead to false positives if the table changes but the script is not updated accordingly. The test should validate against the actual model names from the table rather than hardcoded values.

laya_class_model() {  # <class> -> the model the shipped table names for it
    case "$1" in
        short)    printf 'Qwen3-0.6B-1bit' ;;
        prose)    printf 'ZAYA1-8B-1bit' ;;
        code)     printf 'Qwen3-Coder-30B-A3B-Instruct-1bit' ;;
        long_doc) printf 'Qwen3.8-27B-1bit' ;;
        *)        printf '' ;;
    esac
}
Potential Race Condition in Router Detection

The code to detect the Laya router (LAYA_ROUTER=user.Laya-First) uses a loop to find the 1bit binary. If multiple versions exist, the script might pick an incorrect one, potentially causing the row to fail or use an unintended router. This could lead to inconsistent behavior across different environments.

LAYA_ROUTER=user.Laya-First
LAYA_BIN=
for _b in $(command -v 1bit 2>/dev/null) /opt/1bit-engine/1bit /opt/1bit/1bit /opt/1bit-package/bin/1bit; do
    [ -x "$_b" ] && { LAYA_BIN=$_b; break; }
done
if [ -n "$LAYA_BIN" ]; then
    _n=$("$LAYA_BIN" laya-first 2>/dev/null | sed -n 's/.*"model_name": *"\([^"]*\)".*/\1/p' | head -1)
    [ -n "$_n" ] && LAYA_ROUTER=$_n
fi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant