Skip to content

os/v3 puppy: vm-check asserts the console names the engine (#1041), so the row cannot go missing again - #1053

Merged
bong-water-water-bong merged 2 commits into
mainfrom
os/v3-c25-console-engine
Nov 5, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
os/v3-c25-console-engine

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

os/v3 puppy: vm-check asserts the console names the engine (#1041), so the row cannot go missing again

Root cause (what was unasserted)

#1041 (0c862e64) made the console name the engine that is serving — the TUI gains a LEMONADE engine: llamacpp (gpu)
row and stops printing web UI http://-:13305 before it has an address. No gate asserted either:

  • vm-check.sh greps /etc/inittab for the 1bit-console respawn line, and checks the boot console — but not what the
    screen says.
  • 1bit-release-check (release rows) has no console row either.
  • The only things holding os/v3/puppy: the console names the engine that is serving, and stops printing a URL before it has one #1041 in place were pi-tui's own unit tests and my out-of-tree cut check
    (~/wt/os3i/c24-tui-check.sh), both of which run before an image exists. An image shipping a TUI that lost the row
    would have passed every gate we run.

That is the same silent-absence class as pi-release's efivarfs.ko finding: the artifact is checked for presence, not
for the behaviour the cycle added.

The row, and why it is not written off live facts

("U2 console engine", "1bit-console-tui --snapshot 80x25 --fixture 2>&1 | grep -o 'engine: [a-z]* ([a-z]*)' | head -1; "
                 "1bit-console-tui --status > /tmp/ct.txt 2>&1; echo status-rc=$?; "
                 "echo invented-urls=$(grep -c 'http://-:' /tmp/ct.txt)",
                 "engine: llamacpp (gpu)"),

The obvious row — run 1bit-console-tui --status and grep engine: — would be red on a healthy appliance, so it
would have cost a cycle a red nobody could act on. Measured on this box, nothing loaded:

LEMONADE  up (pid 2298)  http://192.168.50.110:13305  API key: none (blank mode), press 1

No engine line at all: the fact is None until a model is loaded, by design (facts.rs: engine is None when nothing
is loaded). So the row renders the shipped binary's own frame with its own fixture instead. That asserts what the image
can do without needing a model, a device or a network — and the transcript next to it carries the second half of #1041
(invented-urls=0).

Measured, before and after

Every claim below was measured, not quoted. The "before" arm uses the exact bytes the c24 stick ships, extracted from its
own sfs, so it is the image's behaviour and not a rebuild:

arm binary the row's command verdict
before — c24 image's /usr/local/bin/1bit-console-tui 5261cf56… (1,567,800 B, extracted with unsquashfs -cat) --snapshot 80x25 --fixture | grep -o 'engine: …' → no output row FAILs
after — #1041 4b369c10… (1,581,392 B) same command → engine: llamacpp (gpu) row PASSes
vm-check on the shipped c24 image with this patch 1bit-os-puppy.img (9355f9f6…) full run, UEFI + Secure Boot 58 PASS / 1 FAIL / 3 SKIP — the single FAIL is this row (FAIL U2 console engine ... status-rc=0 | invented-urls=0), so nothing else moved
vm-check on an image built from the #1041 tree with CONSOLE_TUI_BIN=4b369c10… 1bit-os-puppy.img (~/wt/puppy-out-c25test, gate-proof artifact, not a hand-off) full run, UEFI + Secure Boot 59 PASS / 0 FAIL / 3 SKIP, vm-check rc=0 — the row prints PASS U2 console engine engine: llamacpp (gpu) | status-rc=0 | invented-urls=0, and every other row is unchanged

Expected red on pre-#1041 images, by design. The c24 stick carries 5261cf56…, so this row is red there and green
from the first image that carries #1041. Rows move with the tree — U3 mdns out is the same story from c24 — and this is
stated so nobody reads a c24 re-run as a regression.

The two images are a controlled comparison, so the delta is attributable: same payload (opt/1bit-package/1bit/1bit =
502d2a81… in both), same etc/init.d/10lemonade (ba543b6a…), and only usr/local/bin/1bit-console-tui differs
(5261cf56… → 4b369c10…).

Checks

bash -n clean, shellcheck -S warning clean, tools/copyright.py --check rc 0, and the row list still parses (74
rows, U2 console engine present) — verified by compiling the driver's own Python block rather than by reading it.

Not in this PR

  • No row for the console's live engine text (needs a loaded model; the gate does not load one, and a row that
    silently SKIPs is worse than no row). When a gate row loads a model, the live assertion is one line on top of this.
  • No change to the c24 artifact or to pi-release's release rows.

…o the row cannot go missing again

#1041 made the console name the engine that is serving, and no gate asserted it: vm-check greps /etc/inittab
for the 1bit-console respawn line and checks the boot console, but not what the screen says, and release-check
has no console row either. The only holders were pi-tui's unit tests and my out-of-tree cut check, both of
which run before an image exists - so an image shipping a TUI that lost the engine row passed every gate.

The row renders the shipped binary's own frame with its own fixture rather than grepping --status for
'engine:': measured here, a box with nothing loaded prints no engine line at all (the fact is None until a
model is loaded), so the obvious row would be red on a healthy appliance. The transcript carries the other
half of #1041 too (invented-urls=0).

Measured: the c24 image's own /usr/local/bin/1bit-console-tui (5261cf56..., unsquashfs -cat) prints no
engine line -> this row FAILs on it; 4b369c10... prints 'engine: llamacpp (gpu)' -> PASSes.
vm-check on the shipped c24 image: 58 PASS / 1 FAIL / 3 SKIP, the single FAIL being this row, so nothing else
moved. Red on pre-#1041 images is by design and is stated in the PR.
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit c2d8232)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

1041 - Partially compliant

Compliant requirements:

  • The console now displays the engine that is serving in the format engine: <recipe> (<backend>)
  • The console no longer prints a URL before it has an address
  • The vm-check.sh now asserts that the console names the engine that is serving
  • The console correctly shows no engine row when nothing is loaded
  • The console no longer shows invented URLs before an address is available

Non-compliant requirements:

  • The PR does not update documentation or README files to reflect the new console behavior

Requires further human verification:

  • Verification that the console behavior is consistent across different hardware configurations
  • Confirmation that the snapshot tests are correctly capturing the expected console output
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Assertion Logic

The new assertion in vm-check.sh relies on comparing the output of --snapshot 80x25 --fixture against a known good binary. This approach is fragile and depends on the exact binary version being tested. If the binary changes, the assertion may fail even if the functionality is correct.

("U2 console engine", "1bit-console-tui --snapshot 80x25 --fixture 2>&1 | grep -o 'engine: [a-z]* ([a-z]*)' | head -1; "
                 "1bit-console-tui --status > /tmp/ct.txt 2>&1; echo status-rc=$?; "
                 "echo invented-urls=$(grep -c 'http://-:' /tmp/ct.txt)",
                 "engine: llamacpp (gpu)"),

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit a92c607

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at a92c607e — the row bites on the old binary and passes on the new one, and I reproduced both halves myself.

The control that matters for a gate like this is "does it go red on the image that lacks the behaviour". Built the console at 0c862e64^ (the commit before #1041) and at a head that carries it, and ran the row's own command against each:

PRE-#1041  frame: LEMONADE up 8882 | GPU HRX0 43C | NPU idle | MODELS 2 | FANS controlled
           grep -o 'engine: [a-z]* ([a-z]*)'  ->  (nothing)              <- the row FAILs, as claimed
POST-#1041 same grep                            ->  'engine: llamacpp (gpu)'   <- the row PASSes

and the row's second half on this box:

1bit-console-tui --status   -> status-rc=0, invented-urls=0

so the transcript next to the assertion carries the other half of #1041 (no http://-: before there is an address) and does not go red on a healthy appliance.

The design decision behind the row is the right one and is stated in the comment: asserting off live facts would have made this row red on every healthy appliance, because engine is None until a model is loaded — measured here, --status with nothing loaded prints no engine line at all. Rendering the shipped binary's own frame from its own fixture asserts what the image can do without a model, a device or a network, which is what a vm-check row is for.

Two notes, neither blocking:

  1. The row pins the fixture's answer, engine: llamacpp (gpu). That is correct today (the fixture loads one llamacpp/gpu model) but it means a future fixture that changes the model or its device moves this row too — worth the one comment line it already has, and worth remembering when the fixture is next touched.
  2. MERGEABLE BLOCKED at the moment I read it was just the two CodeQL jobs still running on this head (build and CodeQL had not reported); nothing in the diff is waiting on a human.

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit c2d8232

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at c2d82321 (rebase-only; the review stands).

The PR's own file is unchanged by the rebase — the diff between the two heads is other PRs' content (#1050's facts.rs, #1052's stop-token test) arriving with the new base:

git diff a92c607e…c2d82321 -- os/v3/puppy/vm-check.sh     (no output)

So the row is the same one I verified: the frame grep FAILs on the pre-#1041 binary (I built 0c862e64^ and the frame has no engine: segment at all) and PASSes on a head with #1041 (engine: llamacpp (gpu)), with status-rc=0 invented-urls=0 beside it.

@bong-water-water-bong
bong-water-water-bong merged commit 8207baa into main Nov 5, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the os/v3-c25-console-engine branch November 5, 2026 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant