Skip to content

os: write the onebit manifest the check reads, report skip reasons honestly - #579

Merged
bong-water-water-bong merged 3 commits into
mainfrom
1bit/os-manifest-secureboot
Oct 26, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
1bit/os-manifest-secureboot

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Three defects that a hardware run of candidate 5 exposed. All three are in the image's own tooling, not in the release itself — the image booted signed with Secure Boot enabled and our MOK, and passed 14/16 rows on the board.

1. models could never pass: nothing wrote /etc/1bit/onebit-models.txt

release-check.sh:233 reads the manifest and, when it is missing, fails with "the build writes it from server_models.json". Nothing in the tree ever wrote it — a grep for onebit-models across os/ returned exactly one hit, the consumer. So every image failed models on hardware, and chat-hrx failed one row later because no onebit entry was registered to answer.

mkimage.sh now derives the manifest from the Lemonade catalogue it already stages at /opt/lemonade/bin/resources/server_models.json, in exactly the format release-check.sh parses:

mode=live
hrx Qwen3-0.6B-Q4_K_M-1bit
npu Qwen3-4B-npu-1bit

Derived rather than hand-kept, so it cannot drift from what Lemonade actually ships. A catalogue with no onebit entries writes a manifest with none — reported as such by the check, rather than failing. The walk accepts both a list of model dicts and a dict of sections, so a catalogue layout change produces a manifest instead of silently producing an empty one.

2. chat-hrx / chat-npu blamed the environment for a missing entry

Both rows said "no AMD GPU in this VM" whenever their model id was absent — including on a machine with a working GPU, where the real cause was that Lemonade listed no -1bit entry. A wrong skip reason is how a real failure gets filed as an environment limitation. Both now distinguish no device (/dev/kfd, /dev/accel/accel*) from no matching entry listed.

3. secureboot skipped on a machine that had it enabled

The row used mokutil, which is not shipped in the image, so it always fell through to a skip — on a board that reported Secure Boot enabled in dmesg and had just booted our MOK-signed kernel. That is the row the owner cares about most for a permanent install, and it was hiding a pass.

It now reads the firmware's own variable, SecureBoot-8be4df61-93ca-11d2-aa0d-00e098032b8c, byte 5 of 5. No new binary:

byte 5 row
1 PASS Secure Boot enabled; the MOK-signed kernel booted
0 PASS Secure Boot disabled (the firmware reports it off)
unreadable SKIP

Testing

  • os/mini/release-check.sh --self-test → PASS, all 7 verdict cases.
  • bash -n clean on both scripts.
  • The manifest producer was exercised against a synthetic catalogue: produces mode=live, 2 × hrx, 1 × npu, and the parser's grep -c '^hrx ' / sed 's/^mode=//' read them as intended.
  • The efivar parse was checked with synthesised 5-byte variables (1 → enabled, 0 → disabled).

Not in this PR: the chat-hrx "did not answer" failure is a separate, real defect — the engine execs hrx/llama/bin/llama-server as a relative path, while the image ships llama-server at /opt/1bit-llama/llama-server (lemond.log: "Failed to execute: hrx/llama/bin/llama-server"). That is engine server-path resolution and gets its own change.

…nestly

Three defects a hardware run of candidate 5 exposed, all in the image's own tooling rather than in
the release itself.

1. `/etc/1bit/onebit-models.txt` had no producer. release-check.sh read it and, when it was absent,
   failed with "the build writes it from server_models.json" - but nothing in the tree ever wrote it,
   so the `models` row could never pass on any image. mkimage.sh now derives it from the Lemonade
   catalogue it already stages at /opt/lemonade/bin/resources/server_models.json, in the format
   release-check parses: `mode=<live|installed>` then one `hrx <id>` / `npu <id>` per entry. Deriving
   it means it cannot drift from what Lemonade actually ships; a catalogue with no onebit entries
   writes a manifest with none, which the check reports as such instead of failing.
   The JSON walk accepts both a list of model dicts and a dict of sections, so a layout change
   produces a manifest rather than silently producing an empty one.

2. The `chat-hrx` and `chat-npu` skips blamed the environment for what is often a missing entry.
   Both now distinguish the two cases: no /dev/kfd (or /dev/accel/accel*) means the machine cannot run
   the entry, while a present device with no matching model id means Lemonade listed nothing to ask.
   On the hardware run this mattered: the row said "no AMD GPU in this VM" on a machine with a GPU.

3. The `secureboot` row skipped whenever mokutil was absent, though mokutil is not shipped - so a
   machine that had Secure Boot ON and had just booted our MOK-signed kernel was reported as unknown.
   It now reads the firmware's own SecureBoot EFI variable (byte 5 of SecureBoot-8be4df61-...), which
   needs no extra binary: 1 passes "Secure Boot enabled; the MOK-signed kernel booted", 0 passes on a
   firmware that reports it off, and only an unreadable variable stays a skip.

self-test passes (7/7 verdict cases). Apache-2.0 headers untouched.
bong-water-water-bong added 2 commits October 25, 2026 23:26
Candidate 5 shipped an HRX runtime from before the pin that carries HRX_ALLOW_NO_SVM. The engine sets
HSA_USE_SVM=0 on a CWSR-on kernel and needs that gate to still see the device, so the shipped runtime
registered no usable HRX0 - while every other release-check row stayed green. chat-hrx then failed with
"an entry did not answer", which says nothing about the runtime underneath it.

This row asks the smallest question with a yes/no answer: on a machine with /dev/kfd, run
`llama-server --list-devices` from the engine's installed location, with the same libhsa the engine
resolves (/etc/1bit/libhsa), and require an HRX0 line.

  PASS hrx-device "HRX0: <name>"
  FAIL hrx-device "no HRX0 listed: ..." (quoting the runtime's own INCOMPATIBLE line when it says one)
  SKIP hrx-device when there is no /dev/kfd

An HRX0 that registered but reports INCOMPATIBLE also prints "HRX0: INCOMPATIBLE (...)", so that is
rejected before a match is accepted - otherwise the row would pass on the exact failure it exists to
catch (cp. the old lemonade row, which false-PASSed on an empty body).

The row runs before chat-hrx, and after the manifest row it belongs with: models says the entries were
registered, hrx-device says the device they run on exists, chat-hrx says one of them answered.

Checked against three synthetic --list-devices outputs: a healthy HRX0 passes, a CPU-only list fails,
and "HRX0: INCOMPATIBLE (only systems with SVM are supported)" fails. self-test still PASS.
The row exported HSA_RUNTIME_LIBRARY_PATH, which nothing reads. The runtime resolves libhsa through
IREE_HAL_AMDGPU_LIBHSA_PATH (iree/hal/drivers/amdgpu/util/libhsa.c) - the only libhsa knob in
libggml-hrx/libhrx - and init.sh exports it from /etc/1bit/libhsa. A row that sets a variable nobody
reads would have tested whatever libhsa the loader found first, which is the confusion this row exists
to remove.

Now: inherit IREE_HAL_AMDGPU_LIBHSA_PATH when it is already set (so the check tests exactly what the
engine will), and otherwise seed it from /etc/1bit/libhsa, which holds an exact library path - the
variable accepts either that or a directory, so it is passed through unchanged.

self-test PASS; the three synthetic --list-devices cases still behave (healthy HRX0 passes, CPU-only
fails, "HRX0: INCOMPATIBLE" fails).
@bong-water-water-bong
bong-water-water-bong merged commit 1ea01f1 into main Oct 26, 2026
8 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/os-manifest-secureboot branch October 26, 2026 02:31
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Manifest Generation Logic

The new logic for generating the /etc/1bit/onebit-models.txt manifest from server_models.json correctly handles both list and dict formats of the catalogue, but does not validate that the id field is present and non-empty for each model. This could lead to empty or malformed entries in the manifest if the catalogue contains invalid model entries.

if "id" in node and isinstance(node.get("id"), str):
    yield node
Secure Boot Check Improvement

The updated secure boot check now reads the EFI variable directly to determine the Secure Boot state, which is a more accurate method than relying on mokutil. However, the check assumes that the EFI variable path is consistent across all UEFI implementations, which may not be true in all cases.

SBVAR=/sys/firmware/efi/efivars/SecureBoot-8be4df61-93ca-11d2-aa0d-00e098032b8c
if [ -r "$SBVAR" ]; then
    sbv=$(od -An -tu1 -j4 -N1 "$SBVAR" 2>/dev/null | tr -d ' \n')
    case "${sbv:-}" in
        1) pass secureboot "Secure Boot enabled; the MOK-signed kernel booted";;
        0) pass secureboot "Secure Boot disabled (the firmware reports it off)";;
        *) skip secureboot "Secure Boot state unreadable from $SBVAR";;
    esac
HRX Device Check Logic

The new hrx-device check introduces a more precise way to determine if an HRX device is registered by parsing the output of llama-server --list-devices. However, the check may fail to detect a device if the output format changes or if the device is registered but not listed in the expected format.

HRXDEV=$(printf '%s\n' "$LD" | sed -n 's/^[[:space:]]*\(HRX[0-9][0-9]*\): *\(.*\)$/\1: \2/p' | grep -vi 'INCOMPATIBLE' | head -1)
if [ -n "$HRXDEV" ]; then
    pass hrx-device "$HRXDEV"

bong-water-water-bong pushed a commit that referenced this pull request Oct 26, 2026
…gs, install, shell)

First hardware boot of candidate 5 left the console unusable: kernel lines over a bare "# " prompt
and nothing to do with it. This is what the owner asked for instead - a dialog-style full-screen
menu - as one POSIX sh script, os/mini/1bit-menu, with no new binary in the image (there is no
dialog, whiptail or ncurses on the stick, and terminfo is not guaranteed on a text console, so it
draws with ANSI escapes in the 16-colour set).

tty1 only. init.sh starts the menu in place of the bare `sh -i` loop; ttyS0 (the serial line the
QEMU gate drives) and ssh keep exactly the plain shell they have today, so no prompt matcher on the
gate changes, and the gate builds nothing new: the same init.sh reaches the same serial shell.

The menu shows the banner facts it was previously printed as text (edition, LIVE/INSTALLED, address,
Lemonade URL, where the API key is) and drives only commands the appliance already has - it starts
neither the engine nor Lemonade, which are up by the time init gets here:

  Chat     pick a model from the Local models `lemonade list` reports (the onebit ones marked) and
           run `lemonade chat <id>` in the terminal; back on exit
  Models   `lemonade list` and the GGUFs on /data/models, paged by a built-in pager
  Status   the tail of /data/selftest.log and the PASS/FAIL/SKIP rows of the LATEST
           /data/release-check.log run (each run is delimited), refreshing once a second; r runs
           `1bit-release-check` again and shows its output
  Logs     tail -40 of /tmp/lemond.log, /data/boot.log and /tmp/fans.log, any key refreshes
  Install  the internal NVMe with its size, model and partition table, plus 1bit-install's own
           refusal text (run without --yes, so the warning cannot drift from the tool); typing
           INSTALL is required, then `1bit-install /dev/nvme0n1 --yes`, then an offer to reboot.
           When the machine is installed it shows what it runs from instead
  Shell    `sh -i` with PS1 "1bit-os:\w # "; exit comes back to the menu
  Power    reboot or power off, confirmed with y

Keys: up/down or j/k, 1-7 jump, Enter chooses, q quits to the plain shell. `1bit-menu --shell`, or
holding q as it starts, skips the menu entirely.

The console can never be swallowed. The menu restores the terminal on EXIT, INT, TERM and HUP, and
init runs `stty sane` on tty1 before every restart (an addition asked for after review: a crash must
not leave raw mode behind). Any exit other than 0 restarts the menu; q means "give me the shell" and
drops to it. Two details found while proving this:

  * init has to run that loop in the foreground, not as `prog &`: POSIX has an asynchronous list
    ignore SIGINT and SIGQUIT, and the child inherits it - so a menu started with & could not be
    interrupted, and neither could the lemonade chat or the shell it runs. init has nothing else to
    do once the appliance is up, and both loops never return.
  * key reads wait in 0.5 s slices instead of blocking, so a menu killed with SIGKILL cannot leave a
    dd blocked in a read; a straggler holds tty1's session and then nothing can take the tty again.

Rebased onto baf5996 (#583): main's #582 (ignore_loglevel/printk off before the console, and
PS1="1bit-os:\w # " on tty1, ttyS0 and ssh through /etc/profile), #580 (ONEBIT_HRX_LLAMA_SERVER,
the libhsa preference), #583 (explicit HSA staging) and #579 (the onebit manifest) are all kept. The
menu replaces only the tty1 shell loop; the plain-shell fallback below it keeps #582's PS1, and the
menu's Shell entry sets the same one. The printk lines now pay off twice: a boot line painted over the
frame is a mess the menu can only clean up on the next keypress.

Proof, on ryzen, no GPU and no strixhalo reboot: bash -n / sh -n / dash -n clean; run in an 80x25
tmux pane with TERM=linux and a stub `lemonade` (plus stub 1bit-install and 1bit-release-check) on
PATH - it draws, navigates, and runs Chat, Models, Status (including re-running the check), Logs,
Install and Shell, and comes back from each; q and Ctrl-C both leave the terminal with icanon and
echo back. The init loop itself was driven on a pty nothing else owns: a SIGKILLed menu leaves raw
mode behind, the loop's `stty sane` restores it and the menu comes back, Ctrl-C is restarted by the
loop, and q falls through to the plain shell.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bong-water-water-bong added a commit that referenced this pull request Oct 26, 2026
…gs, install, shell) (#584)

First hardware boot of candidate 5 left the console unusable: kernel lines over a bare "# " prompt
and nothing to do with it. This is what the owner asked for instead - a dialog-style full-screen
menu - as one POSIX sh script, os/mini/1bit-menu, with no new binary in the image (there is no
dialog, whiptail or ncurses on the stick, and terminfo is not guaranteed on a text console, so it
draws with ANSI escapes in the 16-colour set).

tty1 only. init.sh starts the menu in place of the bare `sh -i` loop; ttyS0 (the serial line the
QEMU gate drives) and ssh keep exactly the plain shell they have today, so no prompt matcher on the
gate changes, and the gate builds nothing new: the same init.sh reaches the same serial shell.

The menu shows the banner facts it was previously printed as text (edition, LIVE/INSTALLED, address,
Lemonade URL, where the API key is) and drives only commands the appliance already has - it starts
neither the engine nor Lemonade, which are up by the time init gets here:

  Chat     pick a model from the Local models `lemonade list` reports (the onebit ones marked) and
           run `lemonade chat <id>` in the terminal; back on exit
  Models   `lemonade list` and the GGUFs on /data/models, paged by a built-in pager
  Status   the tail of /data/selftest.log and the PASS/FAIL/SKIP rows of the LATEST
           /data/release-check.log run (each run is delimited), refreshing once a second; r runs
           `1bit-release-check` again and shows its output
  Logs     tail -40 of /tmp/lemond.log, /data/boot.log and /tmp/fans.log, any key refreshes
  Install  the internal NVMe with its size, model and partition table, plus 1bit-install's own
           refusal text (run without --yes, so the warning cannot drift from the tool); typing
           INSTALL is required, then `1bit-install /dev/nvme0n1 --yes`, then an offer to reboot.
           When the machine is installed it shows what it runs from instead
  Shell    `sh -i` with PS1 "1bit-os:\w # "; exit comes back to the menu
  Power    reboot or power off, confirmed with y

Keys: up/down or j/k, 1-7 jump, Enter chooses, q quits to the plain shell. `1bit-menu --shell`, or
holding q as it starts, skips the menu entirely.

The console can never be swallowed. The menu restores the terminal on EXIT, INT, TERM and HUP, and
init runs `stty sane` on tty1 before every restart (an addition asked for after review: a crash must
not leave raw mode behind). Any exit other than 0 restarts the menu; q means "give me the shell" and
drops to it. Two details found while proving this:

  * init has to run that loop in the foreground, not as `prog &`: POSIX has an asynchronous list
    ignore SIGINT and SIGQUIT, and the child inherits it - so a menu started with & could not be
    interrupted, and neither could the lemonade chat or the shell it runs. init has nothing else to
    do once the appliance is up, and both loops never return.
  * key reads wait in 0.5 s slices instead of blocking, so a menu killed with SIGKILL cannot leave a
    dd blocked in a read; a straggler holds tty1's session and then nothing can take the tty again.

Rebased onto baf5996 (#583): main's #582 (ignore_loglevel/printk off before the console, and
PS1="1bit-os:\w # " on tty1, ttyS0 and ssh through /etc/profile), #580 (ONEBIT_HRX_LLAMA_SERVER,
the libhsa preference), #583 (explicit HSA staging) and #579 (the onebit manifest) are all kept. The
menu replaces only the tty1 shell loop; the plain-shell fallback below it keeps #582's PS1, and the
menu's Shell entry sets the same one. The printk lines now pay off twice: a boot line painted over the
frame is a mess the menu can only clean up on the next keypress.

Proof, on ryzen, no GPU and no strixhalo reboot: bash -n / sh -n / dash -n clean; run in an 80x25
tmux pane with TERM=linux and a stub `lemonade` (plus stub 1bit-install and 1bit-release-check) on
PATH - it draws, navigates, and runs Chat, Models, Status (including re-running the check), Logs,
Install and Shell, and comes back from each; q and Ctrl-C both leave the terminal with icanon and
echo back. The init loop itself was driven on a pty nothing else owns: a SIGKILLed menu leaves raw
mode behind, the loop's `stty sane` restores it and the menu comes back, Ctrl-C is restarted by the
loop, and q falls through to the plain shell.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant