[bong-water-water-bong]

Today's Sunday run (a rehearsal: no READY file yet) died at the 1bit OS package and never reported.

• Cause: os/mini/mkimage.sh calls modprobe, which lives in /usr/sbin. The weekly wrapper's PATH leaves that out, so the image's kernel-modules step failed on every try. Run from a login shell it passes, which is why yesterday's hand-run rehearsal worked. mkimage.sh now adds /usr/sbin:/sbin itself.
• Misleading log: guard() logged "memory guard stopped it" for any non-zero exit and retried 10 times in 2 seconds. It now retries only mem-guard's own kill (exit 137) and reports anything else as FAILED (exit N).
• Outside the repo: the wrapper on strixhalo (~/.local/bin/1bit-weekly) is fixed separately, backup kept. Its PATH gains /usr/sbin, and its failure branch no longer dies under set -e when grep finds no error line. That was why nobody was told about the failure.

Checked: mkimage.sh builds the image with the wrapper's PATH, and a rehearsal rerun is under way on strixhalo.

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit 1ca36f5

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

⚡ Recommended focus areas for review Possible Issue The mkimage.sh script is responsible for building the 1bit OS image. The PR adds a new wrapper to handle failures in the weekly build process. However, the guard in scripts/weekly.sh might not correctly handle failures when the memory guard exits with an error other than 137 (SIGKILL), which could lead to false negatives in reporting the failure. This is especially relevant if the memory guard's exit code is not consistently 137, causing the wrapper to report a failure incorrectly. Additionally, the addition of /usr/sbin:/sbin to the PATH in mkimage.sh ensures that modprobe can find the necessary kernel modules, which is crucial for the GPU and NPU operations. Without this, the image build might fail to load required modules, as seen in the rehearsal failures. The guard's current implementation retries 10 times, but if the memory guard fails to pass within those tries, the overall build fails, and the error might not be clearly communicated to users unless properly logged or reported. Ensuring that the wrapper correctly reports failures and provides meaningful logs is essential for a reliable build process. Repeated failures could indicate issues with memory, disk, or other dependencies, and users should be informed of these failures appropriately. The current setup assumes a login shell passes, which might not hold in cron jobs, hence the need for the wrapper to explicitly prepend necessary paths. It is also important to verify that the memory guard's exit code is consistently 137, as this triggers the retry mechanism. If the exit code varies, the guard might not correctly identify the failure condition, leading to potential misreporting of the build status. Clear logs and error messages are necessary to diagnose issues, especially when the build fails silently or reports an incorrect status. The PR also mentions fixing the wrapper for 1bit-weekly, which suggests that there might be additional issues with the build process that need to be addressed for reliable image creation. It is crucial to ensure that the build process is robust and provides accurate feedback to users, even in automated or scheduled environments like cron. This includes handling edge cases such as insufficient memory, disk full, or module loading failures, and reporting them explicitly rather than masking them as successful retries. export PATH="$PATH:/usr/sbin:/sbin" out=${1:?usage: mkimage.sh <out> [engine build dir]} Possible Issue The guard() function in scripts/weekly.sh retries 10 times, but it only handles exit code 137 (SIGKILL) as a failure that can be retried. If the mem-guard.sh exits with any other error code (e.g., 1, 2, or a process that has terminated abnormally), the guard will not retry and will report the failure. This could lead to the build appearing to succeed when it actually has not, especially if the memory guard fails after some retries but before the loop ends. For example, if the memory guard exits with 136 (SIGHUP) or 135 (SIGTERM), the guard will not retry and will log a failure, but the build might continue or fail later for other reasons. The current implementation assumes that only SIGKILL is a retryable error, but other signals or exit codes might also indicate a problem that prevents the build from completing successfully. This is particularly relevant in the weekly build process, where failures might occur in background tasks or after a certain time has passed, and the guard might prematurely conclude success if the memory guard exits with a non-137 code. It is essential to ensure that the guard correctly identifies all failure conditions and provides appropriate retries or error reporting to prevent undetected build failures. The build process relies on the memory guard to ensure sufficient memory is available, and any failure here should halt the build or clearly indicate the issue to the user. Without proper error handling for non-137 exit codes, users might assume the build succeeded when it has not, leading to potential missing dependencies or system issues. This could be exacerbated if the build is run in an environment where the memory guard's exit conditions are not consistently triggered, such as in a cron job with a different user context or resource constraints. Therefore, the guard should be adjusted to handle a broader range of failure codes or to explicitly report any non-zero exit codes as failures, rather than relying solely on exit code 137. room || return 1 rc=0; "$SRC/scripts/mem-guard.sh" 4 "$@" || rc=$? [ "$rc" = 0 ] && return 0 # mem-guard kills with SIGKILL (exit 137); any other status is the command's own failure, # which a retry cannot fix if [ "$rc" != 137 ]; then say "FAILED (exit $rc): $*"; return "$rc"; fi say "memory guard stopped it (try $try): $*" done return 1