Skip to content

release: cut-release.sh — reproducible release bundle from a clean tree (vms-d73) - #279

Merged
baron-3dl merged 2 commits into
mainfrom
worktree-agent-a30e7760796101156
Aug 10, 2026
Merged

baron-3dl merged 2 commits into
mainfrom
worktree-agent-a30e7760796101156

Conversation

@baron-3dl

Copy link
Copy Markdown
Contributor

Summary

Foundation of the Release Engineering pillar (epic vms-a84). Implements vms-d73.

  • tools/cut-release.sh: git-archives the requested commit to a clean /tmp tree (never builds into the repo, per the standing containerized-build rule), drives the existing distro/Dockerfile.bootable build via docker buildx build --output=local, and writes dist/release-<version>/ containing vmlinuz, initramfs-ovmx-slim.cpio.gz, ovmx-distrib.img, ovmx-os.kit, SHA256SUMS, and an OVMX-defined (Rule 8, explicitly labeled) release-manifest.json mapping component → version → sha256, including the OS kit's own internal file listing (asserts DCL.EXE/LOGINOUT.EXE/STARTUP.COM present, from the shipped kit itself).
  • Version is single-sourced from the archived src/libvms/include/ovmx_identity.h (OVMX_PRODUCT_NAME/OVMX_PRODUCT_VERSION) — no version literal in the script.
  • CMakeLists.txt: optional make release convenience target wrapping the script.

Reproducibility required normalizing every wall-clock timestamp the build stamps into shipped bytes:

  • distro/Dockerfile.bootable gains a SOURCE_DATE_EPOCH build-arg (default 0), threaded into every initramfs pack step (file mtimes normalized, cpio member order sorted, gzip -n instead of gzip).
  • tools/ovmx_kit_pack.c (kh_build_time) and tools/vmsfs_master.c (fh_created/fh_modified/fh_accessed) now read SOURCE_DATE_EPOCH instead of time(NULL) when set.
  • cut-release.sh defaults SOURCE_DATE_EPOCH to the cut commit's own timestamp, so two independent cuts of the same commit are deterministic automatically.

CI (.github/workflows/ci.yml, new cut-release-reproducible job, gated behind a new release path filter):

  1. Cuts the same commit twice, each with --no-cache (no shared build cache between them — proves reproducibility, not cache reuse).
  2. Diffs all four artifacts + SHA256SUMS byte-for-byte between the two cuts.
  3. Recomputes SHA256 against the shipped bytes and compares against release-manifest.json (defense against a bug in the script's own checksum step).
  4. Reuses tests/qemu/test_distrib_boot.sh unmodified — bind-mounts the freshly-cut bundle's own vmlinuz/initramfs-ovmx-slim.cpio.gz/ovmx-distrib.img at the paths it expects — to boot the actual cut bundle to a real SYSTEM login prompt, rather than reinventing the boot proof.

Verification so far

  • ctest (dev build): 123/124 pass. The 1 failure, env_identity_census_negctl, is a pre-existing, already-tracked dev-host-load timeout (vms-3f9) unrelated to any file this PR touches (it source-scans src/ for env-var census declarations) — reproduced the same timeout in isolation, confirming it's host contention, not a regression.
  • shellcheck tools/cut-release.sh: clean.
  • actionlint .github/workflows/ci.yml: no new findings (pre-existing warnings only, all outside the new job).
  • Dockerfile edits spot-checked by building the link-native stage locally; full Dockerfile parses and the touched builder-stage blocks were reviewed line-by-line for correct &&/\ chaining.
  • Manifest JSON generation and the CI job's grep/sed checksum-extraction logic dry-run tested against a synthetic fixture — valid JSON, correct extraction.
  • SOURCE_DATE_EPOCH override verified directly against ovmx_kit_pack: two packs with a fixed epoch are byte-identical; a pack without the override differs, confirming the override actually takes effect.

Not yet run: the actual containerized 2×-build-plus-boot proof — that only runs in CI (cut-release-reproducible), which this PR triggers. Two full from-scratch builds (~25–30 min cold each) plus the boot-smoke test make it the heaviest job in the workflow; watching CI now.

Test plan

  • ctest baseline (pre-existing flake noted, not caused by this change)
  • shellcheck / actionlint clean
  • CI cut-release-reproducible job passes (byte-identical artifacts across two independent --no-cache cuts + boot-to-login proof) — watching this before merge

🤖 Generated with Claude Code

alice and others added 2 commits August 10, 2026 14:53
…e from a clean tree, byte-reproducibly (vms-d73)

Foundation of the Release Engineering pillar (epic vms-a84): replaces the
hand-rolled per-cut process (build, copy artifacts, hand-write checksums)
with tools/cut-release.sh, which git-archives HEAD to a clean /tmp tree
(never builds into the repo), drives the existing containerized
distro/Dockerfile.bootable build via `docker buildx build --output=local`,
and writes dist/release-<version>/ with vmlinuz, initramfs-ovmx-slim.cpio.gz,
ovmx-distrib.img, ovmx-os.kit, SHA256SUMS, and an OVMX-defined (Rule 8)
release-manifest.json. Version is single-sourced from the archived
src/libvms/include/ovmx_identity.h -- no literal in the script.

Byte-reproducibility required normalizing every wall-clock the build stamps
into a shipped artifact: SOURCE_DATE_EPOCH now threads through
Dockerfile.bootable (new build-arg, default 0) into cpio/gzip packing
(mtime-normalized + sorted file lists + gzip -n) and into
tools/ovmx_kit_pack.c / tools/vmsfs_master.c, which previously stamped
time(NULL) into the OS kit header and the mastered VMSFS image's file
timestamps.

.github/workflows/ci.yml adds cut-release-reproducible: cuts the same
commit twice with --no-cache, diffs all four artifacts + SHA256SUMS
byte-for-byte, recomputes checksums against the emitted manifest, and
reuses tests/qemu/test_distrib_boot.sh (unmodified) to boot the freshly-cut
bundle to a real SYSTEM login. Gated behind a new `release` path filter.

CMakeLists.txt adds an optional `make release` convenience target wrapping
the script.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI's cut-release-reproducible job caught a real byte diff in
initramfs-ovmx-slim.cpio.gz between two independent --no-cache cuts of the
same commit (vmlinuz, ovmx-distrib.img, ovmx-os.kit were all already
byte-identical).

Root-caused by direct measurement, not guessing: isolated reproducers for
both the kernel modules (vms.ko/vmsfs.ko via kbuild) and the static musl
binaries (STARTUP.EXE/INITIALIZE.EXE/vms_mount_helper) proved BYTE-IDENTICAL
across two independent --no-cache container builds -- ruling out the
"kernel modules embed build metadata" hypothesis. The actual cause: GNU
cpio's "newc" format embeds the REAL FILESYSTEM INODE NUMBER of every entry,
which is not deterministic across independent container builds even when
every file's content and mtime are identical (verified directly with two
byte-identical directory trees built independently on the same host).

Fix: add --reproducible to every `cpio -o -H newc` invocation in
distro/Dockerfile.bootable (fat/slim/noexec/nodev initramfs packing) -- GNU
cpio's own name for --ignore-devno --ignore-dirnlink --renumber-inodes.
Verified locally: two independently-created, byte-identical directory trees
now produce byte-identical cpio archives with this flag; without it they
differ at the same header offset (the inode field) the CI failure pointed
to.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@baron-3dl
baron-3dl force-pushed the worktree-agent-a30e7760796101156 branch from 3a30d51 to 677df2a Compare August 10, 2026 14:53
@baron-3dl
baron-3dl merged commit 5f6c9d2 into main Aug 10, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant