vms-6a2: give the corpus conformance gate real teeth (floor 90/229, vacuity guard, path fix) - #492
Merged
Merged
Conversation
The corpus gate was documented as falsifiable but had gaps the item flagged: a total=0 baseline would make it vacuous, run_corpus.sh hardcoded /src paths (only runnable inside the CI container), and the run-pass count was flaky at the boundary so any floor set to the high value would redden CI intermittently. Changes: - run_corpus.sh: derive REPO_ROOT from SCRIPT_DIR (= /src in CI, unchanged) instead of hardcoding /src for includes, -L, and LD_LIBRARY_PATH, with an OVMX_BUILD_LIB_DIR override. The harness now runs from any checkout/worktree. - run_corpus.sh: redirect program stdin from /dev/null. The input routines (lib$get_input/get_command/get_foreign/lookup_key) flapped run-pass<->run-fail on an inherited tty, making the run-pass count non-deterministic (90 vs 86). /dev/null = the closed stdin CI already gives, so the measurement (and thus the floor) is honest and reproducible: 90 run-pass, stable over 5 runs. - run_corpus.sh: hard-fail (exit 2) when total==0 — a zero-total run is never a conformance pass; that vacuity is the exact defect this item exists to kill. - ci.yml: reject total==0 on the report OR the committed baseline in the floor step (independent backstop), and document the vacuity guard. - corpus_baseline.json: real measurement against current HEAD libs, floor ratcheted 89 -> 90 run-pass / 229 total (compile-fail 66, link-fail 52, run-fail 7, run-crash 14). Timestamped, per-program status. Proven teeth (all foreground, exit codes read directly): - current state PASSES (run_corpus exit 0, floor held 90>=90); - per-entry regression reddens run_corpus.sh (exit 1); - aggregate floor reddens the CI step (current 90 < simulated floor 91); - total==0 hard-errors both run_corpus.sh (exit 2) and the CI floor step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl
force-pushed
the
vms-6a2-corpus-baseline
branch
from
August 13, 2026 19:46
00d3564 to
8d0bfa3
Compare
baron-3dl
pushed a commit
that referenced
this pull request
Aug 13, 2026
…for MMK's clean rundown (no leaked executive resources)
test_syssvc_mmk_drive gated on MMK.EXE reaping within a tight grace
("MMK.EXE completed ... rather than deadlocking in $HIBER"). Under
contended TCG (the Kernel Executive job runs ~76 suites in one QEMU VM
near a 120s wall) MMK echoes the DCL-computed marker OVMXB23:42 and drives
the build correctly, but its teardown-and-exit lags the marker → the reap
assertion reddens → the whole Kernel Executive gate goes red on unrelated
PRs (#499, #492).
Fix, keying on the PROOF like spine #6 (test_syssvc_mmk_build, vms-d1b):
success is the marker echoed back over the mailbox drive (the DCL-computed
independent oracle), NOT MMK's self-exit timing. A genuine mid-drive
$HIBER deadlock echoes no marker → got_marker stays 0 → the bound elapses
→ fails HARD. The negctl (facility_defects.sh mmk-drive-command-not-sent,
zero command bytes → wedged DCL → no marker) still reddens via the
require_fail assertion; its knock_on_fail (the removed reap assertion) is
cleared so the observed red set still equals require_fail + knock_on_fail
exactly.
Round 2 (teardown hygiene against the REAL shared executive, Rule 9):
success is keyed on the marker, but the parent STILL WAITS (bounded,
unasserted) for MMK to exit on its OWN. MMK $CREMBXed its command + result
mailboxes, armed a write-attention AST, and lib$spawned a persistent DCL
subprocess, and it tears all of that down in close_subprocess()/sp_close()
($FORCEX + $DELPRC the DCL, $DASSGN both mailboxes) as its last act before
exiting. SIGKILLing MMK the instant the marker appears skipped that
teardown, orphaning the DCL grandchild (which held this suite's stdout
FIFO open and sat blocked on the leaked mailbox) and leaking the
mailboxes/AST into the executive the next suite shares — deterministically
wedging the suite AFTER this one (test_syssvc_procnam), stalling the whole
run at ~60/76 at any wall. Now the parent polls for the marker while
draining MMK's output until MMK exits on its own, letting sp_close()
reclaim everything; SIGKILL is a last resort reached only by a genuine
wedge, which occurs only under the isolated one-defect-per-boot negctl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl
added a commit
that referenced
this pull request
Aug 13, 2026
…for MMK's clean rundown (no leaked executive resources) (#502) test_syssvc_mmk_drive gated on MMK.EXE reaping within a tight grace ("MMK.EXE completed ... rather than deadlocking in $HIBER"). Under contended TCG (the Kernel Executive job runs ~76 suites in one QEMU VM near a 120s wall) MMK echoes the DCL-computed marker OVMXB23:42 and drives the build correctly, but its teardown-and-exit lags the marker → the reap assertion reddens → the whole Kernel Executive gate goes red on unrelated PRs (#499, #492). Fix, keying on the PROOF like spine #6 (test_syssvc_mmk_build, vms-d1b): success is the marker echoed back over the mailbox drive (the DCL-computed independent oracle), NOT MMK's self-exit timing. A genuine mid-drive $HIBER deadlock echoes no marker → got_marker stays 0 → the bound elapses → fails HARD. The negctl (facility_defects.sh mmk-drive-command-not-sent, zero command bytes → wedged DCL → no marker) still reddens via the require_fail assertion; its knock_on_fail (the removed reap assertion) is cleared so the observed red set still equals require_fail + knock_on_fail exactly. Round 2 (teardown hygiene against the REAL shared executive, Rule 9): success is keyed on the marker, but the parent STILL WAITS (bounded, unasserted) for MMK to exit on its OWN. MMK $CREMBXed its command + result mailboxes, armed a write-attention AST, and lib$spawned a persistent DCL subprocess, and it tears all of that down in close_subprocess()/sp_close() ($FORCEX + $DELPRC the DCL, $DASSGN both mailboxes) as its last act before exiting. SIGKILLing MMK the instant the marker appears skipped that teardown, orphaning the DCL grandchild (which held this suite's stdout FIFO open and sat blocked on the leaked mailbox) and leaking the mailboxes/AST into the executive the next suite shares — deterministically wedging the suite AFTER this one (test_syssvc_procnam), stalling the whole run at ~60/76 at any wall. Now the parent polls for the marker while draining MMK's output until MMK exits on its own, letting sp_close() reclaim everything; SIGKILL is a last resort reached only by a genuine wedge, which occurs only under the isolated one-defect-per-boot negctl. Co-authored-by: alice <alice@workspace.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem (vms-6a2)
The corpus conformance gate was documented as a falsifiable floor but had real gaps:
total=0baseline/report makes the whole gate exit 0 forever (an empty floor can never be undercut). This is the original toothless-gate defect the item was filed on; nothing mechanically prevented the baseline reverting tototal=0.run_corpus.shhardcoded/src/...for includes,-L, andLD_LIBRARY_PATH, so it only ran correctly inside the CI docker mount, not from a plain host checkout/worktree.What changed
tests/conformance/run_corpus.shREPO_ROOTfromSCRIPT_DIR(= /srcin CI, unchanged) instead of hardcoding/src; addsOVMX_BUILD_LIB_DIRoverride. Runs from any checkout/worktree now./dev/null. The input routines (lib$get_input/get_command/get_foreign/lookup_key) flapped run-pass↔run-fail on an inherited tty./dev/null= the closed stdin CI already gives, so the measurement is deterministic (root-cause fix, not a retry/timeout band-aid).exit 2) whentotal==0..github/workflows/ci.ymltotal==0on the report or the committed baseline (independent backstop to run_corpus.sh's exit 2).tests/conformance/corpus_baseline.jsonReal measured numbers (current HEAD, x86_64 workshop)
Note for the record: the closed
vms-801.4"80%+ compile and run" claim is not backed by the tree — real run-pass is 39%, compile-or-better 48%. The committed baseline now carries the honest number either way.Proof the gate has teeth (all foreground, exit codes read directly)
lib_attach)total==0(report)total==0(baseline)Runs clean from a fresh worktree checkout (proved here — the worktree is not at
/src), and CI behavior is unchanged sinceREPO_ROOTresolves to/srcthere.🤖 Generated with Claude Code