vms-562: mmk_drive — key completion on the DCL-computed marker, not MMK self-exit timing - #502
Merged
Merged
Conversation
…for MMK's clean rundown (no leaked executive resources)
test_syssvc_mmk_drive gated on MMK.EXE reaping within a tight grace
("MMK.EXE completed ... rather than deadlocking in $HIBER"). Under
contended TCG (the Kernel Executive job runs ~76 suites in one QEMU VM
near a 120s wall) MMK echoes the DCL-computed marker OVMXB23:42 and drives
the build correctly, but its teardown-and-exit lags the marker → the reap
assertion reddens → the whole Kernel Executive gate goes red on unrelated
PRs (#499, #492).
Fix, keying on the PROOF like spine #6 (test_syssvc_mmk_build, vms-d1b):
success is the marker echoed back over the mailbox drive (the DCL-computed
independent oracle), NOT MMK's self-exit timing. A genuine mid-drive
$HIBER deadlock echoes no marker → got_marker stays 0 → the bound elapses
→ fails HARD. The negctl (facility_defects.sh mmk-drive-command-not-sent,
zero command bytes → wedged DCL → no marker) still reddens via the
require_fail assertion; its knock_on_fail (the removed reap assertion) is
cleared so the observed red set still equals require_fail + knock_on_fail
exactly.
Round 2 (teardown hygiene against the REAL shared executive, Rule 9):
success is keyed on the marker, but the parent STILL WAITS (bounded,
unasserted) for MMK to exit on its OWN. MMK $CREMBXed its command + result
mailboxes, armed a write-attention AST, and lib$spawned a persistent DCL
subprocess, and it tears all of that down in close_subprocess()/sp_close()
($FORCEX + $DELPRC the DCL, $DASSGN both mailboxes) as its last act before
exiting. SIGKILLing MMK the instant the marker appears skipped that
teardown, orphaning the DCL grandchild (which held this suite's stdout
FIFO open and sat blocked on the leaked mailbox) and leaking the
mailboxes/AST into the executive the next suite shares — deterministically
wedging the suite AFTER this one (test_syssvc_procnam), stalling the whole
run at ~60/76 at any wall. Now the parent polls for the marker while
draining MMK's output until MMK exits on its own, letting sp_close()
reclaim everything; SIGKILL is a last resort reached only by a genuine
wedge, which occurs only under the isolated one-defect-per-boot negctl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl
force-pushed
the
vms-562-mmk-drive-flake
branch
from
August 13, 2026 22:48
5efcc92 to
414632a
Compare
baron-3dl
added a commit
that referenced
this pull request
Aug 15, 2026
…es to TCG variance) (#584) The single Kernel Executive job booted ONE QEMU VM running all ~79 executive suites sequentially under a fixed 600s whole-VM wall. A pathologically slow (~10x) GitHub TCG runner reached only 60/79 in 600s and WALLED, reddening proven work non-deterministically (#502 failed 3x at escalating walls). A fixed wall buys only linear headroom against unbounded runner variance, so it cannot fix the flake; raising it again is rejected (Rule 8). Partition the suites into 6 shards, each a separate CI matrix leg booting its own QEMU VM over ~1/6 of the suites (~11-16/shard, measured) under the SAME unchanged 600s wall -- comfortably covered even at 10x-slow. The assignment is by name-hash: shard(name) = (first 6 hex of md5(name)) mod N, computed identically in tests/qemu/init.sh (busybox ash) and in the ci.yml verification (GNU bash) -- proven byte-identical for all 79 suites. It is a residue-class partition: every suite lands in exactly one shard, the union is the whole set, no gap and no overlap. No suite dropped; coverage unchanged, only spread. - tests/qemu/run_tests.sh: read SHARD_INDEX/SHARD_TOTAL env (default 0/1 = run everything, so the negative-control image and bare `docker run` are unaffected), forward them onto the guest kernel cmdline as ovmx.shard/ovmx.shards. Wall unchanged at 600s. - tests/qemu/init.sh: parse ovmx.shard/ovmx.shards from /proc/cmdline and run only this VM's residue class. Malformed/empty falls back to run-all. - .github/workflows/ci.yml: the positive KE job becomes `kernel-executive-shard` (6-way matrix), each leg asserting only its assigned suites' verdict lines + both module-load lines + `0 suites failed`. A NEW aggregate job `kernel-executive` KEEPS THE EXACT ORIGINAL required-check name 'Kernel Executive (vms.ko via /dev/vms, QEMU)', needs all shards, and is green iff every shard is green (if: always() so a failed shard reddens it rather than skipping it green). The aggregate also runs a union/coverage proof: every suite maps to exactly one shard, sum over shards == total. Preserves the gate's teeth: the negative-control job (job 3c, unsharded) still runs the full set with the executive absent and must red; a dropped/failed suite in shard K reds shard K -> reds the aggregate. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tests/qemu/test_syssvc_mmk_drive(self-host spine #4) asserted MMK.EXE reaped within a tight grace:Under contended TCG (the Kernel Executive job runs ~76 suites in ONE QEMU VM near a 120s wall), MMK echoes the DCL-computed marker
OVMXB23:42and drives the build correctly, but its teardown-and-exit lags the marker — so the reap assertion reddens and the whole Kernel Executive (vms.ko via /dev/vms, QEMU) gate goes red on unrelated PRs (bit #499, #492).Fix — key success on the marker, but wait for MMK's clean rundown
Like spine #6 (
test_syssvc_mmk_build, vms-d1b), success is keyed on the proof captured — the DCL-computed markerOVMXB23:42echoed back over the mailbox drive (the independent oracle) — not on MMK's self-exit timing.DRIVE_TIMEOUT_MSunchanged (25s); no timeout bump.Teardown hygiene against the REAL shared executive (Rule 9): the parent still waits (bounded, unasserted) for MMK to exit on its own. MMK
$CREMBXed its command + result mailboxes, armed a write-attention AST, andlib$spawned a persistent DCL subprocess, and tears all of that down inclose_subprocess()/sp_close()($FORCEX+$DELPRCthe DCL,$DASSGNboth mailboxes) as its last act before exiting. SIGKILLing MMK the instant the marker appears skipped that teardown — orphaning the DCL grandchild (which held this suite's stdout FIFO open and sat blocked on the leaked mailbox) and leaking the mailboxes/AST into the executive the next suite shares — deterministically wedgingtest_syssvc_procnamand stalling the run at ~60/76 at any wall. Now the parent polls for the marker while draining MMK's output until MMK exits on its own, lettingsp_close()reclaim everything; SIGKILL is a last resort reached only by a genuine wedge, which occurs only under the isolated one-defect-per-boot negctl.Discriminating power preserved
A genuine mid-drive
\$HIBERdeadlock echoes no marker →got_markerstays 0 → the bound elapses → fails hard. Negctlfacility_defects.sh mmk-drive-command-not-sent(zero command bytes → wedged DCL → no marker) still reddens via the untouchedrequire_failassertion; itsknock_on_fail(the removed reap assertion) is cleared so the observed red set equalsrequire_fail + knock_on_failexactly (manifest selftest passes). No-executive honest-skip branch unchanged (Rule 9 / INV-6).Proof
Built
tests/qemu/Dockerfilefrom a cleangit archive HEAD(414632aa) and ran the FULL Kernel Executive suite under pure TCG:76/76 — the run reaches
test_syssvc_procnamand everything aftermmk_drive(the old teardown-leak stall at ~60/76 is gone), with 0 failed assertions. CI green-by-SHA on this commit for Build & Test + Kernel Executive.🤖 Generated with Claude Code