vms-feb: executive-resident async AST delivery — $HIBER interrupted by a queued AST (cross-process) - #457
Merged
Conversation
…d by a queued AST, cross-process (unblocks MMK send_cmd_and_wait) A queued AST was drained only on an explicit sys$setast(1) and sys$hiber was a bare pause(), so a process that armed a write-attention AST on a mailbox and then $HIBERed waiting for it was NEVER woken when another process wrote the mailbox — a deadlock, and exactly the $HIBER/$WAKE + write-attention pattern MMK's send_cmd_and_wait uses (spine #4, vms-b23). Make the wait, the wake state and the AST-arrival notification executive-resident (Rule 9 / INV-6 — cross-process, through /dev/vms), keeping AST DISPATCH in userspace (the routine address is only valid in the target process): - VMS_IOCTL_HIBER (vms_ioctl_hiber): block until a $WAKE is pending OR an AST is deliverable at/below the caller's current mode (vms_ast_has_deliverable, same bound as DELIVERAST). Returns woken=1 iff released by a $WAKE (consuming a sticky wake bit), 0 iff by an AST. A bare signal does not end $HIBER. - VMS_IOCTL_WAKE (vms_ioctl_wake): set the sticky wake_pending bit on the target (self when vms_pid==0 — the MMK self-wake; else a VMS PID gated by GROUP/WORLD) and wake it. Replaces the old raw-linux-pid SIGCONT shim. - vms_ast_notify_arrival (vms_ast.c): every AST-enqueue path ($DCLAST, mailbox write-attention in vms_mbx.c, lock completion/blocking ASTs in vms_lock.c) now broadcasts the target's hiber_wq after queuing, under hiber_lock, so a $HIBER waiter wakes to drain. Lost-wakeup-free; no lock-order inversion. - sys$hiber loops vms_kif_hiber() + vms$$deliver_pending_asts(), returning only on a $WAKE (its own, or one an AST issued) — VMS re-hibernates after an AST that does not $WAKE. sys$wake targets a VMS PID (self when NULL). New universals vms_kif_hiber/vms_kif_wake appended to the frozen symbol vector (append-only). Clean-room provenance + design in docs/design-async-ast-delivery-ovmx.md (VSI System Services $HIBER/$WAKE + Programming Concepts AST semantics; OVMX choices labelled). Proof (real /dev/vms, QEMU): tests/qemu/test_syssvc_hiber_ast.c — A arms a write-attention AST and $HIBERs with NO explicit $SETAST; unrelated process B writes the mailbox by name; A's AST runs and A returns from $HIBER (bounded, so a deadlock is a FAIL not a hang); plus a sticky-wake scenario. Negctl hiber-ast-not-delivered removes the arrival broadcast → deadlock → red, while test_syssvc_mbx_wrtattn (explicit $SETAST) stays green (floor 93→94). Booted locally: test_syssvc_hiber_ast 5/0, all 73 suites pass (no regression in test_kmod_ast/lock/eflag, test_syssvc_ast/ast_secmode/mbx_wrtattn/mbx_cmdresp). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…usl link graph, vms-5a2) The native-link/self-host jobs went RED (LINK graph, DCL-native, self-host S1-S4, LIBRARIAN, IMGACT, TCC, PARTS, libvms+vmsrms migration) with Error 127. Root cause: mk_libvms_shr.sh DISCOVERS libvms's exported universals via nm and any not in libvms_shr.vec.frozen go through a `join` reconciliation (mk_libvms_shr.sh:181) that the alpine-musl container lacks (busybox has no join). The prior commit made vms$$deliver_pending_asts NON-STATIC in libvms (so sys$hiber could share the AST drain with sys$setast) -- a new discovered universal absent from the frozen manifest -> non-empty append set -> join -> Error 127. Debug ctest never runs the native-link container, so the local 73/73 QEMU proof did not catch it. Fix (append-only, GSMATCH-safe), the sanctioned workflow the vms$$chan_to_fd family already follows: - libvms_shr.vec + .frozen: append vms$$deliver_pending_asts=PROCEDURE, so it is a frozen universal at a stable index and the join append-path is skipped entirely. - libvmssys_shr.vec.frozen: freeze the three trailing appended entries (vms_kif_mbx_set_wrtattn from vms-9003, plus this branch's vms_kif_hiber and vms_kif_wake) so the libvmssys manifest also has an empty append set. The three must be frozen together to keep .frozen an ordered PREFIX of .vec. Verified: test_symvec_freeze.sh green (both manifests 0 appended, append-only intact), and the FULL VMS-native LINK.EXE graph rebuilds clean in the amd64 alpine:3.20 musl container (cmake --build --target link_native_graph -> 9 artifacts, EM_X86_64, the exact path that was failing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…onest-fail (fix Build&Test)
The authenticity gate tests/integration/test_userspace_service_register.sh went
RED: sys$hiber and sys$wake now REACH the executive (transitive vms_kif_hiber/
vms_kif_wake calls) but were still declared OVMX-USERSPACE ("pause()" / "SIGCONT
by Linux pid") — the register's exact "declared wholly userspace but reaches the
executive" refusal. Debug ctest runs this gate; the QEMU executive subset did not.
Root-cause fix (not a weakening — the declarations were describing the OLD, now-
replaced behavior): upgrade both to the honest split, mirroring sys$setast (which
also has an executive half + a userspace AST-drain half):
- OVMX-PARTIAL sys$hiber (vms-feb): exec = the hibernate WAIT + sticky wake state
are the executive's (VMS_IOCTL_HIBER); OVMX-LOCAL: the AST dispatch + re-hiber
loop run in the calling process (vms$$deliver_pending_asts).
- OVMX-PARTIAL sys$wake (vms-feb): exec = VMS_IOCTL_WAKE sets the sticky wake bit
and resolves a cross-process target by VMS PID; OVMX-LOCAL: NULL-vs-pidadr
selection + prcnam-discard are local.
Also make sys$hiber fail honestly instead of busy-looping when /dev/vms is absent
(Rule 9 / INV-6): vms_kif_hiber now returns the executive's VMS status with the
woken flag via an out-param, and sys$hiber returns that status on a non-success
(even) code rather than spinning. In QEMU the ioctl succeeds (SS$_NORMAL) so the
proven behavior is unchanged.
Verified: userspace_service_register PASSES and the FULL Debug ctest is 173/173,
0 failed (the CI Build & Test set, not the executive subset). Symbol set
unchanged (vms_kif_hiber keeps its name), so the frozen vectors are untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl
added a commit
that referenced
this pull request
Aug 13, 2026
mk_libvms_shr.sh computed the newly-discovered-universals append set with `join -t= append.names disc.vec`. `join` is a coreutils tool ABSENT from the alpine-musl native-link container (busybox) -> Error 127 broke every job that rebuilds LIBVMS$SHR the moment the append set was non-empty. The vms-bd1 freeze left it latent (all symbols frozen => empty append => the join line short- circuited on `[ -s append.names ]`); the next newly-exported libvms universal re-triggered it (vms-ec70 #446, vms-feb #457). Replace the join with awk set-membership: emit each discovered `name=class` whose name is in the append set, iterating disc.vec (which is `sort -u`'d, so name-sorted, names unique) -> byte-identical to join's sorted-by-key output. comm and paste stay (busybox HAS both; verified against alpine:3.20). Proof: alpine:3.20/amd64 `cmake --build --target link_native_graph` with the last universal temporarily unfrozen (350 frozen + 1 appended, NON-EMPTY append set) links the full graph clean (BUILD_RC=0) through LOGINOUT.EXE, with the "appended BEYOND the frozen manifest" note firing. join/awk equivalence checked on host for both empty and non-empty append sets. Freeze gate + negctl green. Co-authored-by: alice <alice@workspace.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl
added a commit
that referenced
this pull request
Aug 13, 2026
…ve (#472) Bump OVMX_PRODUCT_VERSION V0.4-2 → V0.4-3. 15 PRs since V0.4-2. Headline: the self-host toolchain now BUILDS — MMK.EXE drives real compile+link inside OVMX against a live executive. SELF-HOST #4 COMPLETE MMK.EXE genuinely drives compile+link builds vs real /dev/vms (#464 capstone). Full exec-drive substrate: async AST delivery + interruptible $HIBER (#457), IO$M_NOW (#458), DCL-over-mailbox (#460), + crash fixes #463 (32→64 ptr-width) / #464 (IO$M_NOW func-code mask). Freeze-join fix (#459). Component build host-proven (#470). UX FIDELITY SHOW CPU (#465), file protection SET/display (#467), RECALL readline-independent (#468), DCL scripting $STATUS/%X + CALL/SUBROUTINE + DECK/EOD (#469), DIRECTORY wildcards/ellipsis (#461). + swept other threads' merged work Self-host spine #5/#6 (MMK-drives-a-real-component IN QEMU) in flight. Co-authored-by: alice <alice@workspace.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Real VMS asynchronous AST delivery semantics: a queued AST is now delivered asynchronously to a hibernating process, interrupting
$HIBER, instead of only draining on an explicitsys$setast(1). This is the keystone prereq for MMK's exec-drive (spine #4,vms-b23) —tests/corpus/tier3-mmk/build_target.cuses exactlysys$wake(0,0)+sys$hiber().The gap (found by vms-b23 #454): the AST queue existed but there was no async delivery —
sys$hiberwas a barepause(), so a process that armed a write-attention AST on a mailbox and$HIBERed waiting for it was never woken when another process wrote → deadlock.Mechanism
Split the wait (executive) from the dispatch (userspace — the AST routine address is only valid in the target process). Executive-resident, cross-process, through
/dev/vms(Rule 9 / INV-6 — no per-process fake):VMS_IOCTL_HIBERblocks until a$WAKEis pending or an AST is deliverable at/below the caller's mode (vms_ast_has_deliverable, same bound asDELIVERAST); returnswoken=1iff a$WAKEreleased it (consuming a stickywake_pendingbit),0iff an AST did.VMS_IOCTL_WAKEsets the sticky wake bit on the target (self whenvms_pid==0— the MMK self-wake; else a VMS PID gated by GROUP/WORLD) and wakes it. Replaces the old raw-linux-pidSIGCONTshim.vms_ast_notify_arrival— every AST-enqueue path ($DCLAST, mailbox write-attention invms_mbx.c, lock completion/blocking ASTs invms_lock.c) broadcasts the target'shiber_wqafter queuing, underhiber_lock→ lost-wakeup-free, no lock-order inversion.sys$hiberloopsvms_kif_hiber()+vms$$deliver_pending_asts(), returning only on a$WAKE(its own, or one an AST issued) — VMS re-hibernates after an AST that does not$WAKE.sys$wakenow targets a VMS PID (self whenNULL).New universals
vms_kif_hiber/vms_kif_wakeappended to the frozen symbol vector (append-only). Clean-room provenance + full design:docs/design-async-ast-delivery-ovmx.md(VSI System Services$HIBER/$WAKE+ Programming Concepts AST semantics; OVMX choices labelled, Rule 8).Proof (real /dev/vms, QEMU — booted locally)
tests/qemu/test_syssvc_hiber_ast.c: process A arms a write-attention AST (routine self-$WAKEs) andsys$hibers with no explicit$SETASTdrain; unrelated re-exec'd process B assigns the mailbox by name and writes it; asserts A's$HIBERreturns and the AST ran — a bounded coordinator wait, so a deadlock is a named FAIL, not a QEMU-wide hang. Second scenario proves sticky wake ($WAKEbefore$HIBERreturns immediately).test_syssvc_hiber_ast: 5 passed, 0 failed against a real insmod'dvms.ko.test_kmod_ast/test_kmod_lock_mproc/test_kmod_eflag_mproc/test_syssvc_ast/test_syssvc_ast_secmode/test_syssvc_mbx_wrtattn/test_syssvc_mbx_cmdresp./dev/vms(CI negative-control contract).hiber-ast-not-delivered(floor 93→94): removes the arrival broadcast → the AST is queued but$HIBERnever wakes → deadlock → this suite reddens, whiletest_syssvc_mbx_wrtattn(explicit$SETAST, never hibers) stays green. Mutation verified unique + idempotent; the CI per-facility negctl shard exercises it against real/dev/vms.Scope / deferred
$WAKEand single-node$WAKE-by-VMS-PID implemented (self-wake proven).$WAKEby process name (prcnam) not resolved — redirected to self, same limitation the otherprcnam-taking process services carry.vms_ast.c. vms-feb adds when a deliverable AST wakes a hibernating process, not a new preemption model.send_cmd_and_waitat the AST/$HIBERlayer;IO$M_NOW(vms-5df) and DCL-mailbox wiring (vms-786) are still needed for the full MMK drive.🤖 Generated with Claude Code