Skip to content

vms-feb: executive-resident async AST delivery — $HIBER interrupted by a queued AST (cross-process) - #457

Merged
baron-3dl merged 3 commits into
mainfrom
vms-feb-async-ast-delivery
Aug 13, 2026
Merged

baron-3dl merged 3 commits into
mainfrom
vms-feb-async-ast-delivery

Conversation

@baron-3dl

Copy link
Copy Markdown
Contributor

What

Real VMS asynchronous AST delivery semantics: a queued AST is now delivered asynchronously to a hibernating process, interrupting $HIBER, instead of only draining on an explicit sys$setast(1). This is the keystone prereq for MMK's exec-drive (spine #4, vms-b23) — tests/corpus/tier3-mmk/build_target.c uses exactly sys$wake(0,0) + sys$hiber().

The gap (found by vms-b23 #454): the AST queue existed but there was no async deliverysys$hiber was a bare pause(), so a process that armed a write-attention AST on a mailbox and $HIBERed waiting for it was never woken when another process wrote → deadlock.

Mechanism

Split the wait (executive) from the dispatch (userspace — the AST routine address is only valid in the target process). Executive-resident, cross-process, through /dev/vms (Rule 9 / INV-6 — no per-process fake):

  • VMS_IOCTL_HIBER blocks until a $WAKE is pending or an AST is deliverable at/below the caller's mode (vms_ast_has_deliverable, same bound as DELIVERAST); returns woken=1 iff a $WAKE released it (consuming a sticky wake_pending bit), 0 iff an AST did.
  • VMS_IOCTL_WAKE sets the sticky wake bit on the target (self when vms_pid==0 — the MMK self-wake; else a VMS PID gated by GROUP/WORLD) and wakes it. Replaces the old raw-linux-pid SIGCONT shim.
  • vms_ast_notify_arrival — every AST-enqueue path ($DCLAST, mailbox write-attention in vms_mbx.c, lock completion/blocking ASTs in vms_lock.c) broadcasts the target's hiber_wq after queuing, under hiber_lock → lost-wakeup-free, no lock-order inversion.
  • sys$hiber loops vms_kif_hiber() + vms$$deliver_pending_asts(), returning only on a $WAKE (its own, or one an AST issued) — VMS re-hibernates after an AST that does not $WAKE. sys$wake now targets a VMS PID (self when NULL).

New universals vms_kif_hiber/vms_kif_wake appended to the frozen symbol vector (append-only). Clean-room provenance + full design: docs/design-async-ast-delivery-ovmx.md (VSI System Services $HIBER/$WAKE + Programming Concepts AST semantics; OVMX choices labelled, Rule 8).

Proof (real /dev/vms, QEMU — booted locally)

tests/qemu/test_syssvc_hiber_ast.c: process A arms a write-attention AST (routine self-$WAKEs) and sys$hibers with no explicit $SETAST drain; unrelated re-exec'd process B assigns the mailbox by name and writes it; asserts A's $HIBER returns and the AST ran — a bounded coordinator wait, so a deadlock is a named FAIL, not a QEMU-wide hang. Second scenario proves sticky wake ($WAKE before $HIBER returns immediately).

  • test_syssvc_hiber_ast: 5 passed, 0 failed against a real insmod'd vms.ko.
  • Full harness: 73 suites passed, 0 failed — no regression in test_kmod_ast / test_kmod_lock_mproc / test_kmod_eflag_mproc / test_syssvc_ast / test_syssvc_ast_secmode / test_syssvc_mbx_wrtattn / test_syssvc_mbx_cmdresp.
  • Honest-skip: returns 77 with no /dev/vms (CI negative-control contract).
  • Negative control hiber-ast-not-delivered (floor 93→94): removes the arrival broadcast → the AST is queued but $HIBER never wakes → deadlock → this suite reddens, while test_syssvc_mbx_wrtattn (explicit $SETAST, never hibers) stays green. Mutation verified unique + idempotent; the CI per-facility negctl shard exercises it against real /dev/vms.

Scope / deferred

  • Self-directed $WAKE and single-node $WAKE-by-VMS-PID implemented (self-wake proven). $WAKE by process name (prcnam) not resolved — redirected to self, same limitation the other prcnam-taking process services carry.
  • Upward access-mode preemption (a kernel-mode AST trapping a user-mode process) is not introduced — remains the labelled OVMX containment choice already in vms_ast.c. vms-feb adds when a deliverable AST wakes a hibernating process, not a new preemption model.
  • Unblocks MMK's send_cmd_and_wait at the AST/$HIBER layer; IO$M_NOW (vms-5df) and DCL-mailbox wiring (vms-786) are still needed for the full MMK drive.

🤖 Generated with Claude Code

alice and others added 2 commits August 13, 2026 08:08
…d by a queued AST, cross-process (unblocks MMK send_cmd_and_wait)

A queued AST was drained only on an explicit sys$setast(1) and sys$hiber was a
bare pause(), so a process that armed a write-attention AST on a mailbox and then
$HIBERed waiting for it was NEVER woken when another process wrote the mailbox —
a deadlock, and exactly the $HIBER/$WAKE + write-attention pattern MMK's
send_cmd_and_wait uses (spine #4, vms-b23).

Make the wait, the wake state and the AST-arrival notification executive-resident
(Rule 9 / INV-6 — cross-process, through /dev/vms), keeping AST DISPATCH in
userspace (the routine address is only valid in the target process):

- VMS_IOCTL_HIBER (vms_ioctl_hiber): block until a $WAKE is pending OR an AST is
  deliverable at/below the caller's current mode (vms_ast_has_deliverable, same
  bound as DELIVERAST). Returns woken=1 iff released by a $WAKE (consuming a
  sticky wake bit), 0 iff by an AST. A bare signal does not end $HIBER.
- VMS_IOCTL_WAKE (vms_ioctl_wake): set the sticky wake_pending bit on the target
  (self when vms_pid==0 — the MMK self-wake; else a VMS PID gated by GROUP/WORLD)
  and wake it. Replaces the old raw-linux-pid SIGCONT shim.
- vms_ast_notify_arrival (vms_ast.c): every AST-enqueue path ($DCLAST, mailbox
  write-attention in vms_mbx.c, lock completion/blocking ASTs in vms_lock.c) now
  broadcasts the target's hiber_wq after queuing, under hiber_lock, so a $HIBER
  waiter wakes to drain. Lost-wakeup-free; no lock-order inversion.
- sys$hiber loops vms_kif_hiber() + vms$$deliver_pending_asts(), returning only on
  a $WAKE (its own, or one an AST issued) — VMS re-hibernates after an AST that
  does not $WAKE. sys$wake targets a VMS PID (self when NULL).

New universals vms_kif_hiber/vms_kif_wake appended to the frozen symbol vector
(append-only). Clean-room provenance + design in
docs/design-async-ast-delivery-ovmx.md (VSI System Services $HIBER/$WAKE +
Programming Concepts AST semantics; OVMX choices labelled).

Proof (real /dev/vms, QEMU): tests/qemu/test_syssvc_hiber_ast.c — A arms a
write-attention AST and $HIBERs with NO explicit $SETAST; unrelated process B
writes the mailbox by name; A's AST runs and A returns from $HIBER (bounded, so a
deadlock is a FAIL not a hang); plus a sticky-wake scenario. Negctl
hiber-ast-not-delivered removes the arrival broadcast → deadlock → red, while
test_syssvc_mbx_wrtattn (explicit $SETAST) stays green (floor 93→94). Booted
locally: test_syssvc_hiber_ast 5/0, all 73 suites pass (no regression in
test_kmod_ast/lock/eflag, test_syssvc_ast/ast_secmode/mbx_wrtattn/mbx_cmdresp).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…usl link graph, vms-5a2)

The native-link/self-host jobs went RED (LINK graph, DCL-native, self-host S1-S4,
LIBRARIAN, IMGACT, TCC, PARTS, libvms+vmsrms migration) with Error 127. Root
cause: mk_libvms_shr.sh DISCOVERS libvms's exported universals via nm and any not
in libvms_shr.vec.frozen go through a `join` reconciliation (mk_libvms_shr.sh:181)
that the alpine-musl container lacks (busybox has no join). The prior commit made
vms$$deliver_pending_asts NON-STATIC in libvms (so sys$hiber could share the AST
drain with sys$setast) -- a new discovered universal absent from the frozen
manifest -> non-empty append set -> join -> Error 127. Debug ctest never runs the
native-link container, so the local 73/73 QEMU proof did not catch it.

Fix (append-only, GSMATCH-safe), the sanctioned workflow the vms$$chan_to_fd
family already follows:
- libvms_shr.vec + .frozen: append vms$$deliver_pending_asts=PROCEDURE, so it is a
  frozen universal at a stable index and the join append-path is skipped entirely.
- libvmssys_shr.vec.frozen: freeze the three trailing appended entries
  (vms_kif_mbx_set_wrtattn from vms-9003, plus this branch's vms_kif_hiber and
  vms_kif_wake) so the libvmssys manifest also has an empty append set. The three
  must be frozen together to keep .frozen an ordered PREFIX of .vec.

Verified: test_symvec_freeze.sh green (both manifests 0 appended, append-only
intact), and the FULL VMS-native LINK.EXE graph rebuilds clean in the amd64
alpine:3.20 musl container (cmake --build --target link_native_graph -> 9
artifacts, EM_X86_64, the exact path that was failing).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…onest-fail (fix Build&Test)

The authenticity gate tests/integration/test_userspace_service_register.sh went
RED: sys$hiber and sys$wake now REACH the executive (transitive vms_kif_hiber/
vms_kif_wake calls) but were still declared OVMX-USERSPACE ("pause()" / "SIGCONT
by Linux pid") — the register's exact "declared wholly userspace but reaches the
executive" refusal. Debug ctest runs this gate; the QEMU executive subset did not.

Root-cause fix (not a weakening — the declarations were describing the OLD, now-
replaced behavior): upgrade both to the honest split, mirroring sys$setast (which
also has an executive half + a userspace AST-drain half):
- OVMX-PARTIAL sys$hiber (vms-feb): exec = the hibernate WAIT + sticky wake state
  are the executive's (VMS_IOCTL_HIBER); OVMX-LOCAL: the AST dispatch + re-hiber
  loop run in the calling process (vms$$deliver_pending_asts).
- OVMX-PARTIAL sys$wake (vms-feb): exec = VMS_IOCTL_WAKE sets the sticky wake bit
  and resolves a cross-process target by VMS PID; OVMX-LOCAL: NULL-vs-pidadr
  selection + prcnam-discard are local.

Also make sys$hiber fail honestly instead of busy-looping when /dev/vms is absent
(Rule 9 / INV-6): vms_kif_hiber now returns the executive's VMS status with the
woken flag via an out-param, and sys$hiber returns that status on a non-success
(even) code rather than spinning. In QEMU the ioctl succeeds (SS$_NORMAL) so the
proven behavior is unchanged.

Verified: userspace_service_register PASSES and the FULL Debug ctest is 173/173,
0 failed (the CI Build & Test set, not the executive subset). Symbol set
unchanged (vms_kif_hiber keeps its name), so the frozen vectors are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl added a commit that referenced this pull request Aug 13, 2026
mk_libvms_shr.sh computed the newly-discovered-universals append set with
`join -t= append.names disc.vec`. `join` is a coreutils tool ABSENT from the
alpine-musl native-link container (busybox) -> Error 127 broke every job that
rebuilds LIBVMS$SHR the moment the append set was non-empty. The vms-bd1 freeze
left it latent (all symbols frozen => empty append => the join line short-
circuited on `[ -s append.names ]`); the next newly-exported libvms universal
re-triggered it (vms-ec70 #446, vms-feb #457).

Replace the join with awk set-membership: emit each discovered `name=class`
whose name is in the append set, iterating disc.vec (which is `sort -u`'d, so
name-sorted, names unique) -> byte-identical to join's sorted-by-key output.
comm and paste stay (busybox HAS both; verified against alpine:3.20).

Proof: alpine:3.20/amd64 `cmake --build --target link_native_graph` with the
last universal temporarily unfrozen (350 frozen + 1 appended, NON-EMPTY append
set) links the full graph clean (BUILD_RC=0) through LOGINOUT.EXE, with the
"appended BEYOND the frozen manifest" note firing. join/awk equivalence checked
on host for both empty and non-empty append sets. Freeze gate + negctl green.

Co-authored-by: alice <alice@workspace.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@baron-3dl
baron-3dl merged commit 0585d62 into main Aug 13, 2026
60 of 61 checks passed
baron-3dl added a commit that referenced this pull request Aug 13, 2026
…ve (#472)

Bump OVMX_PRODUCT_VERSION V0.4-2 → V0.4-3. 15 PRs since V0.4-2. Headline:
the self-host toolchain now BUILDS — MMK.EXE drives real compile+link
inside OVMX against a live executive.

  SELF-HOST #4 COMPLETE  MMK.EXE genuinely drives compile+link builds vs real
                         /dev/vms (#464 capstone). Full exec-drive substrate:
                         async AST delivery + interruptible $HIBER (#457),
                         IO$M_NOW (#458), DCL-over-mailbox (#460), + crash fixes
                         #463 (32→64 ptr-width) / #464 (IO$M_NOW func-code mask).
                         Freeze-join fix (#459). Component build host-proven (#470).
  UX FIDELITY            SHOW CPU (#465), file protection SET/display (#467),
                         RECALL readline-independent (#468), DCL scripting
                         $STATUS/%X + CALL/SUBROUTINE + DECK/EOD (#469),
                         DIRECTORY wildcards/ellipsis (#461).
  + swept other threads' merged work

Self-host spine #5/#6 (MMK-drives-a-real-component IN QEMU) in flight.

Co-authored-by: alice <alice@workspace.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant