diff_surface MAY_OMIT — subset-tolerant golden diff (vms-c38 part A / mechanism) - #951
Merged
Merged
Conversation
…grounded substrate-omissions) The mechanism half of vms-c38's subset-tolerant gate (conductor ruling b). OVMX renders a FAITHFUL SUBSET of the VMS oracle: it honestly omits sections the substrate lacks (vms-8019 -- the acceptance battery already asserts must_not_have 'Virtual I/O Cache Usage' [no XFC] + 'Slot Usage' [no balance-set slot table]). So a full-VAX golden must not byte-diff OVMX. - diff_surface apply_may_omit: strips each surface-declared substrate-ABSENT SECTION (header + data rows to the next blank) from BOTH golden and OVMX before the MATCH compare. A declared, grounded omission -> MATCH; an UNDECLARED omission leaves the golden's section and reds (HOLLOW/FORMAT-DIVERGENT). MAY_OMIT is for substrate-absent facilities ONLY -- a sourceable-but-unrendered field stays a real HOLLOW gap and must never be listed (INV-6: not an allowlist). - vax-show-memory.surface: MAY_OMIT='Virtual I/O Cache Usage|Slot Usage', each entry grounded in the battery's existing (reviewed) must_not_have. - selftest gains 2 cases: declared substrate-absent section -> MATCH; undeclared omission -> reds. Validated: 8/8 selftest; golden-self MATCH; simulated OVMX faithful-subset (golden minus the 2 declared sections) MATCH; undeclared omission (minus Physical) FORMAT-DIVERGENT. Part B (the battery wiring — a golden_diff assertion in dcl_acceptance_battery.sh + the REDS/GREENS proofs) follows; it needs echo/prompt normalization (run_cmd's $SEG format differs from the golden's) + is CI-iterative (no local OVMX boot). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl
added a commit
that referenced
this pull request
Aug 30, 2026
…— continuous oracle golden-diff gate
The gate itself: dcl_acceptance_battery.sh now runs a golden_diff over the 5 core
SHOW-family goldens (MEMORY/SYSTEM/CPU/DEVICE/PROCESS), upgrading the piecewise
must_haves to a continuous whole-layout diff against the real-VMS oracle.
Two required proofs, per surface, inside the battery:
- GREENS: run the surface's OWN commands (self-contained -- the battery ran
'SHOW DEVICE DKA0:' not the golden's 'SHOW DEVICE D', so golden_diff runs the
golden's exact commands), capture $SEG, and diff_surface -> MATCH (modulo the
surface's grounded MAY_OMIT).
- REDS: a golden_diff_negctl injects a divergence into the SAME output and asserts
it does NOT MATCH -- proving the gate can actually fail (not vacuously green).
diff_surface strip_console: the oracle golden ("$ CMD\n<out>", capture_oracle's
prompt-prefixed echo, no trailing prompt) and run_cmd's $SEG ("CMD\n<out>\n$ ",
bare echo + returned prompt) frame the console differently; strip_console drops
the command-echo + bare-prompt lines from BOTH so the gate compares the OUTPUT
LAYOUT, not console framing. Validated locally: a simulated run_cmd $SEG
faithful-subset -> MATCH; an injected divergence -> FORMAT-DIVERGENT.
Builds on part A (diff_surface MAY_OMIT, #951 -- stacked until it reaps, then
rebased to main). selftest 8/8 still green. The GREENS proof + any ADDITIONAL
grounded MAY_OMIT (Dynamic Memory / Paging File, only if substrate-absent) are
CI-verified on the real OVMX boot -- multi-round expected, never a MAY_OMIT added
just to turn a red green (INV-6).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
baron-3dl
added a commit
that referenced
this pull request
Aug 30, 2026
…nuous oracle golden-diff gate (part B) (#952) * vms-c38 part B: wire the DCL/SHOW acceptance battery to diff_surface — continuous oracle golden-diff gate The gate itself: dcl_acceptance_battery.sh now runs a golden_diff over the 5 core SHOW-family goldens (MEMORY/SYSTEM/CPU/DEVICE/PROCESS), upgrading the piecewise must_haves to a continuous whole-layout diff against the real-VMS oracle. Two required proofs, per surface, inside the battery: - GREENS: run the surface's OWN commands (self-contained -- the battery ran 'SHOW DEVICE DKA0:' not the golden's 'SHOW DEVICE D', so golden_diff runs the golden's exact commands), capture $SEG, and diff_surface -> MATCH (modulo the surface's grounded MAY_OMIT). - REDS: a golden_diff_negctl injects a divergence into the SAME output and asserts it does NOT MATCH -- proving the gate can actually fail (not vacuously green). diff_surface strip_console: the oracle golden ("$ CMD\n<out>", capture_oracle's prompt-prefixed echo, no trailing prompt) and run_cmd's $SEG ("CMD\n<out>\n$ ", bare echo + returned prompt) frame the console differently; strip_console drops the command-echo + bare-prompt lines from BOTH so the gate compares the OUTPUT LAYOUT, not console framing. Validated locally: a simulated run_cmd $SEG faithful-subset -> MATCH; an injected divergence -> FORMAT-DIVERGENT. Builds on part A (diff_surface MAY_OMIT, #951 -- stacked until it reaps, then rebased to main). selftest 8/8 still green. The GREENS proof + any ADDITIONAL grounded MAY_OMIT (Dynamic Memory / Paging File, only if substrate-absent) are CI-verified on the real OVMX boot -- multi-round expected, never a MAY_OMIT added just to turn a red green (INV-6). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * vms-c38 round 2: mount the oracle tooling into the acceptance container so golden_diff can find it Round-1 CI reds were NOT a GREENS content mismatch -- all 5 golden_diff calls failed "surface/tooling not found". Root cause: the acceptance test runs in a Docker container where run_dcl_acceptance_e2e.sh mounts ONLY /test.sh and /lib/dcl_acceptance_battery.sh -- tools/oracle (diff_surface + surfaces) and the goldens are not present inside, so the battery's repo-relative _ORACLE_DIR resolved empty. - run_dcl_acceptance_e2e.sh: mount tools/oracle -> /oracle/tools/oracle and docs/oracle/golden -> /oracle/docs/oracle/golden (a repo-root-like /oracle prefix so diff_surface's own HERE/REPO/GOLDEN_DIR path math resolves), and pass OVMX_ORACLE_DIR=/oracle/tools/oracle. - dcl_acceptance_battery.sh: _ORACLE_DIR = ${OVMX_ORACLE_DIR:-<repo-relative fallback>} so the container uses the mount and a local checked-out-tree run still uses the relative path. selftest 8/8 + golden-self MATCH unchanged (the tooling is untouched; only its availability inside the container). This unblocks the actual GREENS proof -- whether OVMX's output MATCHes each golden -- for the next CI round. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * vms-c38 round 3: log the golden_diff FORMAT-DIVERGENT diff for diagnosis Round 2 (oracle mount fixed) ran the REAL gate: all 5 REDS-negctl PASS (the gate can fail -- REDS proof done), and all 5 GREENS are FORMAT-DIVERGENT. All-five diverging is a systematic cross-system tell (not per-surface substrate-omission), but the battery only logged the classification, not the diff. This logs the full diff_surface output (normalized golden < vs OVMX >) on a red so the exact diverging line/section is diagnosable -- to tell a grounded substrate-absent omission from a value-width/machine-string difference from a real gap, before choosing the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * vms-c38: structure_norm — cross-system value-tolerant compare (mechanism, selftest 12/12) The conductor-ruled Option-1 core (2026-08-30): a byte-exact column-geometry gate is impossible cross-system (OVMX's values legitimately differ from the VAX/Alpha oracle: wider numbers, different machine strings). structure_norm proves STRUCTURAL fidelity -- same sections/labels/headers/field-structure, value-tolerant -- via three symmetric transforms applied to BOTH golden and OVMX after MAY_OMIT/ strip_console: (1) grounded per-surface MACHINE_MASK, (2) collapse-digit-RUN->one token, (3) whitespace-normalize. Guardrails (selftest): a HOLLOW numeric field (blank, no digits) STILL reds; a MACHINE_MASK'd field that is blank/absent STILL reds -- a mask means "value varies," never "ignore the field" (INV-6). Mechanism only; per-surface MACHINE_MASK values await the conductor's sign-off on the round-3 diff finding (the divergence is MIXED: value/machine-string AND real structural fidelity gaps that must stay red). NOT pushed until that sign-off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * vms-c38: Option-A standing gate — hard-gate CLAIMED-FAITHFUL, report-only the rest; graduate show-cpu Conductor ruling 2026-08-30 (Option A). Round-3's diff proved the cross-system divergence is MIXED — value/machine-string AND real structural fidelity gaps — so normalization alone cannot green all 5, and masking the real gaps green would be the exact INV-6 allowlist-cheat. Structure the standing gate in two honest tiers: - HARD-GATE only CLAIMED-FAITHFUL surfaces (a divergence FAILS the leg = regression-proof). vax-show-cpu graduates here: it is genuinely structurally faithful — the model line, the "Multiprocessing is ..." state, and the Active/Configured CPU-ID list are machine-varying VALUES (masked via grounded, label-preserving MACHINE_MASK, DCL-Dictionary-pinned per src/vmsdcl/dcl_cmd_show.c); the labelled structure MATCHes the oracle through the full pipeline (proven, not masked-to-hide-a-gap). (Refutes the round-3 "Active-CPUs extra token = gap" sub-hypothesis: "## ##" is the faithful 2-CPU ID list, not a bug.) - REPORT-only the not-yet-faithful surfaces (new note() primitive: loud, logged, routed to a fidelity item every run, but NO PASS/FAIL touch). Round-3 findings: vax-show-memory HOLLOW (omits Dynamic Memory + Paging File sections; OVMX has pool+pagefile) -> vms-352; vax-show-system HOLLOW (omits State/Pri/I/O columns) -> vms-6b8e; vax-show-device MISSING (%NOSUCHDEV, device-name model) -> vms-ddc (+vms-9f5); vax-show-process HOLLOW (omits Terminal/Base priority/Devices allocated; UIC not resolved to [SYSTEM]) -> vms-1f7. These are TRUE findings the gate exists to drive (vms-050 backlog); each graduates to hard-gate when its item lands and it genuinely MATCHes. This tracks + names every gap loudly (anti-LARP), it does not silently pass them — and it can't permanently-red main's green-by-SHA. GREENS proof = show-cpu greens through the full pipeline (hard-gated) + the diff_surface selftest MATCH cases. REDS proof = the show-cpu negctl (an injected divergence does NOT MATCH) + the selftest, both retained. Register (docs/compat/ ux-surface-register.md) records the structure-tolerant bar + each surface's honest status. No brittle per-defect red-set .tsv (vms-49f). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * vms-c38: tighten MP-STATE mask .* -> .+ so a blank value still reds (VAX-lane guardrail catch) VAX-lane cross-review of 269d662 flagged a guardrail-2 hole: the SHOW CPU MP-STATE mask used '.*', which matches an EMPTY value -- so a future hollow "Multiprocessing is " (label present, state BLANK) would pass the mask, unlike the model mask (bounded by the required ' (Series|system)$' suffix) and the CPU-list masks (bounded by '[#0-9 ]+'). Not a current fidelity issue (OVMX prints a real ENABLED/DISABLED, and a MISSING line still reds) -- a regression-protection hole a future empty MP-state would slip. Fix: '.*' -> '.+' so a blank MP-state reds like the other two masks, making the present+non-empty (anti-hollow) property UNIFORM across all three cpu masks. Adds the matching selftest case (mirrors guardrail 3): a '.+'-terminated value mask matches a present MP-state (ENABLED -> MATCH) but keeps a BLANK one RED -- a future regression back to '.*' fails this test. diff_surface selftest 14/14; show-cpu still MATCHes with the '.+' mask (acceptance leg re-passes identically, OVMX's MP-state is non-empty). Structure VAX already OK'd (#1 anti-facade, #3 note()-driver) unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The mechanism half of vms-c38's subset-tolerant gate (conductor ruling b). Split out as a self-contained, selftest-proven tool increment — the same shape as diff_surface (#946) landing before vms-c38 consumes it. This does NOT close vms-c38 (that's part B, the battery wiring with both proofs); it's the capability the gate will use.
OVMX renders a faithful subset of the VMS oracle: it honestly omits sections the substrate lacks (vms-8019 — the acceptance battery already asserts
must_not_have 'Virtual I/O Cache Usage'[no XFC] +must_not_have 'Slot Usage'[no balance-set slot table]). So a full-VAX golden must not byte-diff OVMX.diff_surfaceapply_may_omit: strips each surface-declared substrate-absent SECTION (header + data rows to the next blank) from both golden and OVMX before the MATCH compare. A declared, grounded omission → MATCH; an undeclared omission leaves the golden's section and reds (HOLLOW/FORMAT-DIVERGENT). ⚠ MAY_OMIT is for substrate-absent facilities only — a sourceable-but-unrendered field stays a real HOLLOW gap and must never be listed (INV-6: not an allowlist to pass).vax-show-memory.surface:MAY_OMIT='Virtual I/O Cache Usage|Slot Usage', each entry grounded in the battery's existing (reviewed)must_not_have.Validation (gate as a tool: the selftest)
diff_surface selftest— 8/8, including the two new MAY_OMIT cases: a declared substrate-absent section → MATCH; an undeclared omission → reds. Plus (local): golden-self → MATCH; simulated OVMX faithful-subset (golden minus the 2 declared sections) → MATCH; undeclared omission (minus Physical) → FORMAT-DIVERGENT.Refs rd
vms-c38(part A / mechanism; the gate itself lands in part B with the REDS+GREENS proofs).🤖 Generated with Claude Code