diff --git a/CHANGELOG.md b/CHANGELOG.md index 06b9bdea..ba09f96f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,25 @@ All notable changes to the claude-plugins project will be documented in this fil The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Entries are listed newest-first; each plugin section is treated as released when merged to `main`. +### vibe v1.1.94 + +#### Added +- New `skills/vibe/scripts/capture-guard.mjs` refuses a screenshot capture that repeats a ticket's main baseline or is too big (ISS-14158). It only ever refuses; nothing requires a capture. + - `begin --spec ` refuses (exit 79) a main commit already captured for the ticket, any second main baseline for it, more than 240 shots (pages times states, declared in the spec), or more than about 20 minutes at the higher of 5 s per shot and the slowest rate the ticket's finished captures measured. A main commit is an ancestor of the capture worktree's origin default branch (`refs/remotes/origin/HEAD`, else `origin/main`); a ref it cannot resolve is refused. + - `end` records a running entry once as ok or failed, with its measured seconds, and refuses an entry that is not running. A running entry holds its place while its owner lives or until its own estimated minutes, never fewer than 20, have passed. `status` prints the ticket's main baseline, finished captures and measured rate. `facts` reads a ref on the host that holds the worktree. + - Overrides: the spec's `override` carries only `quote`. The rulings file comes only from `begin --rulings`. A Ruling line (read with `rulings-check.mjs`'s `recordRulings`) overrides a refusal only in a fixed shape, `- Ruling YYYY-MM-DD (): "" capture-guard override: `: the decision text begins with the line's only double-quoted span (two ASCII double quotes, no curly double quotes), whose content equals the spec's quote, and the ticket (exact slug, `rulingNamesScope`) and the `capture-guard override: baseline` or `capture-guard override: budget` marker follow its closing quote. Any other shape counts for nothing. `references/state.md` states the same shape for recorders. `baseline` lifts only the repeated and second main baseline refusals and `budget` only the shot and minute refusals; a capture that meets both needs both markers. A matching line without the needed marker gets its own refusal naming the marker. + - The ledger is `.json` in `--ledger`; without it, the program's `captures` directory when exactly one fleet program's `board.json` (found through FLEET_HOME and the fleet chat bindings) lists this clone as a writer worktree, else `/vibe-captures`. With both `--ledger` and `--repo` the two ledgers mirror each other. +- `fleet.sh local|remote slot --capture ` runs the guard against `/captures` (mirrored in the local clone's ledger) with `--rulings /rulings.md` before the load check and before any slot is taken. It times the command inside the slot, so the recorded seconds exclude the queue wait, and records how it ended. A TERM, INT or HUP stops the command and frees the slot, locally and on the remote host (a stop marker covers a remote run that has not started), and the entry records the real exit. `capture-guard.mjs` ships to the remote host. A slot without `--capture` is unchanged. + - `fleet.test.mjs` covers a `--capture` spec that is missing or unreadable (exit 2 with the usage line or `capture-guard: invalid spec`, before the load check and before any slot or command, with nothing recorded) and `--capture` given before or after `--containerized-e2e-build` (both honored, the disk preflight still runs). + +#### Changed +- `references/quality-loop-visual-qa.md` gains the block `when-to-capture` ("When to capture screenshots") with ISS-14158's approved criterion 1 text: one visual-qa reviewer on the diff plus the stories by default; captures only for a shared component's pixels, a criterion or ruling that asks for visual proof, or a finding the reviewer cannot settle; one main baseline per ticket; recapture only the states a fix reaches; the final check is one visual-qa pass scoped to the changes since the last reviewed capture. It also says to run `begin` before and `end` after any capture of more than one state. `quality-loop.md` and the fleet QA gate point at it. +- The fleet QA gate's and the verify worker's before and after baseline is the ticket's one main baseline, never a new one because main moved; a difference on a state the ticket's diff does not reach comes from main and is not reported as the ticket's. +- The verify worker captures a screen when the session has a reference image for it, or when the `when-to-capture` block calls for one with `begin` and `end` around it; its shared-component sweep compares against the ticket's one main baseline. Handoff's rerun after visual QA fixes runs visual QA as one pass scoped to the changes since the last reviewed capture. The shared-scope and card-layout sentences in handoff and the verify worker now cite the visual QA reference instead of restating it. +- Fleet's Machines and CPU section adds the `--capture` slot rule, including that only an operator ruling recorded in the program rulings with the marker for the refusal it lifts lets a refused capture run; `references/state.md` adds that the marker goes on a ruling only when the operator's words ask for that override. The reporting rule and the Remaining work report template require every ETA for queued work to be computed from the remaining queue count and restated the moment a step adds more than 30 minutes. +- The README version line and the Python and vibe-bench release-pin tests move to 1.1.94; the prompt-split inventory includes the two new stubs. +- Across all plugin Markdown, including eval prompts and graders, as measured by `plugins/vibe/skills/fleet-orchestrator/scripts/instruction-words.mjs`, vibe is about 496 words longer than in 1.1.93 (origin/main at this update, c6514594). + ### vibe-core v1.0.12 #### Fixed diff --git a/plugins/vibe/.claude-plugin/plugin.json b/plugins/vibe/.claude-plugin/plugin.json index 70a5a3e5..ba0bc142 100644 --- a/plugins/vibe/.claude-plugin/plugin.json +++ b/plugins/vibe/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "vibe", "description": "Vibe sessions, handoff, seed refresh, and ClosedLoop ticket workflows", - "version": "1.1.93", + "version": "1.1.94", "dependencies": ["closedloop-core", "vibe-core"], "author": {"name": "ClosedLoop", "email": "support@closedloop.ai"} } diff --git a/plugins/vibe/.codex-plugin/plugin.json b/plugins/vibe/.codex-plugin/plugin.json index 6681ebb2..d2196b4e 100644 --- a/plugins/vibe/.codex-plugin/plugin.json +++ b/plugins/vibe/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "vibe", - "version": "1.1.93", + "version": "1.1.94", "description": "Vibe sessions, handoff, seed refresh, and ClosedLoop ticket workflows", "author": {"name": "ClosedLoop", "email": "support@closedloop.ai"}, "homepage": "https://github.com/closedloop-ai/claude-plugins", diff --git a/plugins/vibe/README.md b/plugins/vibe/README.md index 71bc6fa8..96fafa5a 100644 --- a/plugins/vibe/README.md +++ b/plugins/vibe/README.md @@ -376,7 +376,7 @@ Run the matching Codex suite from the repository root with ### Runtime files -Version `1.1.93` contains 17 skills and 17 agents, and no hook manifest: its +Version `1.1.94` contains 17 skills and 17 agents, and no hook manifest: its hooks, and every module they run, ship in the `vibe-core` plugin, which vibe declares as a dependency. vibe loads those modules from vibe-core through `shared/vibe-core.mjs`, and the commands among them keep thin wrappers at diff --git a/plugins/vibe/agents/vibe-verify-worker.md b/plugins/vibe/agents/vibe-verify-worker.md index 9ee2ec37..3f4242ad 100644 --- a/plugins/vibe/agents/vibe-verify-worker.md +++ b/plugins/vibe/agents/vibe-verify-worker.md @@ -470,17 +470,23 @@ session touched: runs displayless under the supported displayless harness. Never pass `--allow-write`, and never use `eval` or `eval-main`. 3. For each screen: `pnpm control goto `, `pnpm control wait --settled`, - `pnpm control capture --name -after`, `pnpm control console --level error`, + `pnpm control capture --name -after` when the session has a reference + image for that screen, or when + `../skills/vibe/references/quality-loop-visual-qa.md`, "When to capture + screenshots", calls for one, after `capture-guard.mjs begin` admits it and + with `capture-guard.mjs end` after it, + `pnpm control console --level error`, `pnpm control console --level pageerror` and `pnpm control network`. Add `pnpm control api GET ` for the record the screen shows, and `pnpm control ipc ` read calls for a Desktop screen, - where relevant. For shared scope, compare each consumer page against that - page's approved prototype route; report every control present in the - prototype but missing in production, and report every control with no - source on that surface. -4. When a baseline exists (an earlier control capture of the same entry on - the merge-base or current main, never a commit that is not an ancestor), - return its artifact paths beside the new ones and say what changed. When the session recorded reference images + where relevant. For shared scope (`../skills/vibe/references/quality-loop.md`, + "Visual QA during the session", "What it checks"), report every control + with no source on that surface. +4. When a baseline exists (the ticket's one main baseline from + `../skills/vibe/scripts/capture-guard.mjs status`, never a commit that is + not an ancestor), return its artifact paths beside the new ones and say + what changed; a difference on a state the ticket's diff does not reach + comes from main and is not the ticket's. When the session recorded reference images (`referenceImages`), open each as an image (`Read` in Claude Code, `view_image` in Codex) beside the matching capture and report every visible difference in shadow, elevation, font size, weight, color, @@ -491,7 +497,7 @@ session touched: For a shared component, capture every surface it reaches with the flag off at 390, 412, 1024 and 1280 px wide, rendering the exact shipped story or route -and main's render of the same one; a difference is a finding. +and the ticket's one main baseline of the same one; a difference is a finding. A static or generated page control cannot drive (an HTML page a script or a main-process module writes, such as Desktop's sign-in return page) is still diff --git a/plugins/vibe/skills/fleet-orchestrator/SKILL.md b/plugins/vibe/skills/fleet-orchestrator/SKILL.md index 7cb9ccd4..67d90ad7 100644 --- a/plugins/vibe/skills/fleet-orchestrator/SKILL.md +++ b/plugins/vibe/skills/fleet-orchestrator/SKILL.md @@ -154,6 +154,7 @@ Run [codex-fleet-monitor](../codex-fleet-monitor/SKILL.md) (`vibe:codex-fleet-mo - Run heavy checks (full suites, typecheck, repo-wide lint and other gates, component story runs) through `scripts/fleet.sh local|remote slot heavy `, in the worker's own worktree on that machine ([checklist step 4](./references/setup.md#checklist) gets the branch there). It caps runs at `HEAVY_SLOTS` or `REMOTE_HEAVY_SLOTS` and refuses a machine over `MAX_LOAD_PER_CORE` (exit 4): send the run to the other machine. Only the operator may override that routing by setting `FLEET_FORCE_HERE=1` with `FLEET_FORCE_REASON` to the reason they chose; the reason is written into the slot holder record. Writers never set `FLEET_FORCE_HERE`, its reason, or `FLEET_LOAD`; production admission always reads the target machine. A force is still refused above the fixed hard ceiling of four times the core count. When the force or its reason is absent, normal routing applies. Exit 78 means the containerized E2E disk preflight refused the command or the remote host did not have the exact required vibe version; its stderr names which cause. Keep the operator's daily machine for edits and git; e2e and desktop app runs never go there. - A verified `e2e/run-containerized.sh` or `dagger call e-2-e` build receives the shared Docker backing-disk check inside the acquired slot, immediately before its command. Pass `--containerized-e2e-build` after the worktree path when a wrapper obscures that exact command shape. The check requires 64 GB decimal free; `--no-build`, lint, typecheck, tests and Storybook do not take this Docker disk gate. The remote slot verifies the installed vibe version and invokes its same admission subcommand on the owning host. +- A screenshot capture through a slot passes `--capture ` (`ticket`, `ref`, `pages`, `states`, optional `override`) after the worktree path; `scripts/fleet.sh` runs `../vibe/scripts/capture-guard.mjs` first and refuses it (exit 79) as `../vibe/references/quality-loop-visual-qa.md`, block `when-to-capture`, says; only an operator ruling recorded for that ticket in the program rulings, carrying `capture-guard override: baseline` or `capture-guard override: budget` for the refusal it lifts and quoted in the spec's `override`, lets it run. Before approving any capture, including after a compaction, read `../vibe/scripts/capture-guard.mjs status --ticket --ledger /captures`. - Before reporting or waiting on remaining work, identify true dependencies. Start independent repository selected integration, unit, typecheck and lint lanes at once through available heavy slots on allowed machines, even while e2e is queued or running elsewhere. A lane waiting only for a full slot or load moves to another allowed machine with capacity; report the move. Respect operator exclusions, machine load and slot limits, no dagger restrictions and the daily machine's no visible e2e rule. Keep the repository lane selector and `qa-impact.mjs` decision on which checks are needed. - A shared e2e stack with fixed ports runs only under vibe's machine-wide e2e lock: `node ../vibe/scripts/e2e-lock.mjs run --head --source-hash --owner --ticket ` on the machine that runs it, never a second lock; receipts name HEAD plus diff/tree hashes, including dirty source. Each run gets its own compose project, so a teardown removes only its own stack; name a stack another tool started with `--foreign-compose `. Unit tests, typecheck, story runs and stand-in server tests run outside it. - `scripts/slot-lock.sh --status ` on a machine lists slot holders. @@ -208,7 +209,7 @@ Hard rule: a ticket, PR body, comment, reply or other artifact for people never ## Reporting to the operator - Surface every problem the moment you see it: limits, stalls, failures, new human review comments, blockers. Keep status updates short. -- For a remaining-work answer, compute time to ready from the dependency graph's longest dependent path, not a sum of independent lanes. Use the [Remaining work report](./references/templates.md#remaining-work-report). The response is incomplete until it states the duration and names the dependent steps and assumptions that produce it. +- For a remaining-work answer, compute time to ready from the dependency graph's longest dependent path, not a sum of independent lanes. Use the [Remaining work report](./references/templates.md#remaining-work-report). The response is incomplete until it states the duration and names the dependent steps and assumptions that produce it. Every ETA for queued work is computed from the remaining queue: state how many runs are still queued or running and each one's estimate, and restate the ETA the moment a step adds more than 30 minutes. - Before reporting a count or an "only" claim from a command you ran, check that it exited 0 and that no `head`, `tail` or tool truncation cut its output; otherwise rerun the narrow command or call the figure partial. Principle 2 holds for your own findings too. - When the operator steps away (vibe's `away-protocol` hook puts this checklist in context on "brb", "stepping away" and the like, and the return checklist on "I'm back"), keep `/attention.md` current (decision needed, recommendation, what is blocked), keep reading idle writers' last messages, and lead with that list on return. - Put every finding the operator reviews on one page using the stable `D`, `X`, `N`, `S` and `R` IDs in the [operator review template](./references/templates.md#operator-review-page); never restart numbering across audits. diff --git a/plugins/vibe/skills/fleet-orchestrator/references/fleet-qa-gate.md b/plugins/vibe/skills/fleet-orchestrator/references/fleet-qa-gate.md index b1e01b4d..26a6c712 100644 --- a/plugins/vibe/skills/fleet-orchestrator/references/fleet-qa-gate.md +++ b/plugins/vibe/skills/fleet-orchestrator/references/fleet-qa-gate.md @@ -2,11 +2,14 @@ Moved out of `../SKILL.md`, which says when to read it; later edits to that text land here. Markdown links are relative to this file; other paths stay relative to the skill folder. +> Read `../vibe/references/quality-loop-visual-qa.md` (block `when-to-capture`) before any screenshot capture or visual QA pass, in the loop or at handoff. + - QA happens before merge; never call a PR ready to enqueue before its QA is done. - A unit's PR is ready only once its review of the completed work passed (`references/ticket-units.md`, "Review the completed work"): every member and PRD criterion with a code row and a capture for each page and surface it names. - Each PR moves through independent review, fixes, verification, visual review where needed, one feature-complete QA pass, operator QA, one main merge, then enqueue. - QA evidence records screen, host, state and flag arm. Later diffs use handoff's `qa-impact.mjs` with importers, FEATURE_MAP and lanes. Obey its scope; `None` runs no local lane, screenshot, audit or critic. CI is per push. This also governs the final main merge before `shepherd.mjs`. + - Off by default and an open owner decision: with `FLEET_REVIEW_HANDOFF_ONLY=1`, `node scripts/review-chain.mjs --pr-head --handoff-reviewed-head ` prints `verify-fixes-only` when that review covered the PR's exact head, and the orchestrator then runs only the verification of the fixes in place of its own independent review. Any other output, an unset variable or a head that moved after the handoff review keeps the full chain above. - Before an audit, reconcile operator chat and AskUserQuestion answers plus guide-session and QA records into `/rulings.md`, then run `node ../handoff/scripts/rulings-check.mjs --program /rulings.md --record ...`; audit the acceptance criteria together with those rulings. - Before showing `/review.md`, run `node scripts/review-page-check.mjs /review.md`. @@ -26,5 +29,5 @@ Moved out of `../SKILL.md`, which says when to read it; later edits to that text - When a repository's Storybook CI requires a release-version bump and history entry for new stories, the writer includes them in the reviewed pre-PR candidate so that lane can pass. Pin main once before the final handoff checks as vibe:handoff describes; after QA, settle a version-only conflict once at enqueue in merge order and rerun the affected lane. Do not repeatedly merge main merely to chase another open PR's reserved version. - A fix reaches a preview only once the ref that preview deploys from contains it: check `git merge-base --is-ancestor ` before calling it live. - Track merge order for stacked or overlapping PRs and tell each writer which one merges first. -- A before and after comparison uses the candidate's merge-base or current main as the baseline, never an unrelated branch. +- A before and after comparison uses the ticket's one main baseline (`../vibe/scripts/capture-guard.mjs status`), never an unrelated branch and never a new baseline because main moved. diff --git a/plugins/vibe/skills/fleet-orchestrator/references/state.md b/plugins/vibe/skills/fleet-orchestrator/references/state.md index 92b834f6..c64be1e5 100644 --- a/plugins/vibe/skills/fleet-orchestrator/references/state.md +++ b/plugins/vibe/skills/fleet-orchestrator/references/state.md @@ -48,7 +48,7 @@ One fleet home per machine holds everything (default `~/fleet-orchestrator`; `FL ## Rulings -`standing-rulings.md` holds operator rulings for every program (models, review rounds, test lanes); the program's `rulings.md` holds the rest, separate from the resume log. Each entry is `- Ruling YYYY-MM-DD (operator message|AskUserQuestion|ticket: ): `. `fleet.sh` mirrors both to every worker host; point every worker and QA-prep prompt at them. Per-ticket rulings still go through handoff's `rulings-check.mjs`. A sequencing ruling is written only by that checker's `--record-sequencing` operation after it verifies the live blocker-to-blocked `BLOCKS` receipt; unresolved references and missing links refuse the write, and the status sweep flags later drift. Orchestrator-authored directions remain `(orchestrator)` operational notes, not entries in this file. +`standing-rulings.md` holds operator rulings for every program (models, review rounds, test lanes); the program's `rulings.md` holds the rest, separate from the resume log. Each entry is `- Ruling YYYY-MM-DD (operator message|AskUserQuestion|ticket: ): `. Add `capture-guard override: baseline` or `capture-guard override: budget` to a ruling only when the operator's words ask for that override, with the kind they ask for. A ruling with the marker is written as `- Ruling YYYY-MM-DD (): "" capture-guard override: `, with the operator's words as its only double-quoted text. `fleet.sh` mirrors both to every worker host; point every worker and QA-prep prompt at them. Per-ticket rulings still go through handoff's `rulings-check.mjs`. A sequencing ruling is written only by that checker's `--record-sequencing` operation after it verifies the live blocker-to-blocked `BLOCKS` receipt; unresolved references and missing links refuse the write, and the status sweep flags later drift. Orchestrator-authored directions remain `(orchestrator)` operational notes, not entries in this file. ## Lessons ledger diff --git a/plugins/vibe/skills/fleet-orchestrator/references/templates.md b/plugins/vibe/skills/fleet-orchestrator/references/templates.md index bd521bbf..aeb76e89 100644 --- a/plugins/vibe/skills/fleet-orchestrator/references/templates.md +++ b/plugins/vibe/skills/fleet-orchestrator/references/templates.md @@ -123,7 +123,7 @@ Branch isolation: one writer per branch; helpers never edit or touch the ticket ## Remaining work report -Before sending, verify the reply says which selected integration, typecheck and other lanes start immediately on which allowed machines; what waits on a real prerequisite; what runs concurrently with queued or running e2e; and which lane moved for capacity. Give the readiness estimate from the chain of prerequisites that finishes last, with its named steps and assumptions; never add durations from lanes that can overlap. Do not send a report missing any applicable fact. +Before sending, verify the reply says which selected integration, typecheck and other lanes start immediately on which allowed machines; what waits on a real prerequisite; what runs concurrently with queued or running e2e; and which lane moved for capacity. Give the readiness estimate from the chain of prerequisites that finishes last, with its named steps and assumptions; never add durations from lanes that can overlap. Name the remaining queue count (runs still queued or running, each with its estimate) and compute the ETA from it; restate the ETA the moment a step adds more than 30 minutes. Do not send a report missing any applicable fact. ## Lifecycle cleanup report diff --git a/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.sh b/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.sh index 21bbcaf2..2bdfc6ba 100755 --- a/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.sh +++ b/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.sh @@ -1,8 +1,10 @@ #!/usr/bin/env bash # Usage: fleet.sh session [codex-session.mjs options] -# fleet.sh slot +# fleet.sh slot [--containerized-e2e-build] [--capture ] # fleet.sh cleanup # A heavy slot first runs load-check.sh on that machine and refuses (exit 4) when its load per core is over MAX_LOAD_PER_CORE. +# A --capture slot first runs vibe's capture-guard.mjs against /captures, which refuses (exit 79) a repeated +# main baseline or an over-budget capture before any slot is taken, and records the capture's outcome when it ends. # is a fleet home (its active program) or a program dir; codex-fleet-monitor/scripts/work-dir.mjs # resolves it and loads config.env (references/setup.md); without the REMOTE_* keys only `local` works. # `remote` copies the scripts it needs, and a session's prompt file (with its .evidence/ folder), to /.fleet-orchestrator/ @@ -34,7 +36,7 @@ case "$where" in *) usage ;; esac -ship=(codex-fleet-monitor/scripts/liveness.mjs codex-fleet-monitor/scripts/thread-sidebar.mjs codex-fleet-monitor/scripts/work-dir.mjs fleet-orchestrator/scripts/codex-session.mjs fleet-orchestrator/scripts/fleet-remote-evidence.mjs fleet-orchestrator/scripts/skill-freshness.mjs fleet-orchestrator/scripts/writer-launch.mjs vibe/scripts/owned-state.mjs fleet-orchestrator/scripts/slot-lock.sh fleet-orchestrator/scripts/load-check.sh fleet-orchestrator/scripts/request-visual.mjs) +ship=(codex-fleet-monitor/scripts/liveness.mjs codex-fleet-monitor/scripts/thread-sidebar.mjs codex-fleet-monitor/scripts/work-dir.mjs fleet-orchestrator/scripts/codex-session.mjs fleet-orchestrator/scripts/fleet-remote-evidence.mjs fleet-orchestrator/scripts/skill-freshness.mjs fleet-orchestrator/scripts/writer-launch.mjs vibe/scripts/owned-state.mjs vibe/scripts/capture-guard.mjs fleet-orchestrator/scripts/slot-lock.sh fleet-orchestrator/scripts/load-check.sh fleet-orchestrator/scripts/request-visual.mjs) ship=("${ship[@]/#/skills/}") extra=() @@ -301,13 +303,87 @@ session() { fi } +capture_id='' +capture_ticket='' +capture_timing='' +capture_dir='' +capture_remote_pid='' +# Stops a remote capture: a stop marker keeps a remote line that has not started its slot from starting it, and TERM goes +# to the pid recorded last (the remote shell, which becomes slot-lock, then the command wrapper once the slot admits it). +capture_stop_remote() { + [ -n "$capture_remote_pid" ] || return 0 + local file stop + file=$(printf %q "$capture_remote_pid") + stop=$(printf %q "$capture_remote_pid.stop") + run_remote "mkdir -p $(printf %q "$remote_dir") && touch $stop; p=\$(cat $file 2>/dev/null); case \"\$p\" in ''|*[!0-9]*) ;; *) [ \"\$p\" -gt 1 ] && kill -TERM \"\$p\" 2>/dev/null ;; esac; true" >/dev/null 2>&1 || true +} +# Records how an admitted capture ended, with the seconds its command ran inside the slot (never the queue wait). +capture_end() { + [ -n "$capture_id" ] || return 0 + local seconds='' timed=() + if [ -n "$capture_timing" ]; then + if [ "$where" = local ]; then + seconds=$(cat "$capture_timing" 2>/dev/null) || seconds='' + rm -f "$capture_timing" + else + seconds=$(run_remote "cat $(printf %q "$capture_timing") 2>/dev/null; rm -f $(printf %q "$capture_timing") $(printf %q "${capture_remote_pid:-$capture_timing}") $(printf %q "${capture_remote_pid:-$capture_timing}.stop")" 2>/dev/null) || seconds='' + seconds=${seconds##*$'\n'} + fi + fi + if [[ "$seconds" =~ ^[0-9]+$ ]]; then timed=(--seconds "$seconds"); fi + # A local capture's entry is mirrored in its clone's own ledger, so end updates both. + if [ "$where" = local ]; then timed+=(--repo "$capture_dir"); fi + node "$skills/vibe/scripts/capture-guard.mjs" end --ledger "$orch/captures" --ticket "$capture_ticket" --id "$capture_id" --status "$1" ${timed[@]+"${timed[@]}"} >/dev/null || true + capture_id='' +} + +# Admits a declared capture through capture-guard.mjs, using ref facts read where the worktree lives. +capture_begin() { + local spec=$1 dir=$2 owner=$3 ref facts_out admitted + local facts_args=(--repo "$dir") + if [ "$where" = remote ]; then + ref=$(jq -er '.ref | select(type == "string" and length > 0)' "$spec") || usage + facts_out=$(run_remote "node $(printf %q "$remote_dir/plugins/vibe/skills/vibe/scripts/capture-guard.mjs") facts --repo $(printf %q "$dir") --ref $(printf %q "$ref")") || return + facts_args=(--facts "${facts_out##*$'\n'}") + fi + admitted=$(node "$skills/vibe/scripts/capture-guard.mjs" begin --spec "$spec" --ledger "$orch/captures" --rulings "$orch/rulings.md" --owner "$owner" --pid "$$" "${facts_args[@]}") || return + capture_ticket=$(jq -er '.ticket' <<<"$admitted") || return 2 + capture_id=$(jq -er '.id' <<<"$admitted") || return 2 + capture_dir=$dir + trap 'capture_end $?' EXIT + # A signal ends the run as failed: the EXIT trap then records the entry with this status. + trap 'exit 143' TERM + trap 'exit 130' INT + trap 'exit 129' HUP +} + slot() { [ $# -ge 4 ] || usage local name=$1 owner=$2 dir=$3 slots=1 check= typed= preflight= + local capture='' shift 3 - if [ "${1:-}" = "--containerized-e2e-build" ]; then typed=1; shift; fi + while :; do + case "${1:-}" in + --containerized-e2e-build) typed=1; shift ;; + --capture) { [ -n "${2:-}" ] && [ -f "$2" ]; } || usage; capture=$2; shift 2 ;; + *) break ;; + esac + done [ $# -ge 1 ] || usage + if [ -n "$capture" ]; then capture_begin "$capture" "$dir" "$owner" || exit; fi if [ -n "$typed" ] || node "$skills/vibe/scripts/e2e-lock.mjs" intent "$@" >/dev/null; then preflight=1; fi + if [ -n "$capture" ]; then + # The command writes its own run time once the slot admits it, so the measured rate excludes the queue wait. + if [ "$where" = local ]; then + capture_timing=$(mktemp "${TMPDIR:-/tmp}/fleet-capture-time.XXXXXX") + else + capture_timing="$remote_dir/capture-time-$$-$RANDOM" + capture_remote_pid="$remote_dir/capture-pid-$$-$RANDOM" + fi + # shellcheck disable=SC2016 # expanded by the inner bash, inside the slot + # The command runs in the background so a TERM, INT or HUP sent to this wrapper (slot-lock's release) reaches it. + set -- bash -c 't=$(date +%s); if [ -n "$2" ]; then echo $$ > "$2"; fi; "${@:3}" <&0 & c=$!; trap "kill -TERM $c 2>/dev/null" TERM INT HUP; wait "$c"; r=$?; while kill -0 "$c" 2>/dev/null; do wait "$c"; r=$?; done; echo $(( $(date +%s) - t )) > "$1"; exit $r' capture-timing "$capture_timing" "$capture_remote_pid" "$@" + fi if [ "$name" = heavy ]; then if [ "$where" = local ]; then slots=${HEAVY_SLOTS:-1}; else slots=${REMOTE_HEAVY_SLOTS:-1}; fi check=1 @@ -322,9 +398,23 @@ slot() { local slot_args=(--slots "$slots") if [ -n "$admission_note" ]; then slot_args+=(--admission-note "$admission_note"); fi if [ -n "$preflight" ]; then slot_args+=(--preflight-node "$skills/vibe/scripts/e2e-lock.mjs"); fi - exec bash "$here/slot-lock.sh" "${slot_args[@]}" "$name" "$owner" "$@" + if [ -z "$capture" ]; then exec bash "$here/slot-lock.sh" "${slot_args[@]}" "$name" "$owner" "$@"; fi + # Kept as a child so a signal to this script stops slot-lock (which stops the command and frees the slot) before the entry is recorded failed. + bash "$here/slot-lock.sh" "${slot_args[@]}" "$name" "$owner" "$@" <&0 & + local slot_pid=$! + trap 'kill -TERM "$slot_pid" 2>/dev/null; wait "$slot_pid"; exit 143' TERM + trap 'kill -TERM "$slot_pid" 2>/dev/null; wait "$slot_pid"; exit 130' INT + trap 'kill -TERM "$slot_pid" 2>/dev/null; wait "$slot_pid"; exit 129' HUP + wait "$slot_pid" + return + fi + local gate='' remote_line='' remote_pid_part='' + local remote_pid_first='' + if [ -n "$capture" ]; then + # The pid is recorded before the gate and the load check, so a stop reaches every stage; the marker covers the rest. + remote_pid_first="echo \$\$ > $(printf %q "$capture_remote_pid"); " + remote_pid_part="[ ! -e $(printf %q "$capture_remote_pid.stop") ] && exec " fi - local gate= if [ -n "$check" ]; then gate="MAX_LOAD_PER_CORE=$(printf %q "${REMOTE_MAX_LOAD_PER_CORE:-${MAX_LOAD_PER_CORE:-1.5}}") FLEET_FORCE_HERE=$(printf %q "${FLEET_FORCE_HERE:-}") FLEET_FORCE_REASON=$(printf %q "${FLEET_FORCE_REASON:-}") bash $(printf %q "$remote_dir/plugins/vibe/skills/fleet-orchestrator/scripts/load-check.sh") && "; fi local slot_args=(--slots "$slots") if [ -n "$admission_note" ]; then slot_args+=(--admission-note "$admission_note"); fi @@ -333,10 +423,17 @@ slot() { expected_version=$(jq -er '.version' "$skills/../.codex-plugin/plugin.json") remote_root='$(codex plugin list --json | jq -er '\''.installed[] | select(.name == "vibe" and .installed == true and .enabled == true) | .source.path'\'' | head -n 1)' remote_gate="command -v codex >/dev/null 2>&1 && remote_root=$remote_root && test -n \"\$remote_root\" && test \"\$(jq -er .version \"\$remote_root/.codex-plugin/plugin.json\")\" = $(printf %q "$expected_version") || { printf 'fleet: remote vibe plugin is missing or not version %s; heavy build not started\\n' $(printf %q "$expected_version") >&2; exit 78; }; " - run_remote "${remote_gate}cd $(printf %q "$dir") && $gate$(printf '%q ' bash "$remote_dir/plugins/vibe/skills/fleet-orchestrator/scripts/slot-lock.sh" "${slot_args[@]}")--preflight-node \"\$remote_root/skills/vibe/scripts/e2e-lock.mjs\" $(printf '%q ' "$name" "$owner" "$@")" + remote_line="${remote_pid_first}${remote_gate}cd $(printf %q "$dir") && $gate${remote_pid_part}$(printf '%q ' bash "$remote_dir/plugins/vibe/skills/fleet-orchestrator/scripts/slot-lock.sh" "${slot_args[@]}")--preflight-node \"\$remote_root/skills/vibe/scripts/e2e-lock.mjs\" $(printf '%q ' "$name" "$owner" "$@")" else - run_remote "cd $(printf %q "$dir") && $gate$(printf '%q ' bash "$remote_dir/plugins/vibe/skills/fleet-orchestrator/scripts/slot-lock.sh" "${slot_args[@]}" "$name" "$owner" "$@")" + remote_line="${remote_pid_first}cd $(printf %q "$dir") && $gate${remote_pid_part}$(printf '%q ' bash "$remote_dir/plugins/vibe/skills/fleet-orchestrator/scripts/slot-lock.sh" "${slot_args[@]}" "$name" "$owner" "$@")" fi + if [ -z "$capture" ]; then run_remote "$remote_line"; return; fi + # Kept as a child so a signal here stops the remote slot (slot-lock stops the command and frees the slot) and the entry + # records the remote exit: failed only when the command was stopped. + run_remote "$remote_line" <&0 & + local remote_job=$! + trap 'capture_stop_remote; wait "$remote_job"; exit $?' TERM INT HUP + wait "$remote_job" } cleanup() { diff --git a/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.test.mjs b/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.test.mjs index 8837e6e9..b69ed502 100644 --- a/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.test.mjs +++ b/plugins/vibe/skills/fleet-orchestrator/scripts/fleet.test.mjs @@ -632,3 +632,250 @@ test('a heavy slot refuses a machine whose load per core is over the limit and n assert.match(skill, /Writers never set `FLEET_FORCE_HERE`[\s\S]*or `FLEET_LOAD`/); assert.doesNotMatch(readFileSync(LOAD_CHECK, 'utf8'), /FLEET_LOAD/, 'production admission has no caller-supplied load override'); }); + +/** A fleet fixture whose node runs capture-guard.mjs for real, with a capture worktree cloned from a bare origin. */ +function captureFixture() { + const f = fixture(); + writeFileSync(join(f.root, 'bin', 'node'), `#!/usr/bin/env bash\ncase "$1" in *ship-plugins.mjs|*work-dir.mjs|*skill-freshness.mjs|*owned-state.mjs|*fleet-remote-evidence.mjs|*capture-guard.mjs) exec ${JSON.stringify(process.execPath)} "$@";; *e2e-lock.mjs) [ "$2" = intent ] && exec ${JSON.stringify(process.execPath)} "$@";; esac\nprintf "arg:%s\\n" "$@"\n`); + chmodSync(join(f.root, 'bin', 'node'), 0o755); + const git = (cwd, ...args) => { + const result = spawnSync('git', ['-C', cwd, ...args], { encoding: 'utf8' }); + assert.equal(result.status, 0, result.stderr); + return result.stdout.trim(); + }; + git(f.root, 'init', '-q', '--bare', '-b', 'main', 'origin.git'); + git(f.root, 'clone', '-q', join(f.root, 'origin.git'), 'wt'); + const wt = join(f.root, 'wt'); + writeFileSync(join(wt, 'file.txt'), 'one\n'); + git(wt, 'add', 'file.txt'); + git(wt, '-c', 'user.name=t', '-c', 'user.email=fixture.invalid', 'commit', '-q', '-m', 'one'); + git(wt, 'push', '-q', 'origin', 'main'); + let n = 0; + const spec = (fields) => { + const file = join(f.root, `spec-${n += 1}.json`); + writeFileSync(file, JSON.stringify({ ticket: 'ISS-1', ref: 'origin/main', pages: 2, states: 2, ...fields })); + return file; + }; + const entries = () => JSON.parse(readFileSync(join(f.orch, 'captures', 'ISS-1.json'), 'utf8')).captures; + return { ...f, wt, spec, entries, git }; +} + +test('AC-3 a local --capture slot refuses a main ref already captured for the ticket before any slot or command', TIMEOUT, () => { + const f = captureFixture(); + const first = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'touch first-ran']); + assert.equal(first.status, 0, first.stderr); + assert.ok(existsSync(join(f.wt, 'first-ran')), 'the admitted capture ran in its worktree'); + assert.deepEqual(f.entries().map((entry) => [entry.status, entry.main, entry.owner]), [['ok', true, 'w1 shots']]); + const again = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'touch second-ran'], + { FLEET_TEST_LOAD: '1000' }); + assert.equal(again.status, 79, again.stderr); + assert.doesNotMatch(again.stderr, /load-check/, 'the guard refuses before the load check runs'); + assert.match(again.stderr, /^capture-guard: refused: [0-9a-f]{40} is a main commit already captured for ISS-1; reuse that baseline\./); + assert.equal(existsSync(join(f.wt, 'second-ran')), false, 'a refused capture never runs'); + assert.equal(f.entries().length, 1, 'a refused capture records nothing'); + const undeclared = f.run(['local', 'slot', 'heavy', 'w1 tests', f.wt, 'bash', '-c', 'touch undeclared-ran']); + assert.equal(undeclared.status, 0, undeclared.stderr); + assert.ok(existsSync(join(f.wt, 'undeclared-ran')), 'a slot without --capture is unchanged'); +}); + +test('a failed --capture command is recorded failed and never blocks the retry', TIMEOUT, () => { + const f = captureFixture(); + const failed = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'exit 3']); + assert.equal(failed.status, 3, failed.stderr); + assert.deepEqual(f.entries().map((entry) => entry.status), ['failed']); + const retry = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'true']); + assert.equal(retry.status, 0, retry.stderr); + assert.deepEqual(f.entries().map((entry) => entry.status), ['failed', 'ok']); +}); + +test('AC-4 a remote --capture slot refuses an over-budget capture before the remote slot runs', TIMEOUT, () => { + const f = captureFixture(); + writeFileSync(join(f.orch, 'config.env'), remoteConfig(f.root)); + const over = f.run(['remote', 'slot', 'heavy', 'w2 shots', f.wt, '--capture', f.spec({ pages: 25, states: 10 }), 'bash', '-c', 'touch remote-ran']); + assert.equal(over.status, 79, over.stderr); + assert.match(over.stderr, /capture-guard: refused: 25 pages x 10 states is 250 shots, over the 240 shot budget for one capture/); + assert.equal(existsSync(join(f.wt, 'remote-ran')), false, 'a refused capture never reaches the remote slot'); + const lastRemoteCall = readFileSync(join(f.root, 'ssh-args'), 'utf8'); + assert.match(lastRemoteCall, /capture-guard\.mjs(?:\\)? facts/); + assert.doesNotMatch(lastRemoteCall, /slot-lock\.sh/); + const admitted = f.run(['remote', 'slot', 'heavy', 'w2 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'touch remote-ran']); + assert.equal(admitted.status, 0, admitted.stderr); + assert.ok(existsSync(join(f.wt, 'remote-ran'))); + assert.deepEqual(f.entries().map((entry) => [entry.status, entry.main]), [['ok', true]], 'the program ledger on the orchestrator host records the remote capture'); + assert.ok(Number.isInteger(f.entries()[0].seconds), 'the remote command time comes back'); + const rebaseline = f.run(['remote', 'slot', 'heavy', 'w2 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'true']); + assert.equal(rebaseline.status, 79, rebaseline.stderr); +}); + +test('AC-4 a fleet capture records the seconds its command ran, and the measured rate refuses a later one over 20 minutes', { timeout: 40_000 }, () => { + const f = captureFixture(); + f.git(f.wt, 'checkout', '-q', '-b', 'feature'); + writeFileSync(join(f.wt, 'branch.txt'), 'branch\n'); + f.git(f.wt, 'add', 'branch.txt'); + f.git(f.wt, '-c', 'user.name=t', '-c', 'user.email=fixture.invalid', 'commit', '-q', '-m', 'branch'); + const slow = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({ ref: 'HEAD', pages: 1, states: 1 }), 'bash', '-c', 'sleep 6']); + assert.equal(slow.status, 0, slow.stderr); + const [entry] = f.entries(); + assert.equal(entry.status, 'ok'); + assert.ok(Number.isInteger(entry.seconds) && entry.seconds >= 6, `seconds recorded: ${entry.seconds}`); + const longer = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({ ref: 'HEAD', pages: 21, states: 10 }), 'bash', '-c', 'touch longer-ran']); + assert.equal(longer.status, 79, longer.stderr); + assert.match(longer.stderr, /capture-guard: refused: 210 shots at \d+ s each is about \d+ minutes, over the 20 minute budget for one capture/); + assert.equal(existsSync(join(f.wt, 'longer-ran')), false); +}); + +/** Starts `fleet.sh ` like fixture's run, but asynchronously, with its exit as a promise. */ +function startFleet(f, args, env = {}) { + const child = spawn('bash', [FLEET, f.orch, ...args], { + env: { ...process.env, PATH: `${join(f.root, 'bin')}:${process.env.PATH}`, SLOT_LOCK_DIR: join(f.root, 'locks'), ...env }, + }); + let stderr = ''; + child.stderr.on('data', (chunk) => { stderr += chunk; }); + const exited = new Promise((resolve) => child.on('exit', (code, signal) => resolve({ code, signal, stderr }))); + return { child, exited }; +} + +async function waitFor(check, ms = 15_000) { + const deadline = Date.now() + ms; + while (!check()) { + if (Date.now() > deadline) throw new Error('timed out waiting'); + await new Promise((resolve) => setTimeout(resolve, 100)); + } +} + +function alive(pid) { + try { process.kill(pid, 0); return true; } catch (error) { return error.code === 'EPERM'; } +} + +test('a SIGTERM to a --capture slot stops its command, frees the slot and records the capture failed', { timeout: 40_000 }, async () => { + const f = captureFixture(); + const pidFile = join(f.root, 'command.pid'); + const run = startFleet(f, ['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', `echo $$ > ${pidFile}; sleep 30`]); + await waitFor(() => existsSync(pidFile) && readFileSync(pidFile, 'utf8').trim() !== ''); + const commandPid = Number(readFileSync(pidFile, 'utf8').trim()); + assert.ok(commandPid > 1 && run.child.pid > 1, 'only processes this test started are signalled'); + const signalled = Date.now(); + run.child.kill('SIGTERM'); + const { code } = await run.exited; + assert.equal(code, 143); + assert.ok(Date.now() - signalled < 10_000, 'the slot stops promptly, not when the command would have finished'); + await waitFor(() => !alive(commandPid), 10_000); + assert.equal(existsSync(join(f.root, 'locks', 'heavy-1.lock')), false, 'the slot is freed'); + assert.deepEqual(f.entries().map((entry) => entry.status), ['failed']); +}); + +test('AC-3 AC-4 a fleet capture is overridden only by a ruling in the program rulings.md', { timeout: 30_000 }, () => { + const f = captureFixture(); + assert.equal(f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'true']).status, 0); + const repeat = f.spec({ override: { quote: 'retake the baseline capture' } }); + const refusedRun = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', repeat, 'bash', '-c', 'true']); + assert.equal(refusedRun.status, 79, 'no rulings.md line yet'); + writeFileSync(join(f.orch, 'rulings.md'), '- Ruling 2026-11-13 (operator): "retake the baseline capture" ISS-1 capture-guard override: baseline\n'); + const allowed = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', repeat, 'bash', '-c', 'true']); + assert.equal(allowed.status, 0, allowed.stderr); + assert.equal(f.entries().at(-1).override.rulings, 'rulings.md'); +}); + +test('AC-4 the recorded seconds exclude the time the capture waited for the slot', { timeout: 40_000 }, async () => { + const f = captureFixture(); + const holder = startFleet(f, ['local', 'slot', 'heavy', 'w0 holder', f.root, 'sleep', '8'], { SLOT_LOCK_POLL_SECONDS: '0.2' }); + await waitFor(() => existsSync(join(f.root, 'locks', 'heavy-1.lock', 'owner'))); + const started = Date.now(); + const queued = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', 'true'], { SLOT_LOCK_POLL_SECONDS: '0.2' }); + assert.equal(queued.status, 0, queued.stderr); + const waited = (Date.now() - started) / 1000; + await holder.exited; + const [entry] = f.entries(); + assert.ok(waited >= 4, `the capture queued behind the holder (${waited} s)`); + assert.ok(Number.isInteger(entry.seconds) && entry.seconds < waited - 2, `recorded ${entry.seconds} s against a ${waited} s wait`); +}); + +test('a SIGTERM to a remote --capture slot stops the remote command, frees its slot and records the running capture failed', { timeout: 40_000 }, async () => { + const f = captureFixture(); + writeFileSync(join(f.orch, 'config.env'), remoteConfig(f.root)); + const pidFile = join(f.root, 'remote-command.pid'); + const run = startFleet(f, ['remote', 'slot', 'heavy', 'w2 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', `echo $$ > ${pidFile}; sleep 30`]); + await waitFor(() => existsSync(pidFile) && readFileSync(pidFile, 'utf8').trim() !== ''); + const commandPid = Number(readFileSync(pidFile, 'utf8').trim()); + assert.ok(commandPid > 1 && run.child.pid > 1, 'only processes this test started are signalled'); + assert.deepEqual(f.entries().map((entry) => entry.status), ['running'], 'running while the remote command is alive'); + const signalled = Date.now(); + run.child.kill('SIGTERM'); + const { code } = await run.exited; + assert.ok(Date.now() - signalled < 15_000, 'the remote slot stops promptly, not when the command would have finished'); + assert.notEqual(code, 0); + await waitFor(() => !alive(commandPid), 10_000); + assert.equal(existsSync(join(f.root, 'locks', 'heavy-1.lock')), false, 'the remote slot is freed'); + assert.deepEqual(f.entries().map((entry) => entry.status), ['failed']); +}); + +test('a SIGTERM before the remote slot starts keeps it from starting and records the capture failed', { timeout: 40_000 }, async () => { + const f = captureFixture(); + writeFileSync(join(f.orch, 'config.env'), remoteConfig(f.root)); + writeFileSync(join(f.root, 'bin', 'ssh'), `#!/usr/bin/env bash\ncase "\${@: -1}" in *slot-lock.sh*) sleep 3 ;; esac\nexec bash -c "\${@: -1}"\n`); + chmodSync(join(f.root, 'bin', 'ssh'), 0o755); + const ran = join(f.root, 'remote-ran'); + const run = startFleet(f, ['remote', 'slot', 'heavy', 'w2 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', `touch ${ran}; sleep 30`]); + await waitFor(() => existsSync(join(f.orch, 'captures', 'ISS-1.json')) && f.entries().length === 1); + await new Promise((resolve) => setTimeout(resolve, 500)); + assert.ok(run.child.pid > 1); + const signalled = Date.now(); + run.child.kill('SIGTERM'); + const { code } = await run.exited; + assert.notEqual(code, 0); + assert.ok(Date.now() - signalled < 15_000, 'no wait for a capture that never started'); + await new Promise((resolve) => setTimeout(resolve, 1000)); + assert.equal(existsSync(ran), false, 'the remote command never started'); + assert.deepEqual(f.entries().map((entry) => entry.status), ['failed']); + const remoteDir = join(f.root, 'remote', 'sessions', '.fleet-orchestrator'); + assert.deepEqual(readdirSync(remoteDir).filter((name) => name.startsWith('capture-')), [], 'stop marker, pid and timing files are cleaned'); +}); + +test('a remote capture whose command ignores the stop and exits 0 is recorded ok', { timeout: 40_000 }, async () => { + const f = captureFixture(); + writeFileSync(join(f.orch, 'config.env'), remoteConfig(f.root)); + const pidFile = join(f.root, 'stubborn.pid'); + const run = startFleet(f, ['remote', 'slot', 'heavy', 'w2 shots', f.wt, '--capture', f.spec({}), 'bash', '-c', `trap '' TERM; echo $$ > ${pidFile}; sleep 2; exit 0`]); + await waitFor(() => existsSync(pidFile) && readFileSync(pidFile, 'utf8').trim() !== ''); + assert.ok(run.child.pid > 1); + run.child.kill('SIGTERM'); + const { code } = await run.exited; + assert.equal(code, 0, 'the real remote exit is recorded, not an assumed failure'); + assert.deepEqual(f.entries().map((entry) => entry.status), ['ok']); +}); + +test('a --capture spec that is missing or unreadable is a usage error before any slot or command', TIMEOUT, () => { + const f = captureFixture(); + const ran = join(f.wt, 'spec-ran'); + // A load over the gate makes any slot path that runs first exit 4 with a load-check line instead of the usage error. + const missing = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', join(f.root, 'no-such-spec.json'), 'bash', '-c', `touch ${ran}`], + { FLEET_TEST_LOAD: '1000' }); + assert.equal(missing.status, 2, missing.stderr); + assert.match(missing.stderr, /slot \[--containerized-e2e-build\] \[--capture \] /); + assert.doesNotMatch(missing.stderr, /load-check|slot-lock/, 'a missing spec is refused before the load check and the slot'); + const unreadable = f.spec({}); + chmodSync(unreadable, 0o000); + const locked = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, '--capture', unreadable, 'bash', '-c', `touch ${ran}`], + { FLEET_TEST_LOAD: '1000' }); + chmodSync(unreadable, 0o600); + assert.equal(locked.status, 2, locked.stderr); + assert.match(locked.stderr, /^capture-guard: invalid spec$/m); + assert.doesNotMatch(locked.stderr, /load-check|slot-lock/, 'an unreadable spec is refused before the load check and the slot'); + assert.equal(existsSync(ran), false, 'no command runs'); + assert.equal(existsSync(join(f.root, 'locks')), false, 'slot-lock never started: it creates the lock root on every start'); + assert.equal(existsSync(join(f.orch, 'captures', 'ISS-1.json')), false, 'nothing is recorded'); +}); + +test('--capture and --containerized-e2e-build are accepted in either order', TIMEOUT, () => { + for (const flags of [['--capture', null, '--containerized-e2e-build'], ['--containerized-e2e-build', '--capture', null]]) { + const f = captureFixture(); + const spec = f.spec({}); + const args = flags.map((flag) => flag ?? spec); + const ran = join(f.wt, 'ordered-ran'); + const result = f.run(['local', 'slot', 'heavy', 'w1 shots', f.wt, ...args, 'bash', '-c', `touch ${ran}`]); + assert.equal(result.status, 0, `${args.join(' ')}: ${result.stderr}`); + assert.ok(existsSync(ran), `${args.join(' ')}: the command ran`); + assert.ok(f.argv(result).some((arg) => arg.endsWith('e2e-lock.mjs')), `${args.join(' ')}: the disk preflight ran`); + assert.deepEqual(f.entries().map((entry) => [entry.status, entry.main]), [['ok', true]], `${args.join(' ')}: the guard admitted and recorded it`); + } +}); diff --git a/plugins/vibe/skills/handoff/SKILL.md b/plugins/vibe/skills/handoff/SKILL.md index 6c40dbc3..017ff34d 100644 --- a/plugins/vibe/skills/handoff/SKILL.md +++ b/plugins/vibe/skills/handoff/SKILL.md @@ -535,18 +535,15 @@ change, against the session's acceptance criteria when the ticket or its PRD has them and otherwise as the design pack's best-practice review (`../vibe/references/quality-loop.md`, "Visual QA during the session", "What it checks"); its rendered-control inventory flags anything with no -sourced requirement. For shared scope it compares each consumer page against -that page's approved prototype route and flags every control present in the -prototype but missing in production. Run logical QA (`/logical-qa`, `$logical-qa`) for a data or metric +sourced requirement. Run logical QA (`/logical-qa`, `$logical-qa`) for a data or metric surface, then product-contract QA (`/metal-andrew`, `$metal-andrew`) once that -evidence is visible. Each skill dispatches its own read-only agents. For every -touched metric, summary, or card surface, visual QA compares the card's -container, width, order and responsive placement with the established grid or -that page's official prototype story. Counting cards is never layout evidence. +evidence is visible. Each skill dispatches its own read-only agents. Confirmed findings resume the SAME writer in fix mode for every touched surface, even when they predate the branch, already exist on main or were called the component owner's decision; approved copy is reused and unsourced wording is escalated. Then -steps 3 to 6 run again on the result. Before handoff, the findings file must pass +steps 3 to 6 run again on the result, with visual QA as one pass scoped to +the changes since the last reviewed capture +(`../vibe/references/quality-loop-visual-qa.md`, "When to capture screenshots"). Before handoff, the findings file must pass `node ../fleet-orchestrator/scripts/finding-brief.mjs --findings --require-disposition`; every finding on a touched surface has `fixed: ` or a follow-up `ticket: `. Their reports feed the Design Review evidence in step 8. Owned diff --git a/plugins/vibe/skills/handoff/references/handoff-ui.md b/plugins/vibe/skills/handoff/references/handoff-ui.md index e0dcd07b..ef9f894d 100644 --- a/plugins/vibe/skills/handoff/references/handoff-ui.md +++ b/plugins/vibe/skills/handoff/references/handoff-ui.md @@ -51,8 +51,7 @@ what each proves. A static or generated page control cannot drive (an HTML page a script or a main-process module writes, such as Desktop's sign-in return page) is rendered directly instead: the verify worker -produces its HTML with the repo's own generator, opens it headless and -captures it in light and dark wherever it supports both. "Could not drive" is +produces its HTML with the repo's own generator. "Could not drive" is not accepted for such a page. Any other surface it can neither drive nor render comes back with the exact control command and the reason, never as verified. A console error, page error, failed request or diff --git a/plugins/vibe/skills/vibe/references/quality-loop-visual-qa.md b/plugins/vibe/skills/vibe/references/quality-loop-visual-qa.md index fd0df52e..1d783558 100644 --- a/plugins/vibe/skills/vibe/references/quality-loop-visual-qa.md +++ b/plugins/vibe/skills/vibe/references/quality-loop-visual-qa.md @@ -79,3 +79,34 @@ when the person already declined visual QA for this version of the change since); then the request goes to the writer as an ordinary request under `guardrails-ui.md`, "Making something look better". + + +### When to capture screenshots + +Screenshots were never a standing requirement, and nothing here adds one. + +- Default: one visual-qa reviewer reads the diff, plus the Storybook stories + where they exist. That covers most UI changes: a new control, copy reuse, + styling, a single page or component. +- Capture screenshots only when: (a) a shared component that renders on + several pages changes its pixels: capture the affected pages once, on the + branch, against one main baseline taken at the start; (b) the ticket's + acceptance criteria or an operator ruling ask for visual proof, such as the + PRD grouped-unit per-page check (ruling 2026-11-10); (c) the visual-qa + reviewer cannot settle a finding from code and stories: then take targeted + shots of just that state, never a sweep. +- Limits when screenshots are justified: one main baseline per ticket, never + retaken because main moved. After a fix, recapture only the states the fix + can reach. A final check is never a new capture: it is one visual-qa pass + scoped to the changes since the last reviewed capture. + +Merging or rebasing onto main never triggers a re-baseline capture; +differences that come from main are not reviewed as the ticket's. + +Before any capture of more than one state, run +`scripts/capture-guard.mjs begin`, and `end` with the capture's exit status +after it. It refuses a main commit already captured +for the ticket, a second main baseline, and a capture over 240 shots or about +20 minutes, unless an operator ruling recorded for that ticket overrides it. +It only refuses; it never asks for a capture. + diff --git a/plugins/vibe/skills/vibe/references/quality-loop.md b/plugins/vibe/skills/vibe/references/quality-loop.md index 08ee0fe4..fac6211e 100644 --- a/plugins/vibe/skills/vibe/references/quality-loop.md +++ b/plugins/vibe/skills/vibe/references/quality-loop.md @@ -762,6 +762,8 @@ and the ready message names it as not done yet. > Read `references/quality-loop-visual-qa.md` (block `visual-qa-body`) when a user-visible change is ready to look at or the person asks for polish; skip it when the change shows nothing visible. +> Read `references/quality-loop-visual-qa.md` (block `when-to-capture`) before any screenshot capture or visual QA pass, in the loop or at handoff. + ## Tests: repair, don't author No test authoring in the build loop or at handoff without an explicit top-level grant, including new diff --git a/plugins/vibe/skills/vibe/scripts/capture-guard.mjs b/plugins/vibe/skills/vibe/scripts/capture-guard.mjs new file mode 100644 index 00000000..e7d632ae --- /dev/null +++ b/plugins/vibe/skills/vibe/scripts/capture-guard.mjs @@ -0,0 +1,547 @@ +#!/usr/bin/env node +// Refuses a screenshot capture that repeats a ticket's main baseline or is too +// big (ISS-14158). One ticket's proof scripts once ran more than 30 capture +// runs through the fleet heavy slot in a day, re-shooting main after every +// main merge; one run was 1632 shots. This guard only ever refuses a capture; +// nothing here asks for one (`../references/quality-loop-visual-qa.md`, block +// `when-to-capture`, says when a capture is justified). +// +// Usage: +// capture-guard.mjs facts --repo --ref +// capture-guard.mjs begin --spec (--repo | --facts ) +// [--ledger ] [--owner