Skip to content

ISS-13724: Admit an e2e suite's support modules - #439

Open
cchenault4 wants to merge 1 commit into
mainfrom
fix/iss-13724-root-e2e-test-support
Open

cchenault4 wants to merge 1 commit into
mainfrom
fix/iss-13724-root-e2e-test-support

Conversation

@cchenault4

Copy link
Copy Markdown
Contributor

Purpose

The vibe handoff inventory rejected a Desktop E2E helper that an admitted PR spec imports, so a target repository needed an exact-file allowedFiles entry to get it through. This replaces that workaround with the root-E2E test-support classification the issue asked for.

A suite keeps its helpers in a folder it names itself (helpers/, perf/, fixtures/), and isTestFile recognises only __tests__/fixtures-style folders. So a helpers/ module reached neither test branch of classifyHandoffPath: it fell through to ordinary source and was admitted only when an allowedPrefixes entry happened to cover it. isE2ePath already existed and was exported; the classifier never called it.

What changed

  • skills/vibe/scripts/test-files.mjs — adds isE2eTestSupport: a non-spec, non-snapshot module in a folder under an e2e suite root. The suite root itself is excluded, because that is where the harness lives (playwright.config.ts, auth.setup.ts, the compose files), and editing the harness is not a test repair.
  • skills/handoff/scripts/path-classifier.mjs — classifies an e2e suite's support with the suite's specs, so a helper gets the disposition the spec that imports it gets: pull-request-test when added in the pull-request phase, test-not-admitted in the build phase, test-repair when edited.
  • skills/handoff/scripts/handoff-inventory.mjs — its header said only root specs are admitted in the pull-request phase, which the above makes wrong.
  • Tests — test-files.test.mjs covers the predicate; path-classifier.test.mjs covers the classification: the consuming spec, a module outside every suite root, a forbidden prefix, a snapshot, and the suite root's own harness files.
  • plugins/vibe-closedloop/ — rebuilt from tools/vibe-closedloop-build; manifests bumped to 1.1.102 in both .claude-plugin and .codex-plugin; CHANGELOG.md entry added via the repo's documentation workflow format.

Affected plugins: vibe (source) and vibe-closedloop (generated output).

Why the suite root is excluded

My first attempt was "under an e2e root, not a spec, not a snapshot", and handoff-inventory.test.mjs's existing case — "a test repair never admits a harness or a path the session may not edit" — caught it: that predicate swallowed e2e/playwright.config.ts into testRepairs. The leak was wider than the one file the test named; the real suite roots also hold auth.setup.ts, compose*.yml, Dockerfile.db-tools and *.sh, all of which would have become editable test support. Support lives in subdirectories, which is where the originally-allowlisted helpers are, so that is the rule.

Note for review

Widening the test branch means an edited e2e support module now classifies test-repair rather than allowed-path, which puts it in the inventory's testRepairs list for the guardrails reviewer. Admission is unchanged (both are Allowed), and a helper carries no assertions for that reviewer to flag, so the practical impact should be nil — but it is a real change to a second gate's input, and it follows from the issue's "keep admission and test classification consistent" clause rather than being asked for directly.

Commands run

node --test <19 suites importing test-files.mjs or path-classifier.mjs>
  → 358 tests, 358 pass, 0 fail, 0 cancelled
  (includes handoff-inventory.test.mjs and pr-facts.test.mjs)
python3 plugins/vibe/tools/python/vibe_generic_scan.py check
  → vibe runtime files are generic
node tools/vibe-closedloop-build/build.mjs
  → built 305 files; only the expected files differ

Counterfactual: reverting only the source change makes the new positive classification test fail, while the boundary test (forbidden prefix, snapshot, harness) passes either way — it guards against over-reach rather than proving the change.

CI's node-tests job runs all 209 plugins/**/*.test.* files; I ran the 19 that import the changed modules, per this repo's "targeted test runs for changed plugin paths".

🤖 Generated with Claude Code

https://claude.ai/code/session_01PRK1nj8soNurp47ZRpLvwX

- Add isE2eTestSupport: a non-spec, non-snapshot module in a FOLDER
  under an e2e suite root. The suite root itself is excluded, because
  that is where the harness lives (playwright config, auth setup,
  compose files), and editing the harness is not a test repair.
- Classify those modules with the suite's specs in
  classifyHandoffPath, so a helper gets the same disposition as the
  spec that imports it instead of falling through to source and
  needing a broad allowedPrefixes entry or an exact-file allowlist.
- Correct the inventory header, which said only root specs are
  admitted in the pull-request phase.
- Cover the predicate in test-files.test.mjs and the classification in
  path-classifier.test.mjs: the consuming spec, a module outside every
  suite root, a forbidden prefix, a snapshot, and the suite root's own
  harness files.
- Move vibe to 1.1.102: both manifests, the README version line, the
  two Python release-pin contract tests and the vibe-bench pin, then
  rebuild plugins/vibe-closedloop from the build tool.
- Add the CHANGELOG entry with its measured word count. This change
  edits no plugin Markdown, so the net is 0 across `--path plugins`.

Testing: node --test on the 19 suites that import test-files.mjs or
path-classifier.mjs (358 tests, all passing, including
handoff-inventory and pr-facts) plus instruction-words (5/5);
uv run pytest on the two release-pin contract tests (30 passed);
vibe-bench typecheck and validate:plugins; instruction-words.mjs
--changelog --version 1.1.102 exits 0 against the measured net;
vibe_generic_scan.py check clean; the vibe-closedloop build reproduces
with only the expected files changed.
Risks: an edited e2e support module now classifies test-repair rather
than allowed-path, so it appears in the inventory's testRepairs list
for the guardrails reviewer. Admission is unchanged (both Allowed), and
a helper carries no assertions for that reviewer to flag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRK1nj8soNurp47ZRpLvwX
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant