Repository navigation
test(cli): the restore guard covers the test files bun runs (#603) - #625
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
Claim sweep — cli#603 (
|
…estructured property's identity, and parses TSX Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s not treated as that global Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
tps-kern
left a comment
There was a problem hiding this comment.
APPROVE — head cd9ebed3, reviewed for architecture/verification. Repo visibility checked before writing: tpsdev-ai/cli is public (via repos/tpsdev-ai/cli .visibility).
Discovery matches what Bun actually runs — verified empirically
I ran this host's Bun 1.3.10 (the same version this repo pins) against a sandbox tree containing every form the shared rule [scripts/check-test-reports.mjs TEST_FILE_NAME:137, discoverTestFiles:147] admits and every form it rejects: bun ran exactly the 13 admitted files (.test/_test/.spec/_spec × .ts/.tsx/.js/.jsx/.mjs/.cjs/.mts/.cts, dot-FILES admitted, dot-dirs and node_modules skipped, dist/ walked) and none of the rejected forms (foo-spec.ts, foo.test.helper.ts, plain.ts, dot-dirs). Pattern-for-pattern and exclusion-for-exclusion, measured — not inferred from comments.
discoverCliTestFiles[packages/cli/test/helpers/mock-restore-guard-scan.ts:528] now delegates to that same shared function over the resolved cli root, anddiscoverTestFilesthrows on any unreadable directory — the guard fails closed on a partial walk.- Backstops: the forms test [mock-restore-guard.test.ts:39] pins the exact 13-form list; the loaded-set assertion [:58] fails a run when a file the preload recorded (i.e. one bun actually loaded) is outside the scanned set — its stated limit (files loaded after the assertion runs) is acknowledged in the PR; relative/absolute-root equivalence is pinned [:131].
- Census: the guard scans 207 files under packages/cli today, all currently
.test.ts— the widened rule is forward coverage for the day a.spec/.tsxfile lands. - Mutation I ran: re-narrowing discovery to
.test.tsfails exactly the forms test (1 failure); restore verified, tree clean.
Scope resolution — no case found that reports a shadowed name or misses an outer alias
resolveScope [:208] walks to the nearest enclosing scope that declares the name; guarded-root aliases and module objects are keyed by declaring scope (moduleObjectAt [:241], guardedRootOf). Probes I ran through analyzeSource beyond the fixtures: a shadow nested two functions deep clears (resolves to the param, not the module-scope alias); an outer alias is reported through two nested scopes; a block-hoisted var alias is reported; a value-imported name shadowing a guarded root (import { process }) is reported as a module patch. The new fixtures pin the boundary cases: outer-alias leaked [module-alias-outer-scope-leaked], param shadows cleared for both module objects and their aliases [module-shadow-param-cleared, module-alias-shadow-cleared], loop-var reported both after the loop and from a nested function (the For-scope BlockScoped fix), process.env extraction exempt at every nesting (never an alias), globalThis-rooted destructures selecting process reported.
- Mutation I ran: flattening
moduleObjectAtto ignore scope fails exactly the two cleared-fixture tests (2 failures — matching the PR's own evidence for the same mechanism); restore verified.
Per-write assertions (verified by reading)
The multi-write red fixtures assert exact detail arrays (toEqual), so each exempt write's silence and each leaked write's finding are pinned individually — a dropped finding fails and a spurious one fails. Single-write reds assert the finding kind, which write-removal also kills. Matches the PR's claim.
Registry sanity (Sherlock's lane)
createPatchRegistry [packages/cli/test/helpers/patch-shared.ts] restores LIFO everything patched before cleanup and sets the cleanedUp flag before restoring; a patch requested afterwards throws PatchAfterCleanupError before savePatch runs — a post-cleanup patch is never applied and cannot leak. The throw is synchronous at call time, so an async caller surfaces it as a rejection. The two redundant late re-patches removed from agent-seed.test.ts / memory-learn.test.ts are real-world cases of exactly this refusal.
Tests run on this head
Fresh worktree, bun install --frozen-lockfile && bun run build green; suite launcher with isolated HOME and TMPDIR=/tmp: the four touched test files — 98 pass / 0 fail. I characterized no CI lanes.
Findings — non-blocking
- [mock-restore-guard-scan.ts:161
statementNamesvs :335moduleObjectNames] Type-only imports are counted as scope declarations but skipped as module objects.import type { process } from "./x.js"erases at runtime, yet for the resolver the name shadows the guarded root, so a global write through that name is unreported. Probe-confirmed:analyzeSourceonimport type { process } from "./x.js"; process.exit = () => {};returns no findings, while at runtime the write reaches the real global. Exotic — tsc rejects the value-use, so only a file executed by bun alone (bun ignores type errors) could carry it — and compile-refuted in typechecked code. The fix is one line: checkimportClause.isTypeOnlyinstatementNames' import branch. Recommend a follow-up. - The PR's stated limits (array bindings and computed binding keys of guarded roots untracked; the loaded-set assertion sees only files loaded before it runs) are honest and match the implementation as read.
Not fully seen
hoistedVars and the head of statementNames were pinned by the loop/block-var probes rather than read line-by-line; the guard test's assertion list beyond ~lines 25–140 was sampled; the PR body's evidence section was read through the mutation summary only.
tps-sherlock
left a comment
There was a problem hiding this comment.
Verdict: APPROVE — cli#625, head cd9ebed3.
Repo visibility checked before writing: tpsdev-ai/cli is PUBLIC (.visibility: "public"). The single non-blocking observation below is filed by location and severity; its reachability analysis is sent to the maintainer privately.
Security focus — no patch path (sync throw, async rejection, nested registry) can leave a global patched after cleanup: holds.
The guard is in the factory, packages/cli/test/helpers/patch-shared.ts:58:
if (cleanedUp) throw new PatchAfterCleanupError();
const restore = savePatch(target, property, value);
The refusal runs before savePatch, so a post-cleanup patch is rejected before the property is ever written. The cleanup hook (lines 53–56) sets cleanedUp = true first, then drains restores in reverse (while (restores.length > 0) restores.pop()!()), which is LIFO and matches savePatch's own undo stack.
- sync throw:
test/patch-shared.test.ts:68asserts the restore-then-refuse order and the error name; the real factory is driven through bun's ownafterAllattest/patch-shared.test.ts:106–108. Mutation: deleting thethrowline turns 3 tests red (including theafterAllassertion). Reintroducing agent-seed's lateafterAll(() => patchShared(globalThis, "fetch", _savedFetch))is red withPatchAfterCleanupErrorthrown atpatch-shared.ts:58— so removing the two late re-patches inagent-seed.test.tsandmemory-learn.test.tsis load-bearing, not cosmetic. - async rejection: I drove the factory directly under the isolated launcher — a post-cleanup patch inside both an awaited and an unawaited async continuation surfaces the throw and leaves the target property unchanged (the write never happens). This case has no committed test; the mechanism is the guard-before-
savePatchordering above. - nested registry: two registries over one target clean up independently, and a patch fn from an already-cleaned registry refuses while a fresh registry still patches — probed, holds.
Non-blocking observation [1] — packages/cli/test/helpers/patch-shared.ts:33 (bare patchShared): it applies the patch (savePatch(...)) and only then registers a per-call afterAll(restore); this path has no cleanedUp guard. A call that runs after the file's hooks have completed applies the write and queues a restore hook bun will not run. Severity: low (a test-authoring footgun, unchanged by this PR; patchShared is imported directly by several test files). Reachability detail sent privately. If you want it closed: route bare patchShared through the registry guard, or document that it must be called from file scope or a suite hook.
Confirmed load-bearing, outside my focus:
- Reverting
discoverCliTestFiles(test/helpers/mock-restore-guard-scan.ts:528) to the old*.test.ts-only walk fails "the scan discovers the forms bun discovers, over the cli package". - Collapsing
resolveScope(scan.ts:208) to the immediate parent fails 22 tests (includingmodule-alias-outer-scope-leaked.fixture.ts). - Making
moduleObjectAt(scan.ts:241) name-based instead of scope-based fails the two shadow-cleared fixtures (module-shadow-param-cleared,module-alias-shadow-cleared).
Not seen / environment: the full cli lane on this host ran 3178 tests across 207 files with 10 failures, all in files this PR does not touch and environment-dependent (three "real nono / Landlock·Seatbelt" sandbox cases, validateSshReachable ssh timeout, and a few secrets/nono config cases) — docker is not running here, which explains the sandbox ones. The four files this PR changes pass (98/0). I did not fetch the CI log.
Closes #603
The post-merge audit of #576 and #583 found five gaps in the cli restore guard.
discoverCliTestFilesuses the shareddiscoverTestFilesinscripts/check-test-reports.mjsover the cli package root and returns absolutepaths. The loaded-set assertion checks files recorded by the preload when
the assertion runs. Files parse according to their extension, including
TSX for
.tsxand JSX for.jsx.createPatchRegistry(the body ofcreatePatchShared)throws
PatchAfterCleanupErrorwhen a patch is requested after its cleanuphook ran. Redundant fetch patches in top-level
afterAllhooks are removedfrom
agent-seed.test.tsandmemory-learn.test.ts.string-literal keys preserve the selected property's identity, including
nested bindings:
const { process: p } = globalThisbindsptoprocess.Computed binding keys and array bindings of guarded roots are out of scope.
Values extracted from
process.envare not guarded-root aliases.block-scoped bindings; the existing hoisted-variable handling resolves
var.asserted separately; removing a corresponding write fails its test.
Evidence, measured on 898e78e against 1477f57:
scanners: all eight new assertions fail on 1477f57 and pass here. They cover
writes after a
varloop and inside a nested function; direct environmentdestructuring; environment writes and
p.exitthrough selected processaliases, including nested bindings; JSX followed by a global patch in
.tsxand
.jsx; and discovery from a relative root..test.ts: 1 failure.PatchAfterCleanupErrorrefusal: 3 failures, including realafterAll.var: 2 failures.mock-restore-guard.test.ts74 -> 82 pass;patch-shared.test.ts7 -> 7 pass.With the new assertions and original scanner: 81 pass / 8 fail; corrected:
89 pass / 0 fail.
bun run build(tsc): exit 0.bun run lint:ci: exit 0.node scripts/changelog-fragments.mjs check: 87 fragments valid.🤖 Generated with Claude Code
Summary by CodeRabbit