diff --git a/.claude/agents/code-monkey.md b/.claude/agents/code-monkey.md new file mode 100644 index 00000000..2f2a6895 --- /dev/null +++ b/.claude/agents/code-monkey.md @@ -0,0 +1,38 @@ +--- +name: code-monkey +description: Use this agent to implement one small, precisely-specified code change and verify it with targeted tests — the hands of a larger orchestration loop such as the test-shrink skill. Examples: Context: The test-shrink loop has a candidate to hoist a duplicated server-spawn helper into tests/common. assistant: 'I'll spawn the code-monkey agent with the exact files and the extraction plan, and have it run the affected test targets.' The change is fully specified and scoped; code-monkey implements exactly that and nothing else. Context: A reviewer returned REVISE on a diff because one assertion was dropped during a test merge. assistant: 'I'll send the diff back to a code-monkey agent with the reviewer's notes to restore the missing assertion.' code-monkey applies the revision without expanding scope. +model: sonnet +--- + +You are Code Monkey, a disciplined implementation engineer. You receive one +precisely-scoped change brief and you implement exactly that change — no more, +no less. Your value is fidelity: the orchestrator's verification loop only +works if your diff contains only what the brief asked for. + +Your working rules: + +1. **Scope is a contract.** Implement the brief's change in the files it + names. If completing it genuinely requires touching something the brief + didn't anticipate, make the minimal extra change and call it out + explicitly in your report — never silently expand. +2. **Honor every guardrail in the brief verbatim.** Briefs from the WFL + test-shrink loop include hard rules (never weaken assertions, never delete + coverage, no product-code changes). If the brief's requested change would + violate its own guardrails, stop and report the conflict instead of + picking a side. +3. **Match the surrounding code.** Same idioms, naming, comment density, and + formatting as the file you are editing. Run `cargo fmt --all` on Rust + changes before finishing. +4. **Verify before reporting.** Run the targeted test command(s) given in the + brief (e.g. `cargo test --test `). If they fail, either fix your + change so they pass or revert cleanly and report the failure — never leave + the tree in a half-done state. +5. **Report tightly.** Your final message states: what changed (files and + one-line summary each), the verification commands you ran and their + results, any measured effect the brief asked you to capture (e.g. lines + removed, output bytes saved), and anything surprising you noticed but did + not touch. + +You do not commit, push, review your own work, or make judgment calls about +whether the change is a good idea — those belong to the orchestrator and the +code-reviewer agent. diff --git a/.claude/agents/code-reviewer.md b/.claude/agents/code-reviewer.md new file mode 100644 index 00000000..230c279d --- /dev/null +++ b/.claude/agents/code-reviewer.md @@ -0,0 +1,47 @@ +--- +name: code-reviewer +description: Use this agent to review a diff against explicit acceptance rules and return a KEEP / REVISE / REJECT verdict with evidence — the gatekeeper of orchestration loops such as the test-shrink skill, and useful for any pre-commit review of a focused change. Examples: Context: The test-shrink loop just had a code-monkey merge two overlapping parser tests. assistant: 'Before keeping this, I'll spawn the code-reviewer agent on the diff to confirm every assertion from both original tests survives in the merged test.' Coverage preservation is exactly what code-reviewer exists to verify. Context: A refactor moved duplicated test harness code into tests/common. assistant: 'I'll have the code-reviewer agent check the diff for behavior changes and confirm each call site still exercises the same code path.' A mechanical-looking refactor still needs an independent set of eyes before commit. +model: sonnet +--- + +You are a rigorous, independent code reviewer. You receive a diff (or a way to +produce one, e.g. `git diff`) plus the acceptance rules it must satisfy, and +you return a verdict. You never edit code — your only output is judgment +backed by evidence. + +For WFL test-suite reviews, the binding rules (from root `testing.md`) are: + +- **Coverage may never shrink.** Every assertion, negative/failure-path check, + and side-effect verification present before the diff must still be made + after it. For merged or restructured tests, build an explicit + assertion-by-assertion mapping from old to new — "looks equivalent" is not + evidence. +- **No manufactured green.** Reject retries, `#[ignore]`, loosened tolerances, + removed `-D warnings` surface, or assertions rewritten to be vacuously true. +- **Tests must still test the real boundary.** A shrink that swaps a real + binary/socket/file interaction for a mock has changed what the test + verifies — reject it. +- **Scope discipline.** Changes outside the stated scope (product code under + `src/`, Cargo profiles, CI config) are grounds for REVISE at minimum, with + the out-of-scope hunks named. + +Your review process: + +1. Read the brief's stated intent and rules first, then the full diff — not + just the hunks, but enough surrounding code to understand what each hunk + changes about behavior. +2. Hunt specifically for what the diff *removes*: assertions, error-path + checks, fixture cases, output the old test inspected. +3. Check the claimed benefit is real (e.g. if the brief claims N lines or N + bytes of debug output saved, sanity-check it from the diff). +4. Deliver exactly one verdict: + - **KEEP** — rules satisfied; list the evidence that convinced you. + - **REVISE** — fixable issues; list each one with file/line and what a fix + must accomplish. + - **REJECT** — the change cannot satisfy the rules (e.g. the two "duplicate" + tests actually cover different paths); explain why no revision can save it. + +Be skeptical by default: the cost of wrongly keeping a coverage-losing diff is +far higher than the cost of wrongly sending one back. When you cannot +establish equivalence from the evidence available, say so and return REVISE +with what evidence you'd need — do not guess your way to KEEP. diff --git a/.claude/agents/research-agent.md b/.claude/agents/research-agent.md new file mode 100644 index 00000000..14404d95 --- /dev/null +++ b/.claude/agents/research-agent.md @@ -0,0 +1,47 @@ +--- +name: research-agent +description: Use this agent for read-only reconnaissance that produces a ranked list of findings or candidates — surveying the codebase for optimization opportunities, researching current Rust/tooling best practices on the web, or both. The scout of orchestration loops such as the test-shrink skill. Examples: Context: The test-shrink loop needs to know where the test suite generates excessive debug output. assistant: 'I'll spawn research-agent instances in parallel — one hunting duplicated harness code across tests/, one measuring which test files print the most output, one web-searching current Rust techniques for reducing test debug data.' Parallel read-only scouts, each returning a ranked candidate list, are exactly this agent's shape. Context: The maintainer wants to know which fixtures are oversized relative to what their tests assert. assistant: 'I'll use the research-agent to cross-reference tests/fixtures/ sizes against the assertions that consume them and rank the shrink candidates.' Investigation and ranking without modification — research-agent territory. +model: sonnet +--- + +You are a research scout. You investigate — codebase, measurements, and when +useful the web — and return a ranked, actionable candidate list. You never +modify files: your entire output is knowledge that lets an orchestrator decide +what to do next. (Running read-only commands, including builds or tests whose +output you measure, is fine; changing tracked files is not.) + +Your method: + +1. **Measure before opining.** Ground every claim in a number you actually + collected: line counts, `du` output, bytes of captured test output, grep + hit counts, timing. A candidate without a measured or well-estimated saving + is an anecdote, not a finding. +2. **Search the way the evidence points.** Start from the hunting ground you + were assigned, but follow surprises — the biggest wins are often one + directory over from where you were sent. Note out-of-scope discoveries in + a separate section rather than dropping them. +3. **Use the web for practices, not guesses.** When asked to research + techniques (e.g. shrinking Rust test debug data, Cargo profile options, + test-harness patterns), prefer current primary sources — official docs, + release notes — and say which version/date your findings reflect. +4. **Respect the loop's constraints.** For WFL test-shrink work: candidates + must not weaken what tests verify, must not touch product behavior in + `src/`, and anything requiring a maintainer decision (Cargo profiles, CI + config, `debug = true` in release) is reported under a separate + "maintainer decision" heading, not as an ordinary candidate. + +Your report format — a ranked list where each candidate has: + +- **What & where:** the specific files/patterns involved (with `file:line` + where it helps). +- **Estimated saving:** the metric it improves and by roughly how much, with + the measurement behind the estimate. +- **Risk:** what could go wrong and how likely (e.g. "mechanical, low" vs. + "requires merging tests, medium — coverage mapping needed"). +- **Suggested approach:** the change an implementer would make, in 1–3 + sentences. + +Rank by saving × (1 − risk), best first. End with the out-of-scope +discoveries and maintainer-decision items, each clearly labeled. If a hunting +ground turns out to be barren, say so plainly with the evidence — a confident +"nothing here" is a valuable result. diff --git a/.claude/skills/test-shrink/SKILL.md b/.claude/skills/test-shrink/SKILL.md new file mode 100644 index 00000000..e439b3d5 --- /dev/null +++ b/.claude/skills/test-shrink/SKILL.md @@ -0,0 +1,196 @@ +--- +name: test-shrink +description: Run the WFL test-suite shrink loop — an iterative, subagent-driven optimization pass that reduces the size of the Rust unit/integration tests and the debug data they generate, while keeping every test and every TestPrograms/ program passing. Use this whenever the maintainer asks to shrink, slim, optimize, or reduce the tests, test output, test debug data, test bloat, or test disk usage, or says things like "run the test shrink loop", "make the tests smaller", "the tests generate too much debug data", or "optimize the unit tests". This is an occasional maintenance pass, not part of feature work. +--- + +# WFL Test Shrink Loop + +An iterative optimization pass over the WFL test suite. The goal is to make the +tests **smaller** — less debug data generated, fewer bytes on disk, less +duplicated harness code, faster runs — **without losing one bit of what they +verify**. Coverage is the product here; size is the cost. This loop only ever +reduces cost. + +The loop is: **measure → research → shrink one candidate → review → verify → +keep or revert → repeat** until gains dry up. Subagents do the heavy lifting: +`research-agent` finds candidates, `code-monkey` implements them one at a time, +`code-reviewer` guards coverage before anything is kept. + +## Hard rules (read before anything else) + +These come from the binding Logbie Testing Policy (root `testing.md`) and +`GOVERNANCE.md`. A shrink that violates one of these is a regression, not an +optimization — revert it. + +1. **Never weaken what a test verifies.** No deleted assertions, no relaxed + tolerances, no `#[ignore]`, no removed negative/failure-path checks. Two + tests may be merged only when the merged test provably makes every + assertion of both — the code-reviewer must confirm this from the diff. +2. **Never delete a test file** unless it is demonstrably a strict duplicate + of another (same inputs, same assertions) and the reviewer confirms it. + When in doubt, keep it and shrink something else. +3. **All gates stay green.** `cargo fmt --all -- --check`, + `cargo clippy --all-targets --all-features -- -D warnings`, + `cargo test --all`, and the full `TestPrograms/` integration run must pass + after every accepted change. A change that can't be verified is reverted, + not "probably fine". +4. **Backward compatibility is out of scope.** This loop touches `tests/`, + `tests/common/`, `tests/fixtures/`, and test-output plumbing. It does not + change `src/` behavior. If a real product improvement surfaces (e.g. a + runtime flag to silence trace output), record it as a **proposal** in the + final report instead of implementing it here. +5. **Maintainer-decision items are flagged, not applied.** `[profile.release] debug = true` + is deliberate (release backtraces) — never change the release profile. New + dev-dependencies (e.g. snapshot crates), CI workflow files, linker/tooling + config, and `.repo-hygiene.toml` also go in the report's "Proposals for + Brad" section instead of being applied. **Test-profile tuning + (`[profile.test]`) is an ordinary candidate** — see the playbook's rules, + including the readable-panic check. One carve-out: when an accepted + candidate renames or moves test targets (e.g. suite consolidation), the + **mechanical** updates to references to those target names — in + `testing.md`, `scripts/`, and CI workflow files — are part of that same + candidate, because the gates cannot stay green without them. Substantive + CI changes (new jobs, runners, tools) remain proposals. +6. **Hygiene:** all measurement scratch and reports go under + `target/reports/test-shrink/` or the session temp dir — never into the tree. + +## Phase 0 — Preflight + +1. **Disk check** (per CLAUDE.md): a full build needs ~30 GB of `target/`. + ```bash + [ "$(df -BG --output=avail . | tail -1 | tr -dc '0-9')" -lt 30 ] && cargo clean + ``` +2. **Green baseline.** Run the full gate once: + `cargo test --all`, then `cargo build --release` and + `./scripts/run_integration_tests.sh` (`.ps1` on Windows — wherever this + skill says `.sh`, use the platform's variant). If anything is red **stop** — report + the failure instead of optimizing on a broken base. Shrinking on red makes + it impossible to tell whether a shrink broke something. +3. **Working branch.** Do this work on a dedicated branch (or the branch the + session was given), one conventional commit per accepted candidate + (`test: ...` or `refactor(tests): ...`), so any single shrink can be + reverted later without unwinding the rest. + +## Phase 1 — Measure the baseline + +"Small" must be a number or the loop can't tell whether it's winning. Capture +these into `target/reports/test-shrink/baseline.md` (and re-capture the same +way at the end): + +| Metric | How | +|---|---| +| Test source size | `find tests -name '*.rs' -print0 \| xargs -0 wc -l \| tail -1`; `du -sh tests/` (not a bare `tests/**/*.rs` glob — without globstar it silently skips the top-level files, which is most of the suite) | +| Fixture size | `du -sh tests/fixtures/` | +| Debug/output volume | `cargo test --all 2>&1 \| wc -c` (bytes the suite prints) | +| On-disk artifacts | `du -sh target/test-artifacts/` before vs after a run; list any stray files a run drops elsewhere (hygiene violations — fix on sight) | +| Wall time | time of `cargo test --all` and of the integration script | +| Test binary footprint | `du -sh target/debug/deps/` and count of test binaries — with ~146 separate files under `tests/`, each is its own binary statically linking the whole compiler with debug info; this is usually where most of the "debug data" lives. **Compare like with like:** Cargo never deletes a removed target's old binaries, so a dirty `deps/` diff shows no saving (or growth) after consolidation — measure from equivalent clean states, or inventory only the current target graph's artifacts (`cargo test --no-run --message-format=json` lists them) | + +Not every metric must improve every run — but no metric may get *worse* in an +accepted change without an explicit reason in its commit message. + +## Phase 2 — Research (parallel subagents) + +**First, read `references/rust-test-optimization.md`** — the distilled +playbook of Rust compiler-testing research (single-binary consolidation, +`check` drivers, profile tuning, snapshot testing, rustc hygiene rules) with +WFL-specific migration notes and a suggested priority order. Hand each +research agent the section for its hunting ground; treat the playbook's claims +as hypotheses to verify and measure, not facts. + +Spawn **research-agent** subagents in parallel, one per hunting ground, each +returning a ranked candidate list (what, where, estimated saving, risk, +suggested approach). Hunting grounds that historically pay off in this repo: + +- **Binary proliferation (playbook §1):** every `.rs` file directly under + `tests/` becomes its own statically-linked test binary. Consolidating into a + single suite binary in incremental batches is usually the biggest disk and + link-time win available. +- **Noise:** `println!`/`eprintln!`/`dbg!` and captured `exec_trace!` output in + test code; tests that run the binary with verbose flags they don't assert on; + overly chatty failure messages built eagerly (`format!` on the hot path). +- **Duplication (playbook §3):** the same spawn-server / run-wfl-file / + temp-dir harness re-implemented across files — hoist into `tests/common/` + as `check`-style drivers. With ~146 test files this is usually the biggest + source-line win. +- **Redundancy:** tests that assert a strict subset of another test's + assertions on the same inputs (merge, carefully — rule 1). +- **Fixtures (playbook §5):** oversized `.wfl` fixtures where a minimal + program exercises the same path; generated fixtures that could be built in + the test instead of checked in; non-descriptive issue-number test names. +- **Artifacts:** test output written outside `target/test-artifacts//` + or temp dirs; artifacts never cleaned up; debug dumps written even on pass. +- **Profiles (playbook §2):** `[profile.test]` debug-info and opt-level + tuning — ordinary candidate, but never the release profile, and always with + the readable-panic check. +- **Rust-level:** anything beyond the playbook that a current-best-practices + search turns up for shrinking Rust test debug data (e.g. capturing output + instead of printing, lazy `assert!` messages, splitting mega-tests). The + research agent may use web search, and should verify the playbook's claims + against current docs while it's there. + +Merge the lists, de-duplicate, and order the worklist **highest saving × lowest +risk first**. Cap a single run's worklist at ~10 candidates — this skill runs +occasionally; leftover candidates go in the report for next time. + +## Phase 3 — The shrink loop (serial, one candidate at a time) + +For each candidate, in order: + +1. **Implement — spawn `code-monkey`** with a tight brief: the exact files, + the exact change, the hard rules above verbatim, and the targeted test + command for the affected area (e.g. `cargo test --test `). One + candidate per agent; small focused diffs are what make step 3 meaningful. +2. **Review — spawn `code-reviewer`** on the complete change: run + `git add -A` first, then have it review `git status --short` plus + `git diff --cached HEAD` — a plain `git diff` omits newly created files + and staged-only changes (e.g. a new shared helper, or `git mv`s during + consolidation), letting a KEEP through without the reviewer ever seeing + the code that decides coverage. Its one non-negotiable question: *does + the suite still verify everything it verified before this diff?* It + answers KEEP / REVISE / REJECT with evidence. REVISE goes back to a + code-monkey once; REJECT means revert now. +3. **Verify.** Targeted tests first (fast feedback), then before committing: + `cargo fmt --all`, `cargo clippy --all-targets --all-features -- -D warnings`, + `cargo test --all`. If the change touched anything the end-to-end programs + exercise (fixtures, harness, binary invocation), also rerun + `./scripts/run_integration_tests.sh`. +4. **Keep or revert.** Green → commit (one candidate, one commit, message + states the measured saving). Anything red or reviewer-rejected → restore + tracked files (`git restore --staged --worktree -- .`) **and** delete any + files the attempt created with `git clean -fd` scoped to the paths the + candidate touched (e.g. `git clean -fd -- tests/`) — `git restore` alone + leaves new untracked files behind, silently polluting the next attempt. + Never carry a broken candidate forward while starting the next one. + +Stop the loop when: the worklist is empty, **or** the last 3 candidates were +all rejected/reverted, **or** remaining candidates are estimated under ~1% of +any metric. Diminishing returns are the expected exit — this skill gets run +again another day. + +## Phase 4 — Wrap up + +1. Re-run the **full** gate one final time, including + `./scripts/run_integration_tests.sh` — every TestPrograms/ program must + pass. This is the promise the skill makes. +2. Re-measure Phase 1's metrics; write the before/after table. +3. `python3 scripts/check_repo_hygiene.py --mode static` (structural changes + were made). +4. Add a Dev Diary entry `History/dev-diary//-test-shrink-pass.md` + summarizing: metrics before/after, candidates accepted (with commits), + candidates rejected and why, and the **Proposals for Brad** list + (maintainer-decision items from rule 5 and unimplemented `src/` ideas). +5. Commit the diary entry, push the branch, and report the before/after + numbers as the headline. + +## Subagent quick reference + +| Agent | Role | Mode | +|---|---|---| +| `research-agent` | find & rank shrink candidates; may web-search Rust practices | parallel, read-only | +| `code-monkey` | implement exactly one candidate, run targeted tests | serial | +| `code-reviewer` | guard coverage; KEEP / REVISE / REJECT with evidence | serial, read-only | + +Give every subagent the hard-rules block verbatim in its prompt — subagents do +not see this file unless told, and the rules are the contract that keeps this +loop safe to run unattended. diff --git a/.claude/skills/test-shrink/references/rust-test-optimization.md b/.claude/skills/test-shrink/references/rust-test-optimization.md new file mode 100644 index 00000000..4eeb3a74 --- /dev/null +++ b/.claude/skills/test-shrink/references/rust-test-optimization.md @@ -0,0 +1,164 @@ +# Rust test-suite optimization playbook (WFL profile) + +Distilled from external research on compiler/interpreter test architecture in +Rust (matklad's "How to Test" / "Fast Rust Builds", the rustc dev-guide test +best practices, rust-analyzer's and pydantic-monty's suites), applied to this +repo. Read this during Phase 2 (research) and hand the relevant section to the +agent working a candidate. Treat the *claims* here as hypotheses: research +agents should verify current crate/Cargo behavior against live docs before an +implementation is briefed, and every saving still has to show up in Phase 1's +metrics to be kept. + +## 1. Test-binary proliferation — likely the single biggest win + +Cargo compiles **every `.rs` file directly under `tests/` as its own binary**, +each statically linking the full WFL compiler library with debug info. With +~146 files, that is ~146 near-identical copies of the compiler in +`target/debug/deps/` — this is where most of the "huge debug data" lives, and +why link time dominates test iteration. + +**Fix: the single-binary integration pattern.** One entry point (e.g. +`tests/suite/main.rs`) declaring the former files as `mod` submodules: + +```rust +// tests/suite/main.rs +mod lexer; +mod parser; +mod typechecker; +mod e2e; +``` + +One binary, one link, same tests. Migration notes for this repo: + +- Migrate **incrementally in batches** (e.g. 10–20 files per candidate), each + batch its own commit — moving all 146 at once makes review and bisection + impossible. Cargo ignores subdirectories of `tests/` (except as modules), so + moved files stop being independent targets the moment they're under + `tests/suite/`. +- Fix collisions as they surface: duplicate `mod common` imports, duplicate + helper names across files, `#[path]` attributes, per-file `#![...]` inner + attributes that must move or become `#[allow]` on the module. +- `cargo test --test ` granularity is lost; the equivalent is + `cargo test --test suite ::` filter strings. Update any docs or + scripts that name individual test targets (`testing.md`, `scripts/`, CI, + this skill) **in the same candidate** — SKILL.md rule 5's carve-out permits + these mechanical reference updates, since the gates can't stay green + without them. +- The existing `tests/common/` sharing mechanism becomes a plain module of the + suite — usually *simpler* after consolidation. +- Verify with `cargo test --all` after each batch: the same test count must + run, and no test may silently vanish (compare `cargo test -- --list` counts + before/after — that is the reviewer's coverage evidence). + +Measured effects to expect: `target/debug/deps` shrinks dramatically, full- +suite link time collapses, and per-edit iteration gets faster. Capture +before/after `du -sh target/debug/deps` and suite wall time. + +## 2. Profile tuning for test builds — in-scope, with care + +`cargo test` builds test targets with the `test` profile (which inherits +`dev` by default — `opt-level = 0`, full debug info) and builds dependencies +with the `dev` profile. Full debug symbols across ~146 (or even 1) test +binaries are most of the disk cost; `opt-level = 0` makes interpreter-heavy +tests slow at runtime. + +Candidate settings (verify against current Cargo docs, then measure): + +```toml +[profile.test] +opt-level = 1 # cheap runtime win for interpreter-heavy tests +debug = 1 # line-tables-only: keeps usable panics/backtraces +# Do NOT combine with `strip = "debuginfo"` — stripping removes the same +# line tables `debug = 1` exists to keep, and panics go unreadable. If disk +# is the priority, pick `strip` and full `debug = false` deliberately instead. + +# Dependencies are built with the dev profile, so tune heavy deps there: +[profile.dev.package."*"] # or name specific heavy deps (tokio, sqlx, …) +opt-level = 2 +``` + +Hard boundary: **`[profile.release] debug = true` is deliberate** (release +backtraces, per CLAUDE.md) — never change the release profile. Test/dev +profile changes are ordinary candidates *if* the full gate stays green and +panic output in a deliberately-failed test is still readable (check this +explicitly: break a test on purpose, read the output, restore it). + +## 3. Kill duplicated harness code with `check`-style drivers + +The pattern that keeps rust-analyzer's suite small: tests never touch compiler +internals or re-implement plumbing — they call one shared driver: + +```rust +fn check_interpreter(source: &str, expected: &str) { + // lex → parse → analyze → typecheck → interpret, all in memory + assert_eq!(run_wfl_source(source), expected); +} +``` + +Why it shrinks the suite: hundreds of tests collapse to data (input string, +expected output), and internal API changes touch one driver instead of every +test. This also passes the "neural-network test": a suite asserting on +language-visible behavior survives any internal refactor. + +For WFL: hunt for repeated spawn-the-binary / build-an-interpreter / tempdir +scaffolding across `tests/*.rs` and hoist it into `tests/common/` (or the +suite root module after consolidation) as `check_*` drivers. Prefer in-memory +drivers over spawning the release binary wherever the test's claim doesn't +require the real CLI boundary — but per the hard rules, a test whose *point* +is the real binary/socket/file must keep the real boundary. + +## 4. Snapshot testing — proposal-grade, big payoff for diagnostics + +For AST dumps, diagnostics, and other large expected outputs, snapshot crates +beat hand-maintained `assert_eq!` blocks: + +- **`expect-test`** (rust-analyzer's): inline `expect![[...]]` snapshots, + minimal dependency footprint, update via `UPDATE_EXPECT=1 cargo test`. +- **`insta`**: external `.snap` files + `cargo-insta review` TUI; better for + multi-line diagnostic rendering. +- **`datatest-stable`**: generates a test per file in a directory — maps + naturally onto a `.wfl`-corpus style suite. + +Adding a dev-dependency changes the audited dependency tree, so treat adoption +as a **maintainer-decision proposal** (with measured line-count savings on a +worked example) unless Brad's invocation explicitly green-lights it. + +If snapshots are adopted, the known gotchas — bake these into the driver from +day one: + +- **Span volatility:** never snapshot raw spans/byte offsets; redact them + (custom `Debug`, a sanitizer pass, or insta filters), or every whitespace + edit invalidates the suite. +- **Nondeterminism:** normalize absolute paths to placeholders, strip ANSI + escapes, force `\n` line endings — or snapshots fail across machines/CI. +- **Parametric collisions:** parameterized tests sharing one function need + unique snapshot suffixes per case, or cases overwrite each other. + +## 5. rustc-suite hygiene rules worth copying (cheap, low-risk) + +- **Minimal test programs:** each fixture/test program should contain only + what its assertion needs — shrink oversized fixtures to minimal repros. +- **Descriptive names over issue numbers:** `issue_12345.rs` → + `parser_bare_trait_object_issue_12345.rs`; filterable and self-documenting. +- **Suppress unrelated noise at the source:** `#[allow(...)]`/`#[expect(...)]` + for warnings a test doesn't assert on, so unrelated lint churn doesn't bloat + captured output or break expectations. + +## 6. Environment/tooling — proposals only + +Worth proposing to Brad with estimates, but they change the dev/CI +environment, not the tests, so they are never applied by this loop: + +- **Faster linkers** (`mold` on Linux, `lld` cross-platform) via + `.cargo/config.toml` — large link-step speedups, machine-setup dependent. +- **`cargo-nextest`** — per-test process isolation, better scheduling, cleaner + output; would change the documented test commands in `testing.md`/CI. + +## Priority order for a typical pass + +1. Binary consolidation batches (§1) — biggest disk + link-time win. +2. Harness dedup into `check` drivers (§3) — biggest source-line win. +3. Fixture minimization + naming + noise suppression (§5) — steady small wins. +4. Test-profile tuning (§2) — one candidate, big disk win, needs the + readable-panic check. +5. Snapshot adoption + tooling (§4, §6) — written up as proposals. diff --git a/.repo-hygiene.toml b/.repo-hygiene.toml index 534d47bb..a31281a2 100644 --- a/.repo-hygiene.toml +++ b/.repo-hygiene.toml @@ -24,6 +24,7 @@ allowed-files = [ "Cargo.lock", "Cargo.toml", "GOVERNANCE.md", + "IDEA.md", # WFL language definition consumed by Hermes — maintainer-approved (Brad, 2026-08-14) "LICENSE", "README.md", "REPOSITORY_HYGIENE.md", diff --git a/History/dev-diary/2026/2026-08-14-test-shrink-skill.md b/History/dev-diary/2026/2026-08-14-test-shrink-skill.md new file mode 100644 index 00000000..f00f2b56 --- /dev/null +++ b/History/dev-diary/2026/2026-08-14-test-shrink-skill.md @@ -0,0 +1,73 @@ +# 2026-08-14 — test-shrink skill and orchestration agents + +## What + +Added an on-demand Claude Code project skill, `test-shrink` +(`.claude/skills/test-shrink/SKILL.md`), plus three reusable subagent +definitions it orchestrates: + +- `.claude/agents/research-agent.md` — read-only scout that measures and ranks + shrink candidates (may web-search current Rust practices). +- `.claude/agents/code-monkey.md` — implements exactly one scoped change and + verifies it with targeted tests. +- `.claude/agents/code-reviewer.md` — gatekeeper returning KEEP / REVISE / + REJECT, with coverage preservation as its non-negotiable question. + +## Why + +The Rust test suite (~146 files, ~43k lines) generates a large volume of debug +data and duplicated harness code. The maintainer wants an occasional +maintenance pass that shrinks test size and output as far as possible while +every Rust test and every `TestPrograms/` program keeps passing. + +## How it works + +The skill runs a loop: green-baseline preflight → measure (source lines, +fixture size, captured output bytes, on-disk artifacts, wall time) → parallel +research agents produce a ranked worklist → serial shrink loop (code-monkey +implements one candidate, code-reviewer verdicts the diff, full gate verifies, +keep-as-one-commit or revert) → wrap-up with before/after metrics, hygiene +check, and a fresh Dev Diary entry per pass. + +Guardrails are drawn from the binding Logbie Testing Policy: no weakened +assertions, no deleted coverage, no manufactured green, real boundaries stay +real, and maintainer-decision items (e.g. `[profile.release] debug = true`, +CI config) are reported as proposals rather than applied. + +## Research playbook (same-day follow-up) + +Distilled the maintainer-supplied research on Rust compiler/interpreter test +architecture (matklad's testing/build essays, rustc dev-guide best practices, +rust-analyzer and pydantic-monty case studies) into +`.claude/skills/test-shrink/references/rust-test-optimization.md`. Key content: + +- **Test-binary proliferation:** each of the ~146 files under `tests/` is its + own statically-linked binary with debug info — identified as the primary + source of the suite's disk/debug-data bloat; the playbook documents the + incremental single-binary consolidation migration. +- **`check`-driver pattern** for deduplicating harness code without coupling + tests to compiler internals. +- **`[profile.test]` tuning** promoted to an ordinary candidate (release + profile stays untouchable); snapshot-crate adoption, `cargo-nextest`, and + linker changes remain maintainer proposals. +- **rustc hygiene rules:** minimal fixtures, descriptive names, suppressing + unrelated warning noise. + +SKILL.md's research phase now points agents at the playbook with a priority +order, and treats its claims as hypotheses to verify and measure. + +## Notes + +- Also on this branch: `IDEA.md` (root, present since the initial commit; the + WFL language definition consumed by Hermes) was added to the root + allowlist in `.repo-hygiene.toml` at the maintainer's direction — + `REPOSITORY_HYGIENE.md` §9 names that file as the record of approved + exceptions. This unblocks the previously-failing hygiene gate. +- Bot-review fixes folded in after PR #691 feedback: scoped `git clean` on + revert (untracked leftovers), `find`-based line counting (globstar trap), + corrected `[profile.test]`/`[profile.dev.package]` guidance (strip-vs-debug + conflict, dependency profile), a rule-5 carve-out for mechanical + test-target-name updates during consolidation, and `.ps1` variant notes. +- No product code changed; this is tooling/process only. +- The agents are written to be reusable by other orchestration loops, not + just test-shrink.