Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 6 additions & 26 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1459,33 +1459,13 @@ jobs:
if: needs.changes.outputs.rust == 'true'
run: cargo xtask eval-fixtures --check

# Advisory for now. `cargo xtask check-module-size` is strict on its own;
# only this step is non-blocking. `continue-on-error` keeps the job green
# while the warning annotation and the job summary list the offenders. When
# files are over the limit the step itself shows as failed (red) in the
# job's step list, because it ends in `exit 1`, while the job and the check
# stay green: that is the advisory phase, not a failure to chase.
# Making the limit blocking means deleting `continue-on-error` and the
# capture below, leaving the bare command.
- name: Source files are within the module-size limit (advisory)
# Blocking: a source file over the limit fails this job, which is a required
# check. `merge_group` runs it too (`changes` forces `rust` on every event
# but a pull request), so two PRs that each pass alone cannot together
# push a file over the limit.
- name: Source files are within the module-size limit
if: needs.changes.outputs.rust == 'true'
continue-on-error: true
run: |
report="$RUNNER_TEMP/module-size.txt"
if cargo xtask check-module-size > "$report" 2>&1; then
cat "$report"
else
cat "$report"
echo "::warning title=Module size (advisory)::$(sed 's/%/%25/g' "$report" | tr '\n' ' ')"
{
echo '### Source files over the module-size limit (advisory)'
echo
echo '```'
cat "$report"
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
exit 1
fi
run: cargo xtask check-module-size

# Surface the cache hit rate for the clippy compile so the effect is visible
# in the logs, on the local-only path too — see build-and-test for why this
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ Guardrails:
- new subsystems/subcommands: default their domain implementation to its own file from day one (full domain extraction, e.g. `therock.rs`/`comfyui.rs` — private `mod` in `apps/rocm`, accessed via qualified paths; the clap command enum usually goes in [`cli.rs`](apps/rocm/src/cli.rs) with the rest of the command tree, and its dispatch function goes in the domain module (`comfyui::run`) or a `*_cmd.rs` sibling, with only the top-level `dispatch()` match staying in `main.rs`; `bootstrap.rs` and `remote/mod.rs` are the exceptions that own their enum as well — see `docs/architecture.md`'s `bootstrap.rs` note), not growth inside `main.rs`/`lib.rs` awaiting a future extraction pass; see `docs/architecture.md` for the module map and the mechanical-relocation alternative used for dispatch-adjacent clusters
- `crates/rocm-engine-protocol` is a contract surface; verify all impacted engines after protocol changes
- first-party crate-layering invariants (e.g. `rocmd` must never depend on `rocm`) are enforced by `cargo xtask check-crate-edges` (`xtask/src/crate_edges.rs`); a new first-party dependency edge failing that check means the edge needs review, not a bypass
- source file size is checked by `cargo xtask check-module-size` ([`module_size.rs`](xtask/src/module_size.rs)): no source file may exceed 1,500 production lines (test-gated items excluded), with one limit for every file and no exemption mechanism. A PR that would push a file past it splits the file in the same change; do not raise the limit or opt a file out. Merely editing a file that is already over the limit does not by itself require a split: those files are split under their own tickets, and the check cannot become blocking until none is left. Today the CI step and pre-commit hook only report; see `docs/architecture.md`
- source file size is checked by `cargo xtask check-module-size` ([`module_size.rs`](xtask/src/module_size.rs)): no source file may exceed 1,500 production lines (test-gated items excluded), with one limit for every file and no exemption mechanism. A PR that would push a file past it splits the file in the same change; do not raise the limit or opt a file out. The check blocks merges: the `clippy` CI job, a required check that also runs on merge-queue runs, fails when any source file is over the limit, so a file over it is split before anything else merges. The local pre-commit hook still only reports; see `docs/architecture.md`
- every production source file (a `.rs` file under `apps/*/src`, `crates/*/src` or `engines/*/src`) has an entry in `docs/architecture.md`, enforced by `cargo xtask check-architecture-doc` ([architecture_doc.rs](xtask/src/architecture_doc.rs)) in the `clippy` CI job and in the `architecture-doc` prek hook at commit and push. Add the entry in the same change as the file. The check has no exemption list: document the file. The check reads no prose, so keeping an entry's description true is still on you
- preserve strict GPU-required behavior; do not introduce silent CPU fallback
- respect platform gates (for example, native Windows handling for vLLM)
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ New subcommands and subsystems default to their own file from day one — they s

Moving a `proptest!` block to another file leaves its saved failure cases behind unless they are carried over: proptest keys them by the test's source file (`<crate>/src/<path>.rs` reads `<crate>/proptest-regressions/<path>.txt`, where `src/` is the directory holding `main.rs` or `lib.rs`) and does not warn when the file stops matching. Carry them in the same commit, and how depends on how much moved. When every property test of the file moves and the new file has no regression file yet, `git mv` the old one to the name the new file needs. When only some move, the tests left behind still read the old file, so keep it, and copy the saved cases to the new file's regression file (appending to it when it already exists, so its own cases stay). An `xtask` test fails the build on a regression file no test reads; it cannot notice a regression file that was moved away from tests that still need it, so check a split's diff for that. See [Property tests](testing.md#property-tests).

A flat size limit backs this convention: `cargo xtask check-module-size` ([`module_size.rs`](../xtask/src/module_size.rs)) reports every source file with more than 1,500 production lines, and there is one limit for every file, with no per-file ceilings, baseline or exemption. Production lines exclude test-gated items (inline `#[cfg(test)]` modules and the like), and test-only files are not source files; the exact definition is at the top of that file. Today the check is advisory: the `clippy` CI job and the `cargo-module-size` pre-commit hook print the offenders without failing, until the files over the limit have been split. A PR that would push a file past 1,500 production lines splits it in the same change (a pure move, the tests moving with the code, this document updated in the same PR); raising the limit or exempting a file is not an option for a feature PR. A change that merely edits a file that is already over the limit does not by itself have to split it: those files are split under their own tickets, and the check becomes blocking only once none is left. `too_many_lines = "allow"` in the workspace `Cargo.toml` stays a deliberate function-level choice; this gate is about files.
A flat size limit backs this convention: `cargo xtask check-module-size` ([`module_size.rs`](../xtask/src/module_size.rs)) fails when any source file has more than 1,500 production lines, and there is one limit for every file, with no per-file ceilings, baseline or exemption. Production lines exclude test-gated items (inline `#[cfg(test)]` modules and the like), and test-only files are not source files; the exact definition is at the top of that file. The check blocks merges: the `clippy` CI job, a required check that also runs on merge-queue runs, fails when any file is over the limit, so a file over it has to be split before anything else merges. The `cargo-module-size` pre-commit hook still only reports the offenders. A PR that would push a file past 1,500 production lines splits it in the same change (a pure move, the tests moving with the code, this document updated in the same PR); raising the limit or exempting a file is not an option for a feature PR. `too_many_lines = "allow"` in the workspace `Cargo.toml` stays a deliberate function-level choice; this gate is about files.

## Module map

Expand Down
15 changes: 8 additions & 7 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,14 @@ cargo xtask check-module-size
```

It exits non-zero and names each offending file with its count. There is one limit for
every file: no flag, baseline or exemption. For now the check is advisory in the `clippy`
CI job and in the `cargo-module-size` pre-commit hook, which run
`cargo xtask check-module-size` and `cargo xtask check-module-size-advisory` and report
offenders without failing (in CI the step shows red in the job's step list while the job
stays green); the second is the same check that exits successfully even
when the check itself cannot run. That holds once xtask has built: a commit that leaves
xtask or `rocm-core` not compiling still fails the hook.
every file: no flag, baseline or exemption. The `clippy` CI job runs
`cargo xtask check-module-size` and fails when any file is over the limit; `clippy` is a
required check and also runs on merge-queue runs. The failure text names each file and
points at the split rule in [architecture.md](architecture.md). The `cargo-module-size`
pre-commit hook still only reports: it runs `cargo xtask check-module-size-advisory`, the
same check, but it exits successfully even when the check itself cannot run. That holds
once xtask has built: a commit that leaves xtask or `rocm-core` not compiling still fails
the hook.
The counting rules are unit-tested in `xtask/src/module_size.rs`:

```bash
Expand Down
2 changes: 1 addition & 1 deletion xtask/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ pub(crate) enum Command {
/// run (`cargo metadata` fails, a file does not parse). That holds once xtask
/// has built: if xtask or `rocm-core` fails to compile, `cargo xtask` exits
/// non-zero before this code runs. The local pre-commit hook runs this until
/// the limit is enforced.
/// the hook itself is made blocking; the CI check already is.
CheckModuleSizeAdvisory,
/// Fail if a production source file (a `.rs` file under `apps/*/src`,
/// `crates/*/src` or `engines/*/src`) has no entry in `docs/architecture.md`.
Expand Down
75 changes: 70 additions & 5 deletions xtask/src/module_size.rs
Original file line number Diff line number Diff line change
Expand Up @@ -126,9 +126,10 @@ pub fn run() -> Result<()> {
}

/// `cargo xtask check-module-size-advisory`: the same check, but it only reports.
/// This is the wrapper the local hook runs while the limit is not yet blocking; once
/// xtask has built it cannot fail a commit, including when the check itself cannot
/// run. A compile failure in xtask or `rocm-core` never reaches this function.
/// This is the wrapper the local hook runs while the hook is not yet blocking (CI
/// already is); once xtask has built it cannot fail a commit, including when the
/// check itself cannot run. A compile failure in xtask or `rocm-core` never reaches
/// this function.
pub fn run_advisory() -> Result<()> {
advisory_to(evaluate(), &mut std::io::stdout());
Ok(())
Expand All @@ -140,7 +141,7 @@ fn advisory_to(result: Result<Outcome>, out: &mut impl Write) {
let text = match result {
Ok(outcome) if outcome.offenders.is_empty() => passed_text(&outcome),
Ok(outcome) => format!(
"warning: not blocking yet, but this will fail once the limit is enforced\n{}",
"warning: this hook does not block the commit, but the CI `clippy` job fails on it\n{}",
failed_text(&outcome)
),
Err(error) => format!("warning: the module-size check could not run: {error:#}"),
Expand All @@ -155,6 +156,10 @@ fn passed_text(outcome: &Outcome) -> String {
)
}

/// The document that states the rule for a file over the limit, which the failure
/// text points at.
const SPLIT_RULE_DOC: &str = "docs/architecture.md";

fn failed_text(outcome: &Outcome) -> String {
let mut text = format!(
"{} source file(s) over the limit of {MAX_PRODUCTION_LINES} production lines (of {} checked):",
Expand All @@ -170,6 +175,10 @@ fn failed_text(outcome: &Outcome) -> String {
offender.lines - MAX_PRODUCTION_LINES
);
}
let _ = write!(
text,
"\nto fix: split the file in the same PR, as a pure move with its tests moving along with the code; the rule is in {SPLIT_RULE_DOC}"
);
text
}

Expand Down Expand Up @@ -966,6 +975,59 @@ trait T {
assert!(offenders(&files, MAX_PRODUCTION_LINES).is_empty());
}

fn repo_file(relative: &str) -> String {
let root = Path::new(env!("CARGO_MANIFEST_DIR")).parent().unwrap();
fs::read_to_string(root.join(relative)).unwrap()
}

// The failure text gives remediation advice, so this asserts the advice
// together with the rule it points at: the document exists and states it.
#[test]
fn the_failure_text_points_at_a_split_rule_the_document_states() {
let text = failed_text(&Outcome {
checked: 1,
offenders: vec![Offender {
path: "src/big.rs".to_string(),
lines: MAX_PRODUCTION_LINES + 1,
}],
});
assert!(text.contains(SPLIT_RULE_DOC), "{text}");
assert!(text.contains("same PR"), "{text}");
assert!(text.contains("pure move"), "{text}");
assert!(text.contains("tests moving along with the code"), "{text}");

let doc = repo_file(SPLIT_RULE_DOC);
assert!(
doc.contains("splits it in the same change"),
"{SPLIT_RULE_DOC}"
);
assert!(doc.contains("a pure move"), "{SPLIT_RULE_DOC}");
assert!(
doc.contains("the tests moving with the code"),
"{SPLIT_RULE_DOC}"
);
}

// The advisory text says the CI job fails on an offender, and the docs say the
// check blocks merges. Both rest on this step being the bare strict command.
#[test]
fn the_ci_step_runs_the_strict_check_with_no_non_blocking_wrapper() {
let workflow = repo_file(".github/workflows/ci.yml");
let start = workflow
.find("- name: Source files are within the module-size limit")
.expect("the module-size step is in ci.yml");
let step = &workflow[start..];
let step = &step[..step[1..]
.find("\n - name:")
.map_or(step.len(), |n| n + 1)];
assert!(
step.lines()
.any(|line| line.trim() == "run: cargo xtask check-module-size"),
"{step}"
);
assert!(!step.contains("continue-on-error"), "{step}");
}

#[test]
fn offenders_are_listed_largest_first() {
let files = [
Expand Down Expand Up @@ -1463,7 +1525,10 @@ trait T {
advisory_to(Ok(outcome), &mut out);
let text = String::from_utf8(out).unwrap();
assert!(text.contains("src/big.rs: 1600 production lines"), "{text}");
assert!(text.contains("not blocking yet"), "{text}");
// The claim about CI is proved by
// `the_ci_step_runs_the_strict_check_with_no_non_blocking_wrapper`.
assert!(text.contains("does not block the commit"), "{text}");
assert!(text.contains("CI `clippy` job fails on it"), "{text}");
}

#[test]
Expand Down
Loading