Skip to content

Split benchmarks with features to allow avoiding unnecessary compilation - #25321

Merged
alice-i-cecile merged 8 commits into
bevyengine:mainfrom
JaySpruce:split_benchmarks
Oct 5, 2026
Merged

alice-i-cecile merged 8 commits into
bevyengine:mainfrom
JaySpruce:split_benchmarks

Conversation

@JaySpruce

Copy link
Copy Markdown
Member

Objective

When messing with ECS internals, I tend to run benchmarks often to see what changes make a difference. Of course, even a small change in bevy_ecs means recompiling it, as well as anything else that depends on it. Since benchmarks share the same dependencies, that includes bevy_render, even though I'm only trying to run bevy_ecs benchmarks. bevy_render also happens to take way longer to compile than anything else. I'd like to be able to avoid compiling it.

Solution

Add features to benches to enable specific crates' benchmarks: feature bench_ecs for bevy_ecs, bench_render for bevy_render, etc.

All features are enabled by default, so default behavior should be the same.

The dependencies are normal dependencies instead of dev-dependencies now, but they're also now optional = true. I'm not really sure how/if stuff outside benches interacts with dependencies inside benches, but it's hopefully fine.

Testing

Ran all the benchmarks, both individually with each feature and all together with default features.

@JaySpruce JaySpruce added A-Build-System Related to build systems or continuous integration C-Benchmarks Stress tests and benchmarks used to measure how fast things are S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Aug 7, 2026
@kfc35 kfc35 added C-Usability A targeted quality-of-life change that makes Bevy easier to use D-Straightforward Simple bug fixes and API improvements, docs, test and examples labels Aug 18, 2026
@kfc35
kfc35 self-requested a review August 18, 2026 00:51
@Zeophlite Zeophlite added the S-Merge-Conflicts Merge conflicts :( Add this label on top of other S- labels. label Aug 22, 2026
@JaySpruce JaySpruce removed the S-Merge-Conflicts Merge conflicts :( Add this label on top of other S- labels. label Aug 24, 2026
@Zeophlite Zeophlite added the S-Merge-Conflicts Merge conflicts :( Add this label on top of other S- labels. label Sep 2, 2026

@hymm hymm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me. Just spent a bunch of time recompiling when I only cared about the ecs benches.

@JaySpruce JaySpruce removed the S-Merge-Conflicts Merge conflicts :( Add this label on top of other S- labels. label Sep 23, 2026

@kfc35 kfc35 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only thing that gives me pause is the removal of dev-dependencies and having the default feature include all of the bevy dependencies, but i assume that's fine

Comment thread benches/Cargo.toml
] }

# Other crates
glam = { version = "0.33.2", default-features = false, features = [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the rest of these still be considered [dev-dependencies]?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume they're small enough (relative to the Bevy crates) that it doesn't really matter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't really matter if they're in dev-dependencies or dependencies. This crate is only ever run through cargo bench which compiles with the dev-dependcies.

@kfc35 kfc35 added S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged S-Merge-Conflicts Merge conflicts :( Add this label on top of other S- labels. and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Oct 3, 2026
@JaySpruce JaySpruce added S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged S-Merge-Conflicts Merge conflicts :( Add this label on top of other S- labels. labels Oct 4, 2026
@hymm

hymm commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

@mockersf Is there any consideration for bevy metrics that this pr would need to handle? With default features this runs all the benches.

@alice-i-cecile
alice-i-cecile added this pull request to the merge queue Oct 5, 2026
@alice-i-cecile
alice-i-cecile removed this pull request from the merge queue due to a manual request Oct 5, 2026
@alice-i-cecile
alice-i-cecile added this pull request to the merge queue Oct 5, 2026
Merged via the queue into bevyengine:main with commit 430c448 Oct 5, 2026
45 checks passed
@JaySpruce
JaySpruce deleted the split_benchmarks branch October 5, 2026 17:56
JMS55 pushed a commit to JMS55/bevy that referenced this pull request Oct 5, 2026
Forgot to add new benchmarks to the default features when fixing
conflicts in bevyengine#25321, which I think would make it hard for CI to run them
(unless it always does `--all-features`, but I dunno).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Build-System Related to build systems or continuous integration C-Benchmarks Stress tests and benchmarks used to measure how fast things are C-Usability A targeted quality-of-life change that makes Bevy easier to use D-Straightforward Simple bug fixes and API improvements, docs, test and examples S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants