Add CodSpeed continuous benchmarking - #88
Merged
Merged
Conversation
Add a pytest-codspeed benchmark suite covering the build pipeline, page rendering, file collection, navigation, configuration loading, the search index and the shared utilities, plus a GitHub Actions workflow that runs them on every pull request.
Contributor
Author
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
3 tasks
shenxianpeng
added a commit
that referenced
this pull request
Aug 12, 2026
Extend the benchmark suite merged in #88 with the scenarios needed to track the build-performance work for upstream issue mkdocs/mkdocs#3695: * full builds of the deterministic corpus at several sizes (10/50/200 pages by default, overridable via MKDOCS_BENCH_SIZES for local scaling studies) - a local run at 100/400/1600 pages measured 7.8/9.3/22.1 ms per page, i.e. clearly superlinear growth * a dirty rebuild with no modified files, approximating the 'mkdocs serve --dirty' inner loop * a single page rendered against the largest site's navigation, isolating the O(N^2) sitewide template-rendering term corpus.write_site() gains optional sections/pages_per_section parameters (defaults unchanged, existing benchmarks unaffected).
3 tasks
shenxianpeng
added a commit
that referenced
this pull request
Aug 12, 2026
shenxianpeng
added a commit
that referenced
this pull request
Aug 12, 2026
* Add scaling and serve-loop benchmarks to the CodSpeed suite Extend the benchmark suite merged in #88 with the scenarios needed to track the build-performance work for upstream issue mkdocs/mkdocs#3695: * full builds of the deterministic corpus at several sizes (10/50/200 pages by default, overridable via MKDOCS_BENCH_SIZES for local scaling studies) - a local run at 100/400/1600 pages measured 7.8/9.3/22.1 ms per page, i.e. clearly superlinear growth * a dirty rebuild with no modified files, approximating the 'mkdocs serve --dirty' inner loop * a single page rendered against the largest site's navigation, isolating the O(N^2) sitewide template-rendering term corpus.write_site() gains optional sections/pages_per_section parameters (defaults unchanged, existing benchmarks unaffected). * Update release notes for CodSpeed benchmarking suite (#88, #89)
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds continuous performance measurement to MkDocs NG with CodSpeed. Every pull request now runs a benchmark suite in CI and reports how the change affects build and rendering performance.
What was added
Benchmark suite (
benchmarks/) — 23 benchmarks built onpytest-codspeed, run in CPU simulation mode. The corpus is a generated MkDocs project (31 Markdown pages across 5 nested sections, plus static assets) produced from a fixed pseudo-random seed, so the workload is identical on every run and results only move when MkDocs itself changes.test_build.pymkdocs build(with and withoutuse_directory_urls), Markdown population of all pages, theme template renderingtest_structure.pyget_files,get_navigation,Filecreation, file lookups, page rendering (regular and large page),get_toctest_config.pyload_configwith full validation, YAML parsing, loader constructiontest_search.pytest_utils.pynest_paths, tag strippingcorpus.py/conftest.pyCI workflow (
.github/workflows/benchmarks.yml) — runs on pull requests, pushes tomainandworkflow_dispatch(used by CodSpeed for backtests). It uses OpenID Connect for authentication, so no token secret is needed. Actions are pinned by commit SHA to match the convention used by the existing workflows.Developer ergonomics — a
benchHatch environment (hatch run bench:run) so the benchmarks can be run locally like the other checks, a note inCONTRIBUTING.md, and the CodSpeed badge inREADME.md.Verification
The suite was run locally under CodSpeed's CPU simulation instrument: 23 benchmarks passed and reported, including the full-site build (~1.5 s simulated), page population (~0.9 s) and theme rendering (~0.3 s). Benchmarks also pass as plain tests (
pytest benchmarks, ~3 s) so they stay cheap to run outside CI.ruff check,ruff format,isort,codespellandmarkdownlintwere run on the touched files.Next steps
mainestablishes the performance baseline; subsequent PRs will be compared against it.mkdocs serveincremental rebuilds).