Skip to content

Skip build and tests on docs-only PRs such as release blogs #173

Description

@mpaulosky

Problem

Every release-blog PR (docs: add release blog for PR #N [skip-release]) changes only documentation (docs/blogs/, README.md, docs/README.md), yet runs the full pipeline: build, all 8 test projects (including the Testcontainers and Aspire ones), coverage, and CodeQL's C# analysis. On #167, #169 and #171 that was about 17 runner-minutes and 4 minutes of wall clock per PR, most of it spent proving that unchanged code still builds. Every merged PR produces one of these, so it doubles CI load and delays main catching up.

The largest costs per blog PR:

Check Time
CodeQL Analyze (csharp) ~140s
Tests: AppHost.Tests ~85s
Tests: IssueTracker.PlugIns.Tests.Integration ~80s
Build Solution ~65s
Each other test project ~60s

Constraint

ci.yml can't simply use paths-ignore on pull_request: Build Solution and Test Suite are required checks on main-rules, and a check that never reports blocks the merge (the comment on the trigger in ci.yml says so). A job that is skipped by its if: condition does report, and GitHub treats a skipped required check as passing. So the jobs have to run and skip themselves, not be left out.

Plan

  1. Detect docs-only PRs in ci.yml. Add a small changes job that diffs the PR against its base and outputs code=true|false. A PR is docs-only when every changed path matches docs/** or **/*.md. Anything else (including .github/**, scripts, *.cs, project files) counts as code, so the rule fails safe. Pushes and manual runs always report code=true.
  2. Gate the expensive jobs on it. build, discover-tests, test and coverage get if: needs.changes.outputs.code == 'true'. Build Solution then shows as skipped, which satisfies the ruleset.
  3. Keep Test Suite honest. The report job runs with always(). On a docs-only PR it should pass with a "docs-only change, tests skipped" summary; on a code PR it keeps failing when any test job failed or was cancelled. It must never turn a failed or cancelled run into a pass.
  4. Skip CodeQL on docs-only PRs. Add paths-ignore: ["docs/**", "**/*.md"] to codeql-analysis.yml's pull_request trigger. CodeQL isn't a required check, and its push and weekly scheduled runs still cover main.
  5. Keep the cheap checks. Lint Markdown, Lint YAML, Lint Actions, and the Python and hook tests keep running; they take seconds.
  6. Check auto-merge. pr-automerge.yml waits on workflow_run from Build and Test Suite and CodeQL. Confirm that a docs-only PR, with CodeQL not running and the build skipped, still gets picked up, by the remaining workflow_run events or the 15-minute sweep.

Verify

  • A docs-only PR shows Build Solution and the test jobs as skipped, Test Suite as passed, and auto-merges.
  • A PR that changes one .cs file plus docs still runs everything.
  • A PR where a test fails still fails Test Suite.
  • Measure a blog PR afterwards; the expected cost is a couple of runner-minutes.

Out of scope

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions