Repository navigation
ci: shard the Windows gate and balance PR shards by measured duration - #3863
Merged
Daniel Meppiel (danielmeppiel) merged 2 commits intoNov 18, 2026
Merged
Conversation
Run the Windows Compatibility Gate as three serial, duration-balanced shards on parallel windows-latest runners behind a fan-in that keeps the required check name and fails on any missing, skipped, cancelled or failed shard. Split Lifecycle Smoke into three shards and balance both lifecycle and Build & Test shards by measured durations with scripts/ci_sharding.py, retiring the count-weighted lifecycle sharder. Seeds are weights, not allowlists; every lane still partitions its selection exactly once. Refs #3581 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ction Average the Windows compatibility per-test durations over two hosted runs to damp runner variance, and record the redeployed tests.instructions.md content hash so apm audit --ci and the integrity unit test stay green. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Daniel Meppiel (danielmeppiel)
marked this pull request as ready for review
November 18, 2026 09:26
Daniel Meppiel (danielmeppiel)
requested a review
from Sergio Sisternes (sergio-sisternes-epam)
as a code owner
November 18, 2026 09:26
Daniel Meppiel (danielmeppiel)
deleted the
danielmeppiel-ci-windows-gate-sharding
branch
November 18, 2026 09:28
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block the CI sharding changes.
No open issues 🎉
What changed in this PR
This PR advances the CI performance work in #3581 by parallelizing the Windows compatibility gate and using measured durations to balance required test lanes, while preserving required-check names and test selections.
Changes:
- Splits Windows compatibility testing into three serial shards on parallel runners, with an aggregate requiring all shards to succeed.
- Moves lifecycle smoke testing to three duration-balanced shards and switches the two Linux unit shards to the shared sharding plugin.
- Updates workflow contracts, contributor documentation, and deployed instruction hashes.
| File | Description |
|---|---|
tests/unit/test_windows_compat_gate_workflow.py |
Verifies Windows shard configuration and aggregate failure handling. |
tests/unit/scripts/test_lifecycle_smoke_sharding.py |
Removes tests for the retired lifecycle-specific sharder. |
tests/unit/scripts/test_ci_sharding.py |
Adds composite xdist-group affinity coverage. |
tests/quality/test_ci_topology.py |
Updates lifecycle contracts and partition checks for three shards. |
scripts/lifecycle_smoke_sharding.py |
Removes the lifecycle-specific sharding plugin. |
docs/src/content/docs/contributing/integration-testing.md |
Documents duration-based sharding and updated local commands. |
apm.lock.yaml |
Refreshes deployed test-instruction hashes. |
.github/workflows/ci.yml |
Configures duration-balanced lanes and required shard aggregates. |
.github/instructions/tests.instructions.md |
Updates deployed guidance for the sharded Windows gate. |
.github/ci-durations/windows-compat.json |
Adds measured Windows compatibility scheduling weights. |
.apm/instructions/tests.instructions.md |
Updates source guidance for the sharded Windows gate. |
Review effort: Balanced
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Halve PR time-to-
gateby removing the three long poles on the required path.gatecreated -> successRecent baseline across PRs: median 11:54, p90 13:16. These are single runs, not new medians.
Build & Test runner variance is visible: on the previous head's run the same partition took 251 s vs 224 s (4:35 slowest job), when every required check finished 5:54 after start.
Why the old run was slow:
Changes (all on standard runners, no new dependencies, no ruleset edits):
windows-latestrunners. Execution is unchanged (no xdist, so no MAX_PATH--basetempworkaround). A fan-in job keeps the required nameWindows Compatibility Gateand fails on any missing, skipped, cancelled or failed shard. Seed:.github/ci-durations/windows-compat.json, per-test times averaged over serial job 134549967146 and this PR's first sharded run.linux-x86_64.jsonseed. The count-weightedscripts/lifecycle_smoke_sharding.pyis retired in favour of the canonicalscripts/ci_sharding.py; its composite-label test moves totest_ci_sharding.py..github/ci-durations/linux-unit.json(local full-suite measurement) instead of pytest-split's naive split. Per-shard coverage stays above the 60% floor (75.00% / 75.22%).apm.lock.yamlrecords the redeployedtests.instructions.mdhash (edited in.apm/and redeployed).EXPECTED_CHECKSis unchanged. Seeds are weights, never allowlists.Partition proof (0 missing / 0 overlap vs control)
Local collection, control vs union of shards:
-m windows_compat)Hosted: the Windows shards selected 414 + 288 + 368 = 1,070 = the control selection. At runtime
ci_sharding.partitionfails closed unless the shards are nonempty and partition the selection exactly.Issue and approved scope
Issue: Refs #3581
Human scope-approval comment: the maintainer (Daniel Meppiel (@danielmeppiel)) directed this bounded follow-up in the coordinating session ("but the ci session was working on additional gains (e.g. parallelizing windows compatibility gate job)"). No public scope-record comment is cited for this follow-up; this is an automation-written record, not a human approval comment.
Does this PR complete the issue, or what remains? Partial. It implements item 1's goal (Windows gate parallelism) via cross-runner sharding instead of in-runner xdist, and adds durations-based sharding for the PR lanes. Concurrency cancellation, apt time-boxing and the pylint bump remain open in #3581 / #3582.
Type of change
Testing
Tested locally
All existing tests pass
Added tests for new functionality (if applicable)
pytest tests/unit/test_windows_compat_gate_workflow.py tests/unit/scripts/test_ci_sharding.py: 59 passedpytest tests/quality/test_ci_topology.py: 189 passedapm audit --ci: all 10 checks passedHosted: all 26 PR checks pass on head 35c1a45 (3 skipped by design)
Lint contract (ruff check, ruff format --check, pylint R0801, lint-auth-signals, lint-architecture-boundaries): all clean
Spec conformance (OpenAPM v0.1)