Skip to content

e2e: size the four shards by measured cost - #1893

Merged
joepio merged 2 commits into
developfrom
claude/project-thread-nrrfk7
Sep 29, 2026
Merged

joepio merged 2 commits into
developfrom
claude/project-thread-nrrfk7

Conversation

@joepio

@joepio joepio commented Sep 29, 2026

Copy link
Copy Markdown
Member

Before: the four e2e shards are cut by test count. On run 4783 (the timing-summary run, all green) they took 30.3, about 22.9, 27.1 and 26.5 min, so CI waits for shard 1.

After: PWTEST_SHARD_WEIGHTS=60:98:79:78 for the four-shard profile. Shard 1 ends in a run of cheap ai-* specs (about 8 s per test), so sixty tests instead of seventy-nine sheds about 200 s onto shard 2. Estimated shards: about 26.7, 26.3, 26.9 and 26.3 min, so the slowest drops from 30.3 to about 27 min.

How: weights are proportions, so a growing suite keeps the split; only a shard count with an entry in E2E_SHARD_WEIGHTS is weighted (Playwright wants one weight per shard, separated by colons). Estimates come from the per-file E2E-TIMING lines of run 4783 for shards 1, 3 and 4; shard 2's own log fell outside the fetchable tail, so its 22.9 min is from run 4774, and that is the one number to check on the first run with this change. Set membership was checked with playwright --list --shard at the new weights.


Generated by Claude Code

Run 4783 measured the shards at 30.3, about 22.9, 27.1 and 26.5 min: with equal
test counts, shard 1 also ends in a run of cheap ai-* specs. Sixty tests
instead of seventy-nine puts about 200s on shard 2 instead, which brings the
slowest shard from 30.3 to about 27 min. Playwright's PWTEST_SHARD_WEIGHTS takes
one weight per shard separated by colons; only a shard count with an entry is
weighted.
The first weighted run (4790) showed shard 2 heavier than estimated, because
its own timings were not available when the first weights were set. With every
shard's per-file seconds, 62:90:84:79 puts all four shards within 2252 to 2279
s of that run.
@joepio
joepio marked this pull request as ready for review September 29, 2026 18:58
@joepio
joepio merged commit 0f5dda3 into develop Sep 29, 2026
8 checks passed
@joepio
joepio deleted the claude/project-thread-nrrfk7 branch September 29, 2026 18:58
joepio added a commit that referenced this pull request Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant