Skip to content

ci: stop the plan job fetching the kernel's history; split the host tests - #254

Merged
maralcbr merged 3 commits into
integration/aurorafrom
ci/fast-plan
Oct 29, 2026
Merged

maralcbr merged 3 commits into
integration/aurorafrom
ci/fast-plan

Conversation

@maralcbr

@maralcbr maralcbr commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Summary

Merge-queue run 40621870017 (#249) took 18m55s. 6m48s of that was plan, and 6m40s of the plan was one silent step: Look up earlier builds.

Root cause: lookup-build.py checks for the base commit with git cat-file -e. The plan checkout is a partial clone (filter: blob:none), so the missing object is fetched on demand, without a depth. That pulls the kernel's whole commit and tree history (index-pack ... pack_header=2,8642584, 8.6M objects) before the intended --depth=1 fetch can run.

 ensure_commit(repo, base)
-  git cat-file -e base           # partial clone: lazy fetch of base + all history
+  git --no-lazy-fetch cat-file -e base
   if missing
     git fetch --depth=1 --filter=blob:none origin base   # ~1 s, 3.5 MiB

Host tests, the long pole once the package is quicker (light PR runs, the merge-queue reuse in the follow-up PR):

 host-tests:
-  needs: plan                    # reads no plan output
-  matrix: [tools, aurora-sep]    # aurora-sep: 231 s suite, then the serial suite
+  matrix: [tools, aurora-sep, aurora-sep-slow]

aurora-sep-slow holds test_m3max_kit.py, test_m3_pro_mesa.py and test_m3_air_default.py, so the serial test_m3_max.py no longer waits behind them. Suites, arguments and failure classes are unchanged. The manifest test now takes the allowed groups from the workflow matrix and fails if the manifest uses a group the matrix doesn't run.

Not in this PR: a compiler cache for package. The build runs in omarchy-pkgs' container (fixed mounts and env, no ccache in the image, -j${OMARCHY_BUILD_JOBS:-10} not passed through), so it needs an omarchy-pkgs change, and it changes how attested packages are built. That's a separate proposal. Letting the merge queue take a PR run's results for the same tree is #255, which needs an explicit opt-in.

Evidence

  • Before: lookup-build.py on the merge-queue commit e3ffe88, in a checkout made the way actions/checkout makes it: ensure_commit 625 s, everything else < 3 s. On GitHub: 6m40s (merge queue), 4m01s (PR run 40616429413).
    After: same commit, same checkout: 4 s in total, same key, kernel tree, base tree and baseline run.

  • Before: EnsureCommitTest fails on the old code (first commit present after ensure_commit(base)).
    After: passes; 132 tests OK, actionlint clean, check-commits.py clean.

  • Real runs, before vs this PR (requested build 40627333531, checks run 40627429421):

    job before: queue 40621870017 / PR 40616429413 after: this PR
    plan 6m48s / 4m09s 10s (lookup step 2s)
    host tests (aurora-sep) 7m51s / 6m20s 4m08s
    host tests (aurora-sep-slow) — 4m41s
    host tests (tools) 5m37s / 4m39s 5m41s (now the longest host job)

    This PR changes only CI files, so its build key matched the branch's trusted build: package was reused (48s), and KUnit and boot were skipped. Run wall time: 5m51s.

Measured in a real merge queue: a scratch copy, omacom/linux-aurora-ci-scratch (deleted after the test; run IDs below are from it), running this PR and #255 on the same Namespace profile. Queue entries with a full build took 10m39s (40807898874) and 10m55s (40807900176), against 18m55s before. Each ran plan in 7–8 s, then package (~9 min), boot (~1 min) and result. The host tests (≤ 5m41s) stay off that path. Push runs with a base gain the same; nightly and manual runs don't call ensure_commit, so they don't change.

Merge Danger

Door: two-way

CI only: no kernel, recipe or ruleset change, and the required check means the same. Revert restores the old behaviour.

Blast Radius: CI

Host tests now start even if plan fails (result still fails the run). One more free GitHub runner per run.

Sign-off: @chriskearney (repo), @ryanrhughes (infra; one more GitHub-hosted job per run), @emirb (FYI: Namespace use is unchanged).

lookup-build.py checks whether the base commit is in the checkout with
git cat-file -e. The plan job's checkout is a partial clone (blob:none),
where asking for a missing object fetches it on demand, and that fetch
has no depth: it brought in every commit and tree of the kernel's
history (8.6 million objects) before the intended depth-1 fetch could
run. It took 4 to 7 minutes of every merge queue, push and full pull
request run.

Probe with --no-lazy-fetch (Git 2.45 and later; on older Git the probe
fails and the cheap depth-1 fetch runs instead).

Signed-off-by: Marcelo Alcantara <maralc@gmail.com>
The host tests read nothing from the plan job, so they need not wait for
it. The aurora-sep group was the longest job after the package: its
parallel batch waited on test_m3max_kit.py (231 s) before the serial
test_m3_max.py could start. Move its three longest suites into an
aurora-sep-slow group on a runner of their own; GitHub's runners cost
nothing for this repository.

The manifest test now takes the allowed groups from the workflow's
matrix and fails when a group the manifest uses is not in it, so no
suite can silently stop running.

Signed-off-by: Marcelo Alcantara <maralc@gmail.com>
@maralcbr maralcbr added the ci:build Build, test and boot the kernel for this PR now label Oct 28, 2026
@maralcbr
maralcbr marked this pull request as ready for review October 29, 2026 04:03
@maralcbr
maralcbr merged commit 6388ea7 into integration/aurora Oct 29, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:build Build, test and boot the kernel for this PR now

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant