ci: add a Required Checks aggregator job so main can have a required status check

[andygrove]

Which issue does this PR close?

Part of #5838. Closes #5860.

Rationale for this change

main requires no status checks today, only one approving review. #5838 needs one, since a merge queue only ever waits on required checks. Before we can add one we need a name that is actually safe to require, and right now we don't have one.

Every heavy job in ci.yml is a thin caller of a reusable workflow, and the check name a caller publishes depends on whether it ran:

Caller state	Check runs published
skipped by if:	one run named exactly PR Build (Linux), conclusion skipped
ran	only PR Build (Linux) / Spark 4.1, JDK 17 [exec] and friends, and no bare PR Build (Linux) at all

I confirmed both against real commits: bdd2aeb (md-only, everything skipped) and 36caf8e (heavy jobs ran). There's no name that reports in both cases, so requiring the bare name would block every code change, and requiring a nested name would block every docs-only change. Worse, both of those hang waiting for a check that never arrives rather than failing outright, and a required context that never reports also blocks the merge that would fix .asf.yaml. At that point only INFRA can remove the check by hand. apache/datafusion ran into the same class of problem in apache/datafusion#17538 and had to revert it in apache/datafusion#17629.

What changes are included in this PR?

A required_checks job at the bottom of ci.yml that needs: every other job and publishes a single flat Required Checks context on every event. It runs if: always() and treats skipped as a pass, so it only goes red when an upstream job reports failure or cancelled.

The job's name: is an expression rather than a literal. ci.yml also fires on labeled, and on that event POLICY deliberately skips the PR tier because it already ran at the same commit. A label run's aggregate therefore says nothing about the commit's applicable suites, but GitHub keeps only the most recent check run per name per commit, so publishing it as Required Checks would let a dependencies label landing a minute after a push mark the commit green while the real run was still going (the mechanism from #5007). Label runs publish as Required Checks (label run) instead, a name nothing requires. Skipping the job on labeled would not help, since a skipped check run still carries the name and still counts as passing.

Nothing requires it yet, and that's deliberate. It lands on its own so we can watch it report on real pull requests before .asf.yaml names it, since that's the step that's expensive to get wrong. The follow-up PR does that and turns the queue on.

dev/ci/check-ci-config.py also gains two invariants, in the same spirit as the ones already in there: every ci.yml job except docs has to appear in required_checks.needs, and once .asf.yaml does declare a required context for main, the job's name: has to keep matching it. Both sides of that pair are silent when broken. The job-id regexes accept - and uppercase, which GitHub allows, and the .asf.yaml parse is scoped to the main: entry under protected_branches so a release branch requiring its own context does not produce a false failure. The checker also requires the aggregator's name: to be the label-routing expression, with a labeled branch that differs from the required name, and rejects an .asf.yaml that requires the label-run name.

This PR also closes #5860. Once Required Checks is required, any red job evicts the PR from the merge queue, and the first two runs on this branch each failed in a step that had nothing to do with a test. So the steps that run after the verdict is known, or before any test starts, now retry or tolerate network failures:

• .github/actions/java-test: the test-report upload is continue-on-error: true. It runs on green jobs, nothing downstream consumes the reports, and a FinalizeArtifact 403 should not turn a passing run red. The two failure-only uploads are unchanged.
• iceberg_spark_test_reusable.yml: the shard-inventory upload goes through ./.github/actions/upload-artifact-retry. iceberg-spark-shard-coverage downloads it, so it already fell under the README rule for artifacts a later job consumes.
• pr_build_linux.yml: Lint Scala (syntactic) is split into a retried no-op cs launch scalafix:0.14.6 -- --version that populates the coursier cache, then the real check under cs launch --mode offline. A nonzero exit from the check can now only be a lint violation.
• ci.yml preflight: the actionlint download is retried, and the installer is fetched to a file instead of piped into bash.

The README's "Retrying flaky network operations" section documents all of this.

How are these changes tested?

preflight already runs check-ci-config.py, so the new invariants gate every PR from here on. I mutation-tested the failure modes against a clean baseline: dropping a job from needs, renaming the aggregator job while .asf.yaml requires it, typoing the context in .asf.yaml, adding a new heavy job that forgets to register, adding a hyphenated job id that forgets to register, adding a required context to branch-0.17 only, reverting the aggregator name to a literal, giving both branches of the name expression the same literal, and requiring the label-run name in .asf.yaml. All are caught with a specific message except the branch-0.17 case, which correctly passes.

actionlint and prettier --check "**/*.md" are clean. I ran the offline scalafix invocation locally against a warm coursier cache and it exits 0 on this tree, so --mode offline is compatible with the pinned scalafix:0.14.6 app descriptor.

The job is also observable on this PR itself. Over the first two runs it correctly went red on pr_build_linux: failure while treating the skipped spark_3_4, spark_4_0, and older Iceberg legs as passes. Both of those failures were infra flakes in steps that run after the test verdict is known (an artifact FinalizeArtifact 403, and a Maven Central connection reset in Lint Scala). That is the sort of thing the watch period was meant to surface, and it is why #5860 is folded into this PR rather than left for later.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BtAqq4YJsk8uk42c7vHk8B

[sunchao]

Correctness

Reviewed head e169d3968424aa418c4580e79211749a541492de against base 8320ae481b4eb916beebc9ed611473abb79c3e71. This addresses the unstable check names produced by reusable-workflow callers by adding a flat Required Checks job. The current .asf.yaml remains unchanged, so requiring the context and enabling the merge queue are deliberately separate steps.

Within one run, the dependency list covers all 13 other jobs except the documented site deployment. Including both preflight and changes prevents a failed prerequisite from disappearing behind skipped descendants. always() lets the aggregator inspect completed dependencies, and rejecting failure and cancelled is consistent with the documented dependency result states. The inner failure step's default success condition checks previous steps in this job. It does not undo the outer job's always(). GitHub dependency and expression semantics, status functions.

One P2 remains in the inline comment: label-triggered runs intentionally omit the normal PR suites but publish the same green aggregate name. A verdict based only on that run's needs does not establish that the commit's applicable tests passed. This needs to be addressed before this name can serve its intended role as the single required context. The needs context is limited to direct dependencies in the current run.

The new permissions: {} addresses the existing resolved CodeQL comment. The job needs no checkout, secrets, or token API access, including on fork PRs. Making diagnostic test-report upload nonfatal leaves Maven test failures fatal. The consumed Iceberg shard inventory retains fatal failure after its bounded retries. This PR changes no Spark expressions or operators, so it changes no Spark null, type, ANSI, error, or fallback semantics.

Validation: the current config checker passes locally. Eight in-memory guard cases cover dependency omission, stale names, a new hyphenated job, required-context matching and renames, and independent release-branch contexts. I also checked event coverage and all four documented status values for each dependency. These are local configuration checks, not a GitHub scheduler or branch-protection test. Preflight and syntactic Scala lint passed on merge bf8e498ae53f385f86cfd05cbb2ad384812f54b6, whose parents match this exact base/head and whose tree matches the head. The remaining CI is still running. No local native/JVM suites or live merge experiment ran.

Performance

The aggregator adds a short runner job after the selected suites, with no additional builds or API polling. Download warm-up retries are bounded and leave the actual lint invocation outside the retry loop. This limits repeated work without retrying test or lint failures. Current CI confirms the offline scalafix invocation succeeds, but no end-to-end time saving is claimed. There is no query-runtime change requiring a Spark microbenchmark.

Design

A flat status and a staged rollout before branch protection are appropriate for the reusable-workflow naming problem. The remaining design requirement is that every producer of that status represent complete applicable coverage, including repeated events on the same commit. The existing event policy was designed to avoid duplicate heavy work and now needs to be reconciled with the new aggregate verdict. The site deployment exemption is explicit and consistent with its post-merge purpose. Merge-group triggering belongs to the planned queue-enablement change and is not claimed to work here.

Abstraction & complexity

The result reducer is small and uses existing Actions dependency data. The config checker keeps job membership and the declared required name aligned without adding a runner dependency. Its line parser matches the current repository layout, and focused mutation checks exercise the new rules. The retry changes reuse the established upload action and keep the one-off tool warm-ups local. No additional abstraction is needed to resolve the event-coverage finding.

[sunchao]

Correctness

[P2] Keep label-only runs from publishing an incomplete required verdict

Could every run publishing this shared name validate the applicable PR-tier checks? ci.yml also runs on labeled, while compute-changes.py::event_allows deliberately excludes the ordinary PR tier for that event. Those jobs are therefore skipped even when the changed files require them, and this aggregator reports success after just preflight and change detection, or after the one opt-in suite. Its needs cannot observe pending or failed jobs in the separate commit run. A green label-run result is therefore not evidence that the commit's applicable suites passed. The shared-name reporting problem is already described in #5007. Before this becomes the single required context, please ensure each producer validates the complete applicable test set and add a label-event coverage regression. Simply skipping this job on label events would still emit a passing skipped check under that name.

[andygrove]

Confirmed. The label run's needs only sees its own run, and GitHub keeps the most recent check run per name per commit, so a dependencies label landing a minute after a push would have published a green Required Checks on top of a still-running or red commit run. Same mechanism as #5007.

Fixed in 8074e01 by making the check name depend on the event:

name: ${{ github.event.action == 'labeled' && 'Required Checks (label run)' || 'Required Checks' }}

Label runs now publish under a name nothing requires, so they can never overwrite the commit run's verdict, and they still give the committer an honest aggregate for the opt-in suite they asked for. Skipping the job on labeled was not an option for the reason you gave: a skipped check run still carries the name and counts as passing.

check-ci-config.py enforces the shape: the name must be that expression, the labeled branch must differ from the fallback, the fallback is what has to match the .asf.yaml context, and .asf.yaml may never require the label-run name. Mutation-tested each of those (literal name, identical branches, requiring the label-run name, renaming the fallback with a context declared) plus the earlier cases, all caught with a specific message.

Merging labeled back into the PR tier was the other option and I did not take it, because it re-creates the duplicate-pipeline cost the label policy exists to avoid.

[andygrove]

Merged. Thanks @sunchao and @blaginin.