Repository navigation
ubuntu-slim runners time out after 15 mins while it usually takes <3 mins #64972
Description
Last successful run which landed a PR
git node land was successful
...
+ MULTIPLE_COMMIT_POLICY=--oneCommitMax
+ git node land --autorebase --yes --oneCommitMax 63949
+ cat output
- Loading data for nodejs/node/pull/63949
...Details: https://github.com/nodejs/node/actions/runs/30765170760/job/91542478131
First timeout
...
+ MULTIPLE_COMMIT_POLICY=--oneCommitMax
+ git node land --autorebase --yes --oneCommitMax 64772
Error: The operation was canceled.Details: https://github.com/nodejs/node/actions/runs/30783328197/job/91592125884
Activity
trivikr commentedon Aug 3, 2026
The retry attempt took close to 15 mins, but was successful
https://github.com/nodejs/node/actions/runs/30783328197/job/91601263185
trivikr commentedon Aug 3, 2026
This can be closed if issue doesn't persist.
Renegade334 commentedon Aug 3, 2026
This happens to GHA jobs on the slim runners from time to time.
aduh95 commentedon Aug 4, 2026
There's clearly been an influx of those, I suspect a change on GH side. The issue is that the CQ can time out after removing the commit-queuePRs queued for automated landing through the Commit Queue.
label, but before being able to actually merge, so the PR is effectively kicked out of the CQ without notifying anyone. We could add another GHA job to try clean up, but if the problem is GHA reliability, it might be hopeless to try to solve it with GHA 🫣
tools: lazy-abort failed PR merges in CQ
panva commentedon Aug 7, 2026
This also happens to linters.yml, format-cpp, lint-cpp, and lint-js-and-md i've re-ran more than enough over the span of the last 2 weeks
[-]commit-queue times out after 15 mins while it usually takes <3 mins[/-][+]ubuntu-slim runners time out after 15 mins while it usually takes <3 mins[/+]aduh95 commentedon Aug 8, 2026
I wonder if it's a memory issue, in which case maybe adding some ESLint cache might help for
lint-js-and-md. Switching back toubuntu-latestis also an option, but a costly one (we prepare security releases from a private fork, where we have to pay for GHA usage)tools: delay removal of `commit-queue` label
tools: lazy-abort failed PR merges in CQ
tools: delay removal of `commit-queue` label
tools: lazy-abort failed PR merges in CQ
tools: delay removal of `commit-queue` label
tools: lazy-abort failed PR merges in CQ
9 remaining items
MikeMcC399 commentedon Sep 21, 2026
I'm also continuing to see this issue occurring. It causes delays and additional actions for team members who need to re-run failed workflows.
I have lost count of the number of times that I've restarted the
lint-js-and-mdjob for contributor PR submissions after it has hit the 15 minute limit, running on 1 CPU.What is the financial impact of returning to
ubuntu-latest?MikeMcC399 commentedon Sep 29, 2026
What about migrating the job
lint-js-and-mdin the GitHub Actions workflow .github/workflows/linters.yml fromubuntu-slimtoubuntu-26.04-arm? Although other jobs may be affected, I'm seeing this one fail a lot, just in the PRs that I've been in involved in. Even clean commits to themainbranch are being hit (example https://github.com/nodejs/node/actions/runs/36535043076/job/109297044751).1 CPU and 5 GB that
ubuntu-slimoffers is clearly not enough, and is causing the joblint-js-and-mdto regularly time out after 15 minutes. This wasn't happening before the job was migrated fromubuntu-latesttoubuntu-slim.4 CPUs and 16 GB that
ubuntu-26.04-armincludes would speed things up and stop the timeouts occurring.inoway46 commentedon Sep 30, 2026
+1 to moving this job to ARM. I tested
lint-js-and-mdonubuntu-26.04-armand observed no timeouts in 256 runs. Separately, #66408 improves performance on both runners, but slim still times out.“after” includes the optimization in #66408.
ubuntu-slim(before)ubuntu-slim(after)ubuntu-26.04-arm(before)ubuntu-26.04-arm(after)* Costs use private rates and per-job minute rounding. Private ARM has 2 CPUs versus 4 in these public runs, so actual private costs remain unverified.
Representative batches: baseline slim/ARM (
combined-slim/combined-arm), optimized ARM.MikeMcC399 commentedon Sep 30, 2026
@inoway46
Would you want to submit a PR for this change, since you have done all the preparation? It looks like it would be a positive outcome.
inoway46 commentedon Sep 30, 2026
FYI @MikeMcC399, I closed #66409 after feedback that reducing the workload or using caching would be preferable to changing the runner.
MikeMcC399 commentedon Sep 30, 2026
We do need a solution soon. Viewing the list https://github.com/nodejs/node/actions/workflows/linters.yml from yesterday (CEST timezone) Sep 29, 2026, there are 6 instances of cancelled workflows due to
lint-js-and-mdtimeout:https://github.com/nodejs/node/actions/runs/36577916798/job/109438217698
https://github.com/nodejs/node/actions/runs/36539193751/job/109310277791
https://github.com/nodejs/node/actions/runs/36535043076/job/109297044751
https://github.com/nodejs/node/actions/runs/36535043076/job/109297044751
https://github.com/nodejs/node/actions/runs/36508967793/job/109216667037
https://github.com/nodejs/node/actions/runs/36492431339/job/109163934793
This does not show instances where
lint-js-and-mdtimed out and a repo member re-ran the failed test.If there is another alternative way of solving this that can be implemented soon, then that would be fine. The runner could however be migrated to the more powerful one as an interim solution until the other proposed solution is available.
trivikr commentedon Sep 30, 2026
I think we can do both?
inoway46 commentedon Oct 1, 2026
Yes, we can do both. Sorry, my previous comment was misleading.
MikeMcC399 commentedon Oct 3, 2026
#66412 adds npm caching for dependencies & ESLint caching, both for
lint-js-and-md. In my own tests it did not solve the issue here. We should get this merged and see what the result is in practice.tools: add cache to `lint-js-and-md`
tools: avoid repeated subtree scans in must-call-assert
tools: add cache to `lint-js-and-md`
tools: avoid repeated subtree scans in must-call-assert
tools: add cache to `lint-js-and-md`
tools: avoid repeated subtree scans in must-call-assert