Repository navigation
chore(gha): parallelize provider binding generation for documentation examples - #244
Merged
jsteinich merged 2 commits intoJun 8, 2026
Conversation
X-Guardian
marked this pull request as ready for review
June 5, 2026 11:20
jsteinich
approved these changes
Jun 8, 2026
so0k
pushed a commit
that referenced
this pull request
Jul 3, 2026
🤖 Release PR — merge to cut a new release. Kept open and rebased as commits land on `main`. --- <details><summary>0.23.4</summary> ## [0.23.4](v0.23.3...v0.23.4) (2026-07-02) ### Features * **cli:** validate Terraform CLI feature versions ([#237](#237)) ([32d434c](32d434c)) * declared Terraform/OpenTofu target versions (validation foundation) ([#269](#269)) ([0aa2343](0aa2343)) * **lib:** S3-backend - Add `useLockfile` field and update `dynamodbTable` deprecated JSDoc ([#234](#234)) ([32a408f](32a408f)) * **lib:** validate Fn usage against Terraform/OpenTofu versions ([#268](#268)) ([cdf8115](cdf8115)) ### Bug Fixes * **cli:** don't mistake terraform error output for a missing-variable prompt ([#267](#267)) ([0ea946f](0ea946f)) * **cli:** release terraform state lock when interrupting diff/deploy ([#284](#284)) ([de617db](de617db)) * **cli:** resolve prebuilt Java provider versions via repo1.maven.org ([#285](#285)) ([d5af07d](d5af07d)) * **gha:** Adding CHANGELOG.md to prettier ignore list ([#258](#258)) ([5160469](5160469)) * **gha:** pull jsii-terraform CI image from GHCR instead of Docker Hub ([#261](#261)) ([f19d34b](f19d34b)) * **provider-generator:** copy only publishable files into the jsii compile bundle ([#292](#292)) ([562d4c7](562d4c7)) * **provider-generator:** match shorthand provider sources against OpenTofu schema keys ([#293](#293)) ([fda0103](fda0103)) * **release:** add Changelog title for correct release-please insertion ([#255](#255)) ([004818a](004818a)) * **release:** drop devEngines.packageManager that broke npm publish ([ef508a6](ef508a6)) * **tools:** repair generate-function-bindings tool ([#273](#273)) ([821e1cf](821e1cf)) ### Miscellaneous Chores * Add Gradle to Mise config ([#271](#271)) ([1578270](1578270)) * add pnpm to mise ([#266](#266)) ([48ab91f](48ab91f)) * add uv to mise ([#260](#260)) ([163cf3d](163cf3d)) * **deps:** bump nrwl/nx-set-shas from 38457b511e60ee4e1da09255b658e480748a1af4 to afb73a62d26e41464e9254689e1fd6122ee683c1 ([#236](#236)) ([b5ac018](b5ac018)) * **deps:** bump the github-actions-backward-compatible group across 1 directory with 7 updates ([#235](#235)) ([de67d9f](de67d9f)) * **deps:** replace uuid dependency with node:crypto.randomUUID ([#291](#291)) ([6d3657a](6d3657a)) * ESLint ignore test edge-provider-bindings ([#270](#270)) ([ba407bb](ba407bb)) * Fix Verdaccio logging config ([#272](#272)) ([aded64b](aded64b)) * **gha:** parallelize provider binding generation for documentation examples ([#244](#244)) ([0a9c955](0a9c955)) * Migrate package manager from yarn to pnpm ([#170](#170)) ([d23da4e](d23da4e)) * migrate planning artifacts to cdktn-planning [skip ci] ([968c358](968c358)) * **release:** add release-please automation for versioning ([#243](#243)) ([c759eec](c759eec)) * Remove dependency-pinner tool ([#250](#250)) ([42c27d5](42c27d5)) * Remove unused dependencies ([#281](#281)) ([c7de65e](c7de65e)) * **tests:** run verdaccio in-process ([#248](#248)) ([c17677d](c17677d)) * Update JSII version constraint ([#245](#245)) ([d89558f](d89558f)) * Update scheduled workflows to only run on open-constructs/cdk-terrain ([#256](#256)) ([ba408cf](ba408cf)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: open-constructs-cdktn[bot] <291052431+open-constructs-cdktn[bot]@users.noreply.github.com>
Contributor
|
I'm going to lock this pull request because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The C#, Java, Java-gradle, and Python
*-documentationexamples are the critical-path tail of the examples CI matrix — they finish at ~25–29 min while every other example is long done. Almost all of that time iscdktn getgenerating provider bindings, and it runs fully serially. This PR lets it run in parallel, mirroring howgo-documentationis already configured.Why
Profiling the
csharp-documentationjob showedcdktn gettook 504s of a 691s job — generating + JSII-compiling 6 provider bindings (aws and kubernetes dominate) one at a time. The serialisation is deliberate: these four examples pass--parallelism 1, becausecdktn getdefaults to--parallelism -1(all providers concurrently), and the heavy providers need ~13–17 GB peak RAM together — more than a standardubuntu-latestrunner (16 GB) can hold without OOM.go-documentationalready side-steps this: it runs ondepot-ubuntu-24.04-16with the default parallelism and finishes the same generation step in ~60s (17.6 GB peak).So the throttle and the runner choice are two halves of one decision. Lifting the throttle requires the bigger runner, and the bigger runner is wasted while the throttle is in place. This PR changes both together for the four JSII-translated documentation examples.
Approach
examples.yml. Theruns-onselector wasgo-documentation-only (ondepot-ubuntu-24.04-16); it now also routescsharp,java,java-documentation-gradle, andpythondocumentation examples todepot-ubuntu-24.04-8.go-documentationstays on-16(unchanged).typescript-documentationis intentionally excluded — TypeScript bindings aren't JSII-translated, so--parallelismdoesn't apply and the job is already cheap (~226s).--parallelism 1from thebuildscript of the four examples (csharp/documentation,java/documentation,java/documentation-gradle,python/documentation), lettingcdktn getfall back to its-1default and generate all providers concurrently — matchinggo-documentation.Measured impact (on the shipped
depot-ubuntu-24.04-8runners):csharp-documentation'scdktn getruns in 1m13s, down from 8m24s serial — a ~7× speedup — taking the job from ~11m31s to 4m21s. The other three are comparable:java-documentationget1m24s (job 4m33s),java-documentation-gradleget1m13s (job 4m29s),python-documentationget1m08s (job 3m42s) — each down from ~11–13 min to ~3.5–4.5 min. The serial--parallelism 1builds previously peaked at ~4 GB; running all providers concurrently peaks at 23–26 GB, which would OOM a 16 GBubuntu-latestrunner — confirming the throttle and the runner choice had to move together.Right-sizing the runner —
-8vs-16: the change was first measured ondepot-ubuntu-24.04-16(16 vCPU / 64 GB), then re-run ondepot-ubuntu-24.04-8(8 vCPU / 32 GB) to avoid over-provisioning. Both work:getis only ~10–13s slower on-8(fewer cores → slightly less compile concurrency), and peak RAM (23–26 GB) sits within the-8tier's 32 GB with ~6–9 GB headroom. Per-run variance (~30–60s) exceeds the-8-vs--16gap — at this provider count cores aren't the binding constraint — so the four examples ship on-8.Checklist