Skip to content

refactor(hcl2cdk): global setup/teardown along with parallel tests to speed the b… - #41

Merged
so0k merged 1 commit into
open-constructs:mainfrom
irfn:main
Mar 3, 2026
Merged

so0k merged 1 commit into
open-constructs:mainfrom
irfn:main

Conversation

@irfn

@irfn irfn commented Feb 20, 2026 •

Copy link
Copy Markdown
Contributor

…uild

Related PR

hashicorp/terraform-cdk#2133

Description

This PR optimizes CI for the @cdktn/hcl2cdk package with three changes that together save ~16 min of aggregate CI compute per run:

1. Jest globalSetup / globalTeardown — New files (globalSetup.ts, globalTeardown.ts) that run once before any test worker starts. The setup pre-generates all 12 provider bindings, initializes base projects for required languages, and pre-caches provider schemas via CDKTF_EXPERIMENTAL_PROVIDER_SCHEMA_CACHE_PATH. This eliminates all terraform init / terraform providers schema calls from test workers, which was the root cause of cache collisions under parallelism.

2. Parallel test execution — Removed --runInBand from hcl2cdk test scripts and refactored the test helper (convert.ts) to read from a pre-generated fixtures manifest instead of running expensive cdktn init/cdktn get per-worker. This is now safe because all shared resources are generated once in globalSetup. The hcl2cdk test step drops from ~11m27s (serial) to ~7 min (parallel).

3. Scoped builds in CI — Replaced the full monorepo yarn build in .github/workflows/unit.yml with per-package lerna run --scope builds. Nx's build.dependsOn: ["^build"] automatically resolves transitive dependencies. A conditional step adds cdktn-cli build + yarn package only for the hcl2cdk job that needs the CLI binary. This saves ~35s per job across the 14 non-hcl2cdk jobs (~8 min aggregate).

The design choice to use a shared globalSetup with a JSON manifest (rather than trying to fix per-worker locking) was driven by the history here: parallel workers independently running cdktf get/cdktf init caused Terraform plugin cache collisions, and previous attempts to fix the parallelism (41a2e7f, f2e06fd) failed, leading to the --runInBand workaround (5ba985c). Pre-generating everything once sidesteps the problem entirely.

Verified locally: 27/28 suites passed (1 skipped), 279/280 tests passed (1 skipped), 202 snapshots matched, 0 Terraform cache collision failures.

Checklist

@irfn
irfn requested a review from a team as a code owner February 20, 2026 16:33
@irfn irfn changed the title chore: global setup/teardown along with parallel tests to speed the b… refactor(hcl2cdk): global setup/teardown along with parallel tests to speed the b… Feb 20, 2026

@so0k so0k left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - the CI is running and only some small concerns.

Comment thread docs/ci-optimization-hcl2cdk.md Resolved
Comment thread docs/ci-optimization-hcl2cdk.md Resolved
Comment thread docs/ci-optimization-hcl2cdk.md Resolved
Comment thread packages/@cdktn/hcl2cdk/package.json Resolved
@so0k
so0k merged commit 5c259e9 into open-constructs:main Mar 3, 2026
551 of 627 checks passed
jsteinich added a commit that referenced this pull request Mar 27, 2026
### feat
- feat(gha): publish Maven artifacts to Maven Central
[\#64](#64)
- feat: restore NuGet.org publishing
[\#50](#50)
- feat(provider-generator): don't emit property params
[\#43](#43)


### fix
- fix(cli): Pin init template to TypesScript 5.x
[\#74](#74)
- fix(gha): fix Go releases and update git identity
[\#65](#65)
- fix(provider-generator): handle inline comments with brackets in
variable type parsing
[\#61](#61)
- fix: DEP0044 in TerraformOutput
[\#2](#2)


### chore
- chore: Upgrade dependencies for all
[\#69](#69)
- chore(gha): Adding base14 Scout CI/CD observability workflow to
instrument all the github actions.
[\#52](#52)
- chore(deps): Upgrade minimatch to 5.1.8
[\#49](#49)
- chore: add cdktn keyword to all packages
[\#46](#46)
- chore: update links to point to cdktn.io
[\#45](#45)
- chore: update more references to cdktf
[\#44](#44)
- chore: fix typo in CONTRIBUTING.md
[\#42](#42)
- chore(deps): Upgrade validator to 13.15.22
[\#38](#38)
- chore(deps): Upgrade glob to 10.5.0
[\#30](#30)
- chore: Upgrade dependencies for all
[\#22](#22)
- chore: Upgrade dependencies for lib
[\#18](#18)
- chore(deps): bump the github-actions-backward-compatible group with 4
updates [\#15](#15)
- chore(deps): bump the github-actions-breaking group with 11 updates
[\#14](#14)


### refactor
- refactor(hcl2cdk): global setup/teardown along with parallel tests to
speed the b…
[\#41](#41)
@github-actions

github-actions Bot commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

I'm going to lock this pull request because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Apr 2, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants