Skip to content

ci: one lane per plugin, so every plugin can be worked on in parallel - #37

Merged
michielbdejong merged 3 commits into
mainfrom
claude/parallel-plugin-workflow-de8ef1
Sep 22, 2026
Merged

michielbdejong merged 3 commits into
mainfrom
claude/parallel-plugin-workflow-de8ef1

Conversation

@michielbdejong

Copy link
Copy Markdown
Contributor

Splits the single test job into one CI lane per plugin, so several plugins can be developed at once and a run reports every broken one instead of the first.

What changed

Job graph. changes -> build-server -> { lane (matrix), hosting-surface, e2e-plugin-system }, plus an unsharded shared-checks. The cargo build is the entire cost of this pipeline, so it runs once and uploads the binary; the lane matrix fans out with fail-fast: false. Wall clock becomes build + slowest lane rather than the sum of every lane.

One source of truth. The lane list had been written out three times inside ci.yml — the changes job's outputs:, its filters:, and every step's if:. That is how integrations/calendar/ shipped with no filter entry and never ran in CI at all. It now lives once in integrations/lanes.json, read through integrations/tooling/lanes.mjs, which generates both the paths-filter block and the matrix. lanes.test.mjs fails if a plugin directory has no lane.

Ports. Derived as 19100 + index*10 + role, blocks of 10, indexes permanent. The same derivation in CI and locally, so a failed CI log pastes into a terminal.

One entry point. node integrations/tooling/run-lane.mjs <lane> is what CI runs and what a developer runs. It starts the stack on that lane's ports, runs its tiers cheapest-first, tears down. A bound port is reported as the lane that owns it; a missing server binary prints the cargo build line instead of an async spawn ENOENT after a 90s timeout.

Spec split. At the pinned commit plugins.spec.ts holds ten tests, of which four touch a real integration — one Pets, one Notion, two Clockify. They move to integrations/<lane>/e2e/*.spec.ts, verbatim apart from import paths. The other six drive the generic plugin editor and sandbox and stay upstream, running in e2e-plugin-system alongside plugin.spec.ts.

The one constraint that does not come for free

VITE_PLUGIN_CATALOG_URL and VITE_INTEGRATION_PROXY_URL are baked into the SPA by build.rs — ci.yml even greps dist to prove it — so a binary built for one port set cannot be repointed at another. The typecheck/unit/live tiers are runtime-configured and parallelize freely; the e2e tier cannot. In CI that costs nothing (each matrix job is its own runner, so every e2e lane reuses the same canonicalPorts). Locally it means one e2e lane at a time, behind a lockfile.

integrations/HANDOFF-runtime-urls.md is the atomic-server task that removes this. The useful finding there: both URLs are already runtime-overridable via the plugin-catalog-url and integration-proxy-url localStorage keys — the VITE_* vars only supply the default. So the work is seeding those keys before first fetch (most likely via storageState, which playwright.config.ts already uses for viewTransitionsDisabled), not building runtime config from scratch. That doc also lists the four tests to delete upstream when the pin next moves, or they will run twice.

Two bugs the lanes surfaced by running code that had not been run

  • integrations/timesheets/ could never typecheck or unit-test. It imports devonian/platform-lenses/clockify; devonian/ is a sibling package here, not an installed dependency, so nothing links it into a reachable node_modules. The lens in turn imports @tomic/lib, same problem. Both are now aliased in vitest.config.ts and mapped in tsconfig.json, the same shape as the existing vitest alias. The old workflow ran these exact commands — the failure was simply never reached.
  • canonicalPorts held a "//" comment key, and every value in that object is iterated as a port, so the comment was reported as a bound port. Note moved to a sibling key; validateConfig now rejects a non-integer port; two tests cover it.

Verification

Run locally against the pinned commit 4969872c:

  • lanes, certify, dev-server and mock-proxy tooling tests — 29 passing
  • calendar, issue-tracker, localthought, timesheets — typecheck and unit tiers all green
  • certify.mjs --layer js — money, notion, pets all pass
  • oxlint and oxfmt clean across integrations/

Not verified locally: the live and e2e tiers, which need a built atomic-server binary (a full cargo build). This PR's own CI run is the first exercise of those paths, and of the build-server artifact hand-off between jobs.

Note on scope

integrations/money/ has a lane entry with no tiers, so it produces no job — its typecheck, bundle and fixture tests run inside the unsharded certification step. And MOCK_PROXY_PLATFORMS, which run-lane.mjs already passes, is ignored until the per-platform fixture work (§4 of PARALLEL_LANES.md) lands; every lane's mock proxy still serves the full built-in platform set.

🤖 Generated with Claude Code

Michiel de Jong and others added 3 commits September 22, 2026 13:53
The `test` job did everything in sequence behind a 75-minute timeout: build
atomic-server, then lint, certify, unit-test, live-test and run two Playwright
specs for every plugin in one shell. Wall clock was the sum of all lanes, and
`set -e` meant the first failing plugin hid the rest.

Split it into `changes -> build-server -> { lane (matrix), hosting-surface,
e2e-plugin-system }` plus an unsharded `shared-checks`. The cargo build is the
whole cost, so it runs once and uploads the binary; the lane matrix fans out
one job per plugin with `fail-fast: false`, so a run reports every broken
plugin instead of the first. Wall clock becomes build + slowest lane.

The lane list had been written out three times inside ci.yml (the `changes`
job's `outputs:`, its `filters:`, and every step's `if:`). That is how
integrations/calendar/ shipped with no filter entry and never ran. It now
lives once in integrations/lanes.json, read through integrations/tooling/
lanes.mjs, which generates both the paths-filter block and the matrix;
lanes.test.mjs fails if a plugin directory has no lane.

Ports are derived (19100 + index*10 + role) so CI logs paste into a terminal.
The e2e tier is the exception: VITE_PLUGIN_CATALOG_URL and
VITE_INTEGRATION_PROXY_URL are baked into the frontend by build.rs and cannot
be repointed at runtime, so every e2e run reuses lanes.json's canonicalPorts
and takes a lock locally. integrations/HANDOFF-runtime-urls.md is the
atomic-server task that removes that constraint.

CI and developers share one entry point: `node integrations/tooling/
run-lane.mjs <lane>` starts the stack (serve.mjs) on that lane's ports, runs
its tiers cheapest-first, and tears down. A bound port is reported as the lane
that owns it rather than as EADDRINUSE.

Also splits atomic-server's browser/e2e/tests/plugins.spec.ts: only two of its
eight tests touch a real integration, and they move here as
integrations/pets/e2e/pets.spec.ts and integrations/notion/e2e/notion.spec.ts
so each can be gated on its own directory. The other six drive the generic
plugin editor/sandbox and stay upstream. integrations/tooling/
playwright.config.ts and integrations/tsconfig.e2e.json make them runnable
from this repo; see the handoff for the upstream deletion that must follow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…evonian resolution

Re-splits plugins.spec.ts from 4969872c rather than 02cac45c: the spec grew by
587 lines between the two pins and now holds ten tests, not eight. Four touch a
real integration — the Pets one, the Notion one, and two new Clockify ones —
so timesheets gains an e2e tier alongside pets and notion. The other six drive
the generic plugin editor and sandbox and stay upstream.

Also fixes two things the lanes surfaced by running code that had not been run:

- integrations/timesheets/ could never typecheck or unit-test. It imports
  `devonian/platform-lenses/clockify`, and devonian/ is a sibling package in
  this repo, not an installed dependency, so nothing links it into a reachable
  node_modules; the lens in turn imports `@tomic/lib`, which has the same
  problem. Both are now aliased in vitest.config.ts and mapped in
  tsconfig.json, the same shape as the existing `vitest` alias. The old
  workflow ran these commands too — the failure was simply never reached.

- lanes.json's `canonicalPorts` held a `"//"` comment key, and every value in
  that object is iterated as a port number, so the comment was reported as a
  bound port. The note moved out to a sibling key, validateConfig now rejects a
  non-integer port, and two tests cover it.

serve.mjs now checks for target/e2e/atomic-server up front: spawn's ENOENT is
asynchronous, so without it a missing binary cost the full readiness timeout
before showing the cause.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
atomic-server#1621 landed on feat/plugin-debug — the task
integrations/HANDOFF-runtime-urls.md asked for. Its playwright.config.ts seeds
the plugin-catalog-url and integration-proxy-url localStorage keys through
`storageState`, from PLUGIN_CATALOG_URL / INTEGRATION_PROXY_URL, so the two
URLs no longer have to be compiled into the frontend by build.rs.

That removes the one asymmetry in the lane scheme. Every tier now uses its
lane's own derived port block, e2e included:

- lanes.json loses `canonicalPorts`, and run-lane.mjs loses the lockfile that
  serialized e2e runs across worktrees;
- build-server stops passing the two VITE_* URLs, and the step that grepped
  data-browser/dist to prove the bake had worked goes with the bake;
- index 9 is reserved (`sharedIndex`) for the two CI jobs that are not a lane,
  so they get a block from the same formula rather than a second fixed one.

Also fixes a coverage regression this branch introduced: the e2e-plugin-system
job ran only plugin.spec.ts, so the generic tests in plugins.spec.ts — which
the old single-job workflow did run — were not running at all. It now runs
both files.

#1621 also deleted the Pets and Notion tests from plugins.spec.ts, so the
lane copies added here are no longer duplicates. The two Clockify tests it left
behind still are, so timesheets gives up its e2e tier for now and its Clockify
coverage keeps running ungated in e2e-plugin-system. HANDOFF-runtime-urls.md is
rewritten as HANDOFF-e2e-split.md: what landed, and the upstream deletion that
has to happen in the same change as the next pin bump or those two tests run
twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@michielbdejong

Copy link
Copy Markdown
Contributor Author

Rebased onto atomic-server#1621, which landed the runtime-URL work this PR's handoff asked for. Pin bumped to 057e2b67.

The e2e tier is no longer the odd one out. Its config seeds plugin-catalog-url and integration-proxy-url via storageState from PLUGIN_CATALOG_URL / INTEGRATION_PROXY_URL, so the URLs don't have to be compiled in. Every tier now uses its lane's own derived ports:

  • canonicalPorts is gone from lanes.json, and with it the lockfile that serialized e2e runs across worktrees
  • build-server stops passing the two VITE_* URLs, and the step that grepped data-browser/dist to prove the bake worked goes with the bake
  • index 9 is reserved (sharedIndex) for the two non-lane jobs, so they draw from the same formula rather than a second fixed block

Also fixes a regression this branch introduced. The e2e-plugin-system job ran only plugin.spec.ts, so the generic tests in plugins.spec.ts — which the old single-job workflow did run — were running nowhere. It now runs both files.

On duplication: #1621 deleted the Pets and Notion tests upstream, so the lane copies here are no longer duplicates. It left the two Clockify ones, which still are — so timesheets gives up its e2e tier for now and its Clockify coverage keeps running ungated in e2e-plugin-system. HANDOFF-runtime-urls.md is rewritten as HANDOFF-e2e-split.md covering exactly that: the deletion has to land in the same change as the next pin bump, or those two tests run twice.

Re-verified locally at the new pin: 27 tooling tests, all four non-server lanes' typecheck and unit tiers, certify --layer js, lint and format. The live and e2e tiers still need this CI run — they want a full cargo build.

@michielbdejong
michielbdejong merged commit 9572bca into main Sep 22, 2026
10 checks passed
michielbdejong added a commit that referenced this pull request Sep 22, 2026
ci: fix the three e2e lane failures from #37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant