Skip to content

Lift the quarantine on the E2E: generic plugin system job #71

Closed

Description

@michielbdejong

#70 quarantined the E2E: generic plugin system job, which runs atomic-server's own browser/e2e/tests/plugin.spec.ts and plugins.spec.ts against this repo's catalog. The job has job-level continue-on-error: true in .github/workflows/ci.yml, so it still runs and reports, but it no longer fails CI. This issue tracks lifting that. The other two quarantines are tracked in #52 (pets) and #68 (notion).

State at the old pin 057e2b67 (run 35838050390, job 107110335665)

5 failed, 1 flaky, 4 passed:

  • plugin.spec.ts:26 install a plugin: v1.0.0 never appears in the top-level dialog.
  • plugins.spec.ts:30 a published release is discoverable…: the Search templates and resource types searchbox is missing.
  • plugins.spec.ts:84, :152, the two Clockify tests: [data-integration=clockify] is missing. atomic-server 4bab16ee6 deleted both tests; they're gone at the current pin 50cf5151c, and the move here is timesheets: move the two Clockify e2e tests here from atomic-server #44.
  • plugins.spec.ts:831 a plugin proposes changes… and :883 a run whose target does not exist…: in the failure list, but I didn't read their error details.

Not yet checked at the current pin 50cf5151c. The first run there is https://github.com/ontola/atomic-plugins/actions/runs/35842095258.

Work

  1. Read the plugin-system job's results at 50cf5151c. Playwright traces are now uploaded as the playwright-report-plugin-system artifact, including test-results/.
  2. For each remaining failure, decide:
    • Upstream spec/UI drift: fix it in atomic-server on feat/plugin-debug, then bump .atomic-server-ref.
    • Depends on this repo's catalog: fix it here, or pass PLUGIN_CATALOG_URL differently. Since atomic-server b8eae0b49, leaving it unset makes upstream's config serve its own testdata/atomic-plugins-mock instead.
  3. When the suite passes at the pin, remove continue-on-error: true and its comment from the e2e-plugin-system job.

🤖 Generated with Claude Code

Activity

  1. michielbdejong commented on Sep 23, 2026

    @michielbdejong
    ContributorAuthor

    Triage at pin 50cf5151c (run 35842095258, playwright-report-plugin-system artifact)

    The result was 3 failed, 4 passed, 0 flaky. The two Clockify tests are gone at this pin, as expected. plugins.spec.ts:831/:883 from the old pin are now :71 (passes) and :123 (below).

    Spec Actual error Class Fix
    plugin.spec.ts:26 install a plugin dialog…getByText('v1.0.0') not found. The Install dialog rendered ontola/test-plugin 1.0.0 Upstream UI drift: strings from components added on feat/plugin-debug were missing from the wuchale catalogs, so production builds rendered them blank atomic-server 49d42cfc2 + b875d4a31
    plugins.spec.ts:17 a published release is discoverable and creates an independent draft [data-release]…getByText('Unverified') not found. Discover showed "No plugins to show here." with "Show experimental plugins" unchecked Upstream bug: the visibility toggle was lost on the navigation straight after it atomic-server 1a3a33c69
    plugins.spec.ts:123 a run whose target does not exist is blocked getByRole('button', {name: /Apply/}) not found (expected toBeDisabled) Upstream spec drift: a blocked plan now hides Apply atomic-server b875d4a31

    None depend on this repo's catalog, and none are flaky. All three fixes were already on feat/plugin-debug (the three commits after the pin), so atomic-server needs no new changes.

    Verified at b875d4a31

    I built locally with CI's recipe and ran CI's exact command, with PLUGIN_CATALOG_URL set to this repo's catalog:

    • 1 run with default retries: 7/7 passed.
    • 3 more runs, each on a fresh store and with --retries=0: 7/7 passed each time.

    The PR bumps .atomic-server-ref to b875d4a31 and removes continue-on-error from e2e-plugin-system.

    One caveat, which doesn't affect CI: re-running install a plugin against a store that an earlier run already used fails with a 401. The same zip yields the same Release subject, which stays private to the first run's agent. CI starts from a fresh store. That's an upstream test-isolation follow-up.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions