Repository navigation
Add containerised tier-2 acceptance tests against a real remoteStorage server - #6
Merged
Merged
Conversation
…e server Tier 1 (npm run test:smoke) stays as the fast, backend-less Playwright suite. This adds tier 2: `make accept` / `npm run test:accept` bring up litewrite, a real armadietto remoteStorage server, and Playwright as three docker-compose services, then drive the actual widget connect, create/persist, reload-recovery, and cross-context sync flows through the real OAuth round trip - no mocks, no third-party accounts. Image tags are pinned by digest (node:20-alpine, the official mcr.microsoft.com/playwright image so browsers don't need downloading, and remotestorage/armadietto-monolithic). Verified locally: all four scenarios pass end to end, and pointing the tests at a wrong storage URL makes the suite fail loudly rather than silently passing. Adds .github/workflows/acceptance.yml to run the same command on PRs, and a CLAUDE.md documenting both tiers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gnL38EmFRmq4MDicaJQ9B
CI failed with remotestorage never reporting healthy in time - most likely starved of CPU/disk while the ~750MB Playwright image was being pulled and extracted concurrently. Widen the healthcheck budget (2s/2s/30 -> 3s/5s/40) and have accept.sh print `docker compose logs` before tearing down on any failure, so the next red run shows the containers' own output instead of just "is unhealthy". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gnL38EmFRmq4MDicaJQ9B
The previous CI failure's own log dump showed both app and remotestorage had actually started fine by the time compose gave up waiting on healthchecks - so the healthcheck timeout bump alone didn't address the real cause. The ~750MB Playwright image was still being pulled and extracted concurrently with the healthcheck window, competing for the runner's disk/CPU. Pulling it up front, before app/remotestorage start, removes that contention instead of just widening the timeout further. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gnL38EmFRmq4MDicaJQ9B
Pre-pulling the tests image didn't change anything: both prior CI failures showed app and remotestorage still exhausting all 40 retries in almost exactly the same ~128s, which rules out disk/CPU contention as the cause - a real slowdown wouldn't reproduce that precisely twice. That points to the healthcheck itself failing outright rather than timing out. Likely culprit: `wget http://localhost:8000/` can resolve to ::1 first on a dual-stack host. GitHub Actions runners are dual-stack; this sandbox apparently isn't, which is why it never reproduced locally. The services bind 0.0.0.0 (IPv4 only), so an IPv6-first resolution would fail every probe. Also have the failure-path log dump include `compose ps` and use `--timestamps`, so if this isn't it, the next failure is diagnosable in one round instead of another guess. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gnL38EmFRmq4MDicaJQ9B
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pilots a reusable acceptance-testing setup: a single containerised command that runs the same way on a laptop and in CI, and drives the real remoteStorage connect-and-sync flow rather than mocking it.
Tier 1 (existing, untouched):
npm run test:smoke— fast, no backend.Tier 2 (new):
make accept/npm run test:accept— both just runscripts/accept.sh, which brings up three containers viadocker-compose.acceptance.yml:app— litewrite built and served (docker/app.Dockerfile)remotestorage— a real armadietto server (remotestorage/armadietto-monolithic, password-auth "classic" variant, matching litewrite's OAuth flow)tests— the officialmcr.microsoft.com/playwrightimage (browsers pre-installed, so nothing tries to download them)All three image tags are pinned by digest, not just a mutable tag.
tests/acceptance/covers, each against real server state rather than sleeps:.github/workflows/acceptance.ymlruns the samemake accepton PRs and uploads the Playwright report on failure.deploy.ymlis untouched.CLAUDE.mddocuments both tiers, where things live, the deploy target, and repo conventions.Verification
Ran
make acceptend to end in the dev session and watched all 4 scenarios pass twice in a row from a clean build. Then deliberately pointed the suite at a wrong remoteStorage URL and confirmed it fails loudly (makeexits non-zero with a clearECONNREFUSED, not a silent pass) before reverting. Tier-1 smoke tests and lint (standard) still pass unmodified.One infra note: the
testsandappbuild steps optionally trust$NODE_EXTRA_CA_CERTSif it's set on the host (via a no-op-by-default volume mount / build secret) — harmless for a normal laptop or GitHub Actions, but needed to validate this behind this session's own TLS-inspecting sandbox proxy.Test plan
make accept— all 4 acceptance scenarios passmake accepta second time from a clean build — still greenREMOTESTORAGE_URLat a wrong port and confirmedmake acceptfails loudly, then revertednpm run test:smokeandnpm run lintstill passGenerated by Claude Code