Repository navigation
ci: the OpenID conformance suite checks auth, locally and on every OIDC change - #519
Draft
PiquelChips wants to merge 1 commit into
Draft
PiquelChips wants to merge 1 commit into
PiquelChips wants to merge 1 commit into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
PiquelChips
force-pushed
the
ci/conformance-suite
branch
from
November 13, 2026 22:30
18e20d0 to
b2be6e2
Compare
PiquelChips
marked this pull request as draft
November 13, 2026 22:31
`just conformance [AUTH_URL]` runs the OpenID Foundation's conformance suite, pinned to release-v5.3.4, against a local auth it starts or a preview. A Playwright driver plays the browser with a virtual passkey. Every result is compared with tests/conformance/expected.json, which records each current deviation with the issue that fixes it. CI runs it on OIDC changes, on main and nightly. AUTH_LOOPBACK_LOGOUT_CA, for tests only, lets back-channel logout tokens reach the suite on localhost. Closes #51
PiquelChips
force-pushed
the
ci/conformance-suite
branch
from
November 13, 2026 22:31
b2be6e2 to
ac5ac9e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nothing outside our own tests checked auth against the OpenID specifications. Now
just conformanceruns the OpenID Foundation's conformance suite against auth, and CI runs it on every change to auth's OAuth and OIDC code, onmainand nightly.just conformancebuilds auth and runs it with its own PostgreSQL and Mailpit behind Caddy's HTTPS;just conformance <url>runs against a preview, through its CLI. The suite runs from its prebuilt image, pinned torelease-v5.3.4.tests/conformance/driver.tsplays it in Chromium with a virtual passkey: it signs an account up, signs in, consents, confirms logout and uploads the screenshotsREVIEWtests ask for. Results, screenshots and logs land intarget/conformance, which CI uploads.expected.jsononly shrinks. Every result must equal its entry, or bePASSEDwithout one; an unexpectedPASSEDfails too. Each of the 44 entries links the issue that fixes it.AUTH_LOOPBACK_LOGOUT_CAlets back-channel logout tokens reach it onlocalhost. A preview can't reach the suite on your machine, so it skips that plan.What the run finds on
mainMost of the Basic and logout plans fail at their first authorization: auth requires PKCE from every client and the suite sends none. That's #514, fixed by #521. With PKCE optional for confidential clients, what remains is:
client_secret_postyet; and the warnings and screenshot reviews certification accepts, such as profile claims auth doesn't hold.claimsparameter).The auth fixes (#521, #522, #523) no longer wait on this PR. Whichever merges second updates
expected.json.Closes #51
Note
Written by Claude Opus 5.5 in Claude Code (via T3 Code)