Skip to content

ci(stepup-behat): generate HAProxy dev certificate before compose up - #98

Open
kayjoosten wants to merge 5 commits into
mainfrom
fix/stepup-behat-haproxy-cert
Open

ci(stepup-behat): generate HAProxy dev certificate before compose up#98
kayjoosten wants to merge 5 commits into
mainfrom
fix/stepup-behat-haproxy-cert

Conversation

@kayjoosten

@kayjoosten kayjoosten commented Sep 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Problem

stepup-behat CI (and Stepup-Gateway's test-integration, which brings the
stack up the same way) fails when the HAProxy development certificate has not
been generated before docker compose up.

The SelfService logout flow used by the Behat scenarios also needs to stay
inside the local Stepup stack. The upstream SelfService image redirects logout
to the public SURF pages by default, while the test flow continues through the
local test SP.

Cause

Commit 2291d6f (2026-08-12) stopped committing core/haproxy/haproxy.pem and
moved its generation into core/scripts/create_dev_ca.sh, which is only
invoked by start-dev-env.sh. CI runs docker compose up directly, so the
file is missing unless CI generates it first.

The SelfService image ships with a default parameters.yaml aimed at the
public SURF environment (public gateway/middleware URLs, public logout
redirect, etc.), which doesn't work for this local dev/behat stack.

Fix

Run create_dev_ca.sh in the Init step before compose up and copy the dev CA
certificate into the Stepup stack so the app containers trust HAProxy TLS.

For SelfService, bind-mount a complete local parameters.yaml over the
image's default config, matching the pattern already used by the other
services in this stack. It points the app at the local gateway/middleware/SAML
setup and keeps the logout redirect on the local test SP domain instead of the
public SURF one.

The Behat logout assertion is updated to expect the local test SP domain.

Supersedes #97 (which chased a wrong --remote-allow-origins lead).

@kayjoosten
ci(stepup-behat): generate HAProxy dev certificate before compose up
258072c 
Commit 2291d6f stopped committing core/haproxy/haproxy.pem and moved its
generation into core/scripts/create_dev_ca.sh, invoked only by
start-dev-env.sh. The stepup-behat workflow runs 'docker compose up'
directly, so haproxy.pem was missing; Docker created it as a directory and
the haproxy container failed to start, taking down TLS routing and every
Behat scenario with it.

Run create_dev_ca.sh in the Init step, matching start-dev-env.sh's guard.
@kayjoosten

kayjoosten commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

CI on this branch now gets 155 Behat scenarios running (53 passing) where main gets 0 — the haproxy container starts and the stack comes up. The remaining 102 failures (selectOption() on null, gateway_verify_yubikey_yubikeyInput not found) are a separate pre-existing breakage in the full SelfService/RA behat suite on main (red since ~2026-08-12) and are out of scope for this cert fix. Verified separately: with this change the Stepup-Gateway @functional suite is fully green (25/25).

@kayjoosten
ci(stepup-behat): distribute dev CA cert to app containers
74b4649 
Since the CA became dynamically generated (2291d6f) the app containers no
longer trust the HAProxy TLS certificate: the static stepup/haproxy/haproxy.crt
that used to be committed is gone, and only start-dev-env.sh copies the freshly
generated one into place. CI skipped that copy, so every inter-service HTTPS
call failed with 'cURL error 60: self signed certificate in certificate chain',
cascading into ~100 Behat failures.

Copy core/haproxy/haproxy.crt to stepup/haproxy/haproxy.crt before compose up;
the base image imports /config/haproxy/haproxy.crt and runs update-ca-certificates
on startup.
@kayjoosten kayjoosten mentioned this pull request Sep 8, 2026
Closed
@kayjoosten

kayjoosten commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up on the remaining stepup-behat failures:

What I fixed on this branch:

  • added a mounted SelfService parameters.yaml override so logout_redirect_url stays inside the local test stack instead of using the baked-in production www.surf.nl URLs
  • updated the Behat logout expectation accordingly
  • commit: 2d3ddb0 (fix(stepup-selfservice): keep logout redirects local)

Result:

  • pushed to fix/stepup-behat-haproxy-cert
  • new run: 34231412724
  • scenario count improved from 64 passed / 91 failed to 66 passed / 89 failed
  • the cURL error 7: Failed to connect to www.surf.nl port 443 failures are gone

What remains unexplained:

  • 88 failures are still Form field ... "gateway_verify_yubikey_yubikeyInput" not found
  • 1 failure remains on selfservice.feature:30 (The following tokens are registered for your account. not found)

Root-cause hypothesis for the YubiKey failures:

  • this does not look caused by the HAProxy/cert workflow changes in this PR
  • devconf already uses the current YubiKey field id, and current Stepup-Gateway source still renders that form field
  • an earlier branch run on 2026-08-06 only had the two www.surf.nl failures, so the broad YubiKey breakage appears to be a later upstream stack/image regression rather than something introduced here

I opened follow-up issue #99 to track that separate investigation.

@kayjoosten kayjoosten mentioned this pull request Sep 8, 2026
Open
@kayjoosten

kayjoosten commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Root cause of the remaining 88 gateway_verify_yubikey_yubikeyInput failures found: the ghcr.io/openconext/stepup-gateway/stepup-gateway:test image was frozen at commit fa1c0074 (2026-01-13) — 19 commits behind main. Stepup-Gateway's build-push-test-docker-image workflow only triggered on pushes to the long-merged feature/create_docker_publish_action branch, never on main, so the :test tag stopped updating after that one-off run.

Fixed upstream in OpenConext/Stepup-Gateway#666 (trigger on push to main), and manually rebuilt the :test image now via workflow_dispatch to unblock this PR immediately.

Re-ran this PR's Behat suite against the refreshed image: 155/155 scenarios passing. ✅

@kayjoosten
fix(stepup-selfservice): keep logout redirects local
4faa694 
Mount a selfservice parameters override so logout redirects stay inside the dev/CI environment instead of following the baked-in www.surf.nl URLs. Update the Behat expectation to assert the local test SP target.
@kayjoosten
kayjoosten force-pushed the fix/stepup-behat-haproxy-cert branch from 2d3ddb0 to 4faa694 Compare September 8, 2026 21:31
@kayjoosten
kayjoosten requested a review from johanib September 8, 2026 21:51
@kayjoosten
fix(stepup-selfservice): merge logout redirect override
4713eb4
@kayjoosten
Revert incorrect selfservice config merge change
856a4ee 
The previous commit assumed the docker-compose bind mount of the full
stepup/selfservice/parameters.yaml onto the container's config was a
bug that clobbered the image's baked-in configuration, and replaced it
with a partial merge against that baked-in config via an entrypoint
script.

That assumption was wrong. The full local parameters.yaml is the
intentional, correct pattern already used for this dev/behat
environment: it supplies the gateway, middleware and SAML values this
environment actually needs (dev URLs, dev SAML certificates and
credentials), which are not present in the image's baked-in config at
all. Replacing it with a partial merge dropped all of those values and
broke the SAML login redirect, which is why the Behat suite started
failing (22 scenarios) right after that change.

This reverts back to mounting the full local parameters.yaml, which is
the state the Behat suite was passing against before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@johanib johanib Awaiting requested review from johanib

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kayjoosten