Skip to content

chore(wiki): correct stale SolidPing facts in the competitor comparison - #319

Merged
fclairamb merged 3 commits into
mainfrom
chore/competitor-facts-browser-engine
Sep 4, 2026
Merged

fclairamb merged 3 commits into
mainfrom
chore/competitor-facts-browser-engine

Conversation

@fclairamb

Copy link
Copy Markdown
Owner

Found while rewriting the public /compare pages (solidping-website#28) against the code rather than against this wiki. Four claims here no longer match the implementation, and the website had inherited one of them verbatim.

Claim Reality Evidence
Browser checks use Rod chromedp over CDP server/internal/checkers/checkbrowser, server/go.mod
Screenshot capture is a gap Browser checks capture on failure (opt-in per check); only HTTP-failure capture is missing spec 2026-08-21-01, checkbrowser/config.go
Teams / Telegram / PagerDuty "spec ready", Matrix ❌, 10 native channels All four ship; 16 senders in the registry server/internal/notifications/registry.go, db/models/user_contact.go
SMS / voice: flat ❌ Implemented and working self-hosted with the operator's own Twilio credentials; not included on any hosted plan registry + entitlements/defaults.go (MaxSmsPerMonth 0) + the website's /pricing copy

That last one is the interesting one: flattening it to an X loses the distinction that decides how a comparison should score it. Against Better Stack, UptimeRobot, Pingdom, StatusCake or Site24x7 on hosted terms it is a real loss. Against Uptime Kuma or Gatus on self-hosted terms it is not a loss at all. Both are now stated.

Also adds a "Which SolidPing is being compared" section at the top of comparison/solidping-position.md, with the plan ladder and the self-hosted entitlements side by side and pointers to the two files they come from. The existing "Advantages Over All Three" table wins every single row because it puts our free uncapped build against their paid plans — that is now labelled rather than left to look like a finding.

Docs only.

🤖 Generated with Claude Code

Writing the public /compare pages against the code surfaced four claims in
this wiki that no longer match the implementation, plus one missing rule.

- Browser checks are **chromedp over CDP**, not Rod. Rod was the engine the
  2026-01-05 screenshot idea proposed; the shipped code
  (server/internal/checkers/checkbrowser) uses chromedp. Corrected in
  maintenant.md, betterstack/comparison.md, comparison/monitor-types.md and
  comparison/solidping-position.md — the website had inherited "Rod / headless
  Chromium" from here.
- Screenshot capture: browser checks do capture on failure since spec
  2026-08-21-01. Only HTTP-failure capture is still missing.
- The alerting table listed Microsoft Teams, Telegram and PagerDuty as gaps
  with "spec ready". All three ship. Matrix too. Total channels was "10
  native"; the registry has 16 senders.
- SMS/voice was scored a flat gap. It is implemented and works self-hosted on
  the operator's own Twilio account; what is true is that it is **not included
  on any hosted plan**, because carriers bill per message. That distinction
  decides whether a comparison should score it as a loss, so it is now spelled
  out rather than flattened to an X.

Adds a "Which SolidPing is being compared" section: self-hosted against
self-hosted tools, SolidPing Cloud with its real caps against hosted services,
both for tools that sell both. The existing "Advantages Over All Three" table
wins every row precisely because it compares our free build to their paid
plans, and now says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fclairamb added a commit that referenced this pull request Sep 4, 2026
## Why

`wiki/` holds internal engineering notes and is an input to **no
build**. The published docs site is `web/docs/`, which is separate;
every other reference to `wiki/` in the tree is a comment pointing at
it. So a wiki-only pull request currently spends a ~28-minute E2E run, a
Go build, four frontend builds and a full backend test suite proving
that prose did not break the compiler.

That is not hypothetical — #319 (five files, all under `wiki/`) did
exactly that.

## What changed

A `changes` job asks the API which files the PR touches and sets one
`code` output. The seven build jobs are gated on it. **Nothing else
moves** — the jobs, their steps and their ordering are untouched.

This is safe *specifically* because of how the existing `ci` gate is
written: `always()`, failing only on `failure` or `cancelled`, so a
skipped job already satisfies it. That is the same property the
release-please skip has relied on all along, and `main` has no required
status checks, so there is no check that can hang unreported. If that
gate ever starts treating `skipped` as a failure, this must change with
it — there is a comment saying so.

The gate now also depends on `changes` itself. Without that, a detector
that *failed* would skip every build job and leave the gate looking at
an all-skipped graph — turning a broken detector into a way to merge
untested code.

## Fail-safes

The failure mode here is silently not testing things, so the decision
fails toward building:

- any non-`pull_request` event (pushes to `main`, in particular) always
builds;
- an empty file list from the API means "the API told us nothing", not
"nothing changed" — it always builds;
- the wiki-only decision is a POSIX `case` glob, not `grep -qv
'^wiki/'`.

That last point is not theoretical. `ugrep`, which is `grep` on some
developer machines, returns false for inverted-quiet matching even when
non-wiki files are present: it classified a **138-file code change as
"wiki-only"**, while `grep -cv` on the same input correctly counted 126
non-wiki files. GNU grep on the runner would have behaved correctly, but
a check whose meaning depends on which `grep` is installed is not one
worth shipping.

## Verification

Both live PRs, evaluated with the exact decision logic from the
workflow:

| PR | Files | Result |
|---|---|---|
| #319 | 5, all under `wiki/` | `code=false` → skips |
| #318 | 138 | `code=true` → builds |

YAML parses, and the resulting job graph was checked: all seven build
jobs gain `changes` in `needs`, and the `ci` gate depends on it too.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@fclairamb
fclairamb merged commit cc44e0b into main Sep 4, 2026
13 checks passed
@fclairamb
fclairamb deleted the chore/competitor-facts-browser-engine branch September 4, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant