Skip to content

fix(samples): seed only sample checks that work out of the box - #537

Merged
fclairamb merged 4 commits into
mainfrom
feat/samples-reachable-default
Oct 12, 2026
Merged

fclairamb merged 4 commits into
mainfrom
feat/samples-reachable-default

Conversation

@fclairamb

Copy link
Copy Markdown
Owner

Problem

A fresh install in the default run mode seeded every Default sample of every checker, about 75 checks. Of the first 79 runs, 36 were down. Most samples target services nobody runs locally (PostgreSQL, MySQL, MSSQL, Oracle, Redis, MongoDB, ClickHouse, Kafka, MQTT, RabbitMQ, gRPC, Docker), public game servers, FTP and SFTP test servers, IMAP, SNMP, a browser check, IPv6 targets or a fake domain. The first dashboard a new user sees was half red.

Fix

  • CheckSpec gets a SeedOnInstall flag. checkerdef.SamplesToSeed keeps only the flagged samples in the Default mode.
  • The startup job filters through it. The edit in job_startup.go is 3 lines in the loop and leaves every log line alone.
  • Test and Demo modes seed everything, as before.
  • GetSampleConfigs is unchanged for its other users: every Default sample still shows in the create-check sample picker, the schema generator and the parity tests.
  • Two new HTTP samples, both in the picker too: the GitHub home page, and a JSONPath assertion on the Google Public DNS JSON API ($.Status eq 0).

A fresh install now seeds 11 checks:

Type Slug Target
http http-cloudflare-dns https://one.one.one.one
http http-github https://github.com
http http-google-dns-json https://dns.google/resolve?name=example.com&type=A, $.Status eq 0
dns dns-google google.com A
dns dns-cloudflare cloudflare.com A
dns dns-github-ns-changes github.com NS, change detection
ssl ssl-google-com google.com
ssl ssl-githubproxy.fjygbaifeng.eu.org github.com
tcp tcp-google google.com:443
tcp tcp-solidping-https solidping.io:443, TLS, sends a GET and expects a status line
icmp icmp-cloudflare 1.1.1.1

Left out on purpose: the localhost fake API, api.acme.com, httpbin.org, the Claude status check (down whenever there is an incident), the custom resolver 8.8.8.8:53 (blocked on some networks), both IPv6 samples, and every other check type.

ICMP: the checker opens an unprivileged ICMP datagram socket first and falls back to a raw socket. Docker 20.10+ sets net.ipv4.ping_group_range to all groups on its default network, so the nonroot image pings without NET_RAW. Only one ping sample is seeded, so a host that blocks unprivileged ICMP shows one red check, not four.

Existing installs are unaffected. The samples.loaded org parameter already stops the seed from running a second time.

Docs: intro.md says what the default org starts with. server/internal/checkers/AGENTS.md documents the flag.

Test plan

  • checkerdef: SamplesToSeed keeps only seeds in Default and with nil options, everything in Test and Demo.
  • registry/seed_samples_test.go: seeds are HTTP, DNS, SSL, TCP or ICMP only, at most 3 per type, 10 to 15 in total, unique slugs. Each one passes its checker's Validate. No localhost, 127.0.0.1, 0.0.0.0, ::1, private address or IPv6 literal anywhere in the config, and no ipVersion: ipv6 pin. The guard itself is tested against bad inputs.
  • test/integration: the Default startup seeds exactly the 11 slugs above. The Test mode test is unchanged and passes.
  • go generate ./internal/checkers/schemas/...: no schema change.
  • golangci-lint run ./...: 0 issues. make lint-agent-docs passes.
  • go test -short ./...: all green.
  • Fresh run: PORT=4974 SP_DB_TYPE=sqlite SP_DB_DIR=<empty dir> solidping serve for 95 s, results read from the SQLite DB. All 11 seeded checks were up, with 0 down, 0 timeout and 0 error results. The samples API still lists 74 samples for the picker.

🤖 Generated with Claude Code

A fresh install in the default run mode seeded every Default sample,
about 75 checks. Most of them target services nobody runs locally
(databases, brokers, gRPC), public game servers, FTP and SFTP test
servers, IPv6 or a fake domain, so the first dashboard was half red.

CheckSpec gets a SeedOnInstall flag and checkerdef.SamplesToSeed keeps
only the flagged samples in the Default mode. The startup job filters
through it. Test and Demo modes seed everything as before, and every
Default sample stays in the create-check sample picker.

A fresh install now seeds 11 checks: HTTP (one.one.one.one, the
GitHub home page, a JSONPath assertion on the Google Public DNS JSON
API), DNS (google.com A, cloudflare.com A, the GitHub NS change
detection), SSL (google.com, github.com), TCP (google.com:443, the
solidping.io request/reply sample) and ICMP (1.1.1.1).

Existing installs are unchanged: the samples.loaded org parameter
already stops the seed from running twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@bbe682a). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #537   +/-   ##
=======================================
  Coverage        ?   82.83%           
=======================================
  Files           ?     1181           
  Lines           ?   121706           
  Branches        ?        0           
=======================================
  Hits            ?   100817           
  Misses          ?    20882           
  Partials        ?        7           
Files with missing lines Coverage Δ
server/internal/checkers/checkdns/samples.go 100.00% <100.00%> (ø)
server/internal/checkers/checkerdef/interface.go 100.00% <100.00%> (ø)
server/internal/checkers/checkhttp/samples.go 100.00% <100.00%> (ø)
server/internal/checkers/checkicmp/samples.go 100.00% <100.00%> (ø)
server/internal/checkers/checkssl/samples.go 100.00% <100.00%> (ø)
server/internal/checkers/checktcp/samples.go 100.00% <100.00%> (ø)
server/internal/jobs/jobtypes/job_startup.go 82.29% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fclairamb fclairamb changed the title feat(samples): seed only sample checks that work out of the box fix(samples): seed only sample checks that work out of the box Oct 12, 2026
@fclairamb
fclairamb enabled auto-merge (squash) October 12, 2026 06:14
@fclairamb
fclairamb merged commit 262c2cc into main Oct 12, 2026
26 checks passed
@fclairamb
fclairamb deleted the feat/samples-reachable-default branch October 12, 2026 06:56
fclairamb added a commit that referenced this pull request Oct 12, 2026
#539 implemented the same VAPID spec (2026-10-12-03) on main. Its version
is kept for every file this branch only touched for that spec (webpush,
db app_settings, server.go, scenario harness, dash0 subscription hook,
notifications mock); the branch's duplicate implementation and its
useWebPushSubscription.test.ts are dropped.

#537 (SeedOnInstall) and the idle-memory spec (NeedsBrowser) both add a
CheckSpec flag: both are kept. A fresh default install now seeds no browser
sample, so TestStartupBrowserSamplesDisabled runs in test mode, with
http-fake-stable as the enabled control, and the small-hosts docs say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fclairamb added a commit that referenced this pull request Oct 12, 2026
#539 implemented the same VAPID spec (2026-10-12-03) on main. Its version
is kept for every file this branch only touched for that spec (webpush,
db app_settings, server.go, scenario harness, dash0 subscription hook,
notifications mock); the branch's duplicate implementation and its
useWebPushSubscription.test.ts are dropped.

#537 (SeedOnInstall) and the idle-memory spec (NeedsBrowser) both add a
CheckSpec flag: both are kept. A fresh default install now seeds no browser
sample, so TestStartupBrowserSamplesDisabled runs in test mode, with
http-fake-stable as the enabled control, and the small-hosts docs say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant