fix: anchor heartbeat evaluation on real beats and close stale status-page incidents

[fclairamb]

Five specs from specs/todos/, implemented sequentially on a batch branch, each gated by an independent completeness audit before being archived. Two of the five needed a fix round; the audit caught a regression one of them introduced.

Targeted at the v0.22.2 patch release — the new migration 018_v0_22_2 is named accordingly.

What changed for the user

Heartbeat and email checks now detect a missing beat reliably. The scheduler's periodic evaluation was reading back the newest result row of any origin — which, from the second period after a beat, was its own previous evaluation rather than the beat. A heartbeat that stopped beating therefore stayed Up for a random number of extra periods (a ~50% coin flip per tick, with an unbounded tail), the row that finally opened the incident reported a nonsense overdueBy of a few milliseconds and a lastSignalAt pointing at another evaluation, and the "run started but never completed" branch was unreachable in production because every evaluation re-anchored the run on itself. The evaluation now reads the newest inbound signal specifically, so overdue detection fires on schedule, the diagnostics carried into incidents and notifications name the real beat, and stale runs time out after two periods as designed.

A result row written by the scheduler no longer looks like a beat. Both kinds of row read "Heartbeat received" with status Up, so opening the evaluation row written seconds after a ping showed no Caller card and a bare JSON dump — and people reasonably concluded their caller had not been recorded. Evaluation rows now say so: the on-time message is "Heartbeat on time" (the beat keeps "Heartbeat received"), the recent-results table tags them with a muted Evaluation badge, and the detail page renders a Scheduler evaluation card explaining what wrote the row, when the last real signal arrived, how long before this evaluation it was, and a link straight to that beat. A row whose newest signal was a failed beat now says "Last heartbeat reported failure" instead of the flatly untrue "No heartbeat received". Beat rows are stored byte-for-byte as before, with tests pinning that.

A status-page incident can no longer outlive the outage it describes. Publishing an already-resolved incident minted a fresh publication in the investigating state, as though the outage were live — and because publishing also stamped the entry as human-touched, the if_untouched auto-resolve default behaved exactly like never. One such publication sat open on a public page for ten days, flooring the TV wallboard at "Some Systems Degraded" while every check was up and the ordinary status page said "All Systems Operational". Publishing a resolved incident now produces a resolved publication with the incident's own resolution time and a retroactive post-mortem timeline; linking an incident to a page no longer counts as taking over the narrative; and auto-resolve now covers any publication linked to the resolving incident, hand-published or not (free-form publications with no linked incident stay untouched, as promised). Where the state can still arise, it is visible: dash0 warns on the checks list and the status-page detail page when an incident is open on a public page while every linked check is up, and the TV board now says when its amber comes from an incident rather than a check.

The embedded-devices panel on a heartbeat check is collapsed by default. The TCP/UDP block was fully expanded on every heartbeat check, doubling the height of the endpoint card for the majority of users who only ever hit the HTTPS URL. It now collapses behind a summary line that still names what is enabled (TCP 4001 · UDP 4001 · signed only), with the nc one-liners as their own disclosures. The token-rotation warning deliberately stays outside the collapsed section — it is a security notice and hiding it behind a chevron would defeat it. The 40-line Arduino/ESP sketch it carried inline is gone in favour of a link to the docs page that already published it next to the counter recipe and security discussion, so the two copies can no longer drift.

The "create a weekly uptime report for me" wand produces a report with checks in it. It previously created a schedule with an empty scope, which the backend correctly treats as org-wide but which shows up as two blank pickers and nothing the operator can point at. It now attaches the 10 most recently created checks; an org with no checks yet still gets the org-wide scope.

Notes for review

• Migration: one new partial index, results_raw_signal_idx, appended to this cycle's consolidated 018_v0_22_2 in both dialects. Without it a silent heartbeat's signal lookup would walk every evaluation row inside raw retention on every tick. GetLastResultForChecks is deliberately untouched — it still serves the API's lastResult, where "newest row of any origin" is the right answer.
• API: no schema changes. ?stale=true is added to the org-side status-page incident list, with a stale boolean on each item meaning "the linked incident is resolved"; the output field's description now points at the new docs paragraph.
• The audit caught a regression this batch introduced. Widening auto-resolve to hand-published entries made a pre-existing asymmetry in OnIncidentReopened reachable: such an entry could be auto-resolved on recovery and then silently fail to reopen on relapse, leaving the publication resolved through a live outage and the wallboard green — strictly worse than the bug being fixed. Both directions now gate on the same condition, with a test asserting the public feed is non-empty after a relapse.
• E2E is authored but was not run locally. Playwright's global setup tears down and recreates the shared dev Postgres, which would disrupt the running devloop, so the new specs (check-heartbeat-push, magic-wand, check-heartbeat-evaluation-rows, stale-publication-banner, tv-board) were verified with --list and eslint only. CI is the real gate for them — worth confirming the E2E job actually ran rather than being skipped behind a red upstream job.
• Behaviour change worth a second opinion: incident reopen consults neither the page's autoResolve policy nor humanTouchedAt. That was already true for auto-created entries, and making the two directions symmetric was the point, but it now means a hand-published entry an operator resolved by hand can be reopened by a machine-written relapse update even on a page set to autoResolve: never. The direction of error is safe (the page shows amber during a live outage) and it is out of this spec's scope, but it is not something the spec spelled out.
• Not done, deliberately: the reported acme page still has its stale publication open. That is a one-click ops action on the live instance, not code, and this run was unattended.

Follow-ups spotted, not addressed

• dash0's "This incident is on autopilot" alert is still gated on autoCreated && !humanTouched, so an untouched hand-published entry now self-resolves without that notice.
• A bare ?stale=true with no active=true filters within the newest 200 publications, so an older open+stale row could be invisible on a page with a long history. The only caller always sends active=true.
• Two specs (2026-09-03-01-getting-started-card-modern-redesign, 2026-09-03-01-operator-notifications-support-messages-and-signups) arrived in specs/todos/ from another session while this batch was running and were deliberately left for a later batch.

🤖 Generated with Claude Code