Repository navigation
Startup tunnel gate releases early after a restart (treats a not-yet-stored tunnel config as not linked) #80
Description
Ledger key: r2/startup-gate-early-release. Found while recording item 4's live acceptance (r2/reliability, PR #24).
Problem
After a restart, the startup tunnel gate (McpEvents.awaitStartupTunnel) releases within a few milliseconds, before the T3 Connect tunnel is reachable. Event deliveries then go out while Jarvis still can't call back into T3. That is the gap #24 was meant to close.
Live evidence (Omarchy, deduplicated trace spans; t0 = start of server.startup)
| Restart | t0 | Gate released | First webhook POST | First check: not reachable | First check: reachable |
|---|---|---|---|---|---|
| Reboot, service start on .9 | 01:00:00.543Z | +0.97 s (span 1.3 ms) | +6.1 s (started 01:00:06.688, finished 01:00:07.093) | none logged | 01:00:42.002Z (+41.5 s) |
| Forced update .9 → 20261113.1 | 01:02:23.706Z | +3.39 s (span 8.7 ms) | +3.53 s (started 01:02:27.237, finished 01:02:27.576) | 01:02:58.239Z (no_connector) |
01:03:25.917Z (+62.2 s); the outside probe got HTTP 200 at 01:03:22Z |
On .9 → 20261113.1, deliveries started about 58 s before the tunnel was first confirmed reachable. The .3 → .4 update (t0 10:55:57Z) made no gate call after its restart within the 7 captured minutes, so it gives no measurement.
Likely cause (inferred from the spans, not proven)
The gate's first ConnectReachability.check on each start has only a ServerSecretStore.get child. It has no probeEndpoint and no log event. On a process's first check, every result except not_linked is logged, and reachable needs a probe. So the check most likely returned not_linked, meaning the cloud-endpoint-runtime-config secret wasn't readable yet. The gate treats not_linked as "nothing to wait for", and startupHoldReleased keeps that early answer for the life of the process. Checks about 30 s later read the config and probe the endpoint (no_connector, then reachable). Why the config is unreadable in the first seconds is not shown by the traces. releaseManagedTunnelOnShutdown removing it until the tunnel is provisioned again is one candidate.
Fix direction
Tell "never linked" apart from "linked, tunnel config not stored yet". For example, hold while a cloud link exists (relay URL or linked user) and treat not_linked as final only when there is no link. Otherwise re-check not_linked until the 180 s cap instead of latching it. Add a regression test for a first check that returns not_linked and later checks that return reachable. The live check has to wait for the audit thread's next service update.
Evidence: /home/cohearn/.t3/scratch/2026-11-13-r2-reliability-startup-gate/ (gate-timeline.txt, reachability-checks.txt, and the scripts that made them from .../2026-11-11-i-want-you-to-do-10d87ec0/update-next/traces and update-20261112.4/traces).
🤖 Generated with Claude Code
Activity
chrisohearn commentedon Nov 13, 2026
Fix in #81: a not_linked check holds deliveries only for a CLI managed link, the one link whose tunnel config is deleted at shutdown and stored again after startup.
Remaining limits (review P3, non-blocking). A CLI managed link still waits until the 180 s cap when:
endpointRuntime: null, orreconcile_link(no public cloud config).Possible follow-up: log when the startup hold times out while the check still reads
not_linked, so the case shows in traces.Live check after the next service update (owned by
r2/startup-gate-early-release): the firstMcpEvents.awaitStartupTunnelspan should last until the first reachableConnectReachability.check, and the firstMcpEventsWebhook.postshould come after it. Run/home/cohearn/.t3/scratch/2026-11-13-r2-reliability-startup-gate/gate.pyon that update's traces.🤖 Generated with Claude Code
chrisohearn commentedon Nov 13, 2026
Status: fix merged, awaiting release and live check. PR #81 (merged 2026-11-13 02:03Z,
1a8a07f393) holds deliveries only for a CLI-managed link and keeps the hold while a linked server re-provisions its tunnel. It ships with the next fork release. Closing waits for the live restart acceptance after that deploy. Ledger:r2/startup-gate-early-release(blocked, WAIT_NATURAL).Triage by the round-four GitHub hygiene pass (ledger key
r4/w7-github), 2026-11-13. Left open.