Terminal: idle terminals stop scanning the host process table every second #232

Summary

Every running Terminal session polls its child processes once a second for as long as it lives, even when nothing is happening in it. Each poll refreshes the whole host process table, not just the shell's children. With N terminals open, the server does N full process-table scans per second. An idle machine with a few forgotten terminals keeps paying that cost.

T3 Code made the same poll back off when a terminal is quiet in pingdotgg/t3code#21242. This issue proposes a BiBCode version that also folds the per-terminal scans into one shared scan per tick.

Current behavior (verified at a0ea849ef92b)

The poll (apps/server/src/terminal/manager.rs)

• DEFAULT_SUBPROCESS_POLL_INTERVAL is 1 s (:42), set through TerminalManagerOptions::subprocess_poll_interval (:126, default :191). Zero disables the loop; the worktree tests use that (apps/server/src/production/worktree_runtime.rs:3417, :3513).
• Each terminal start spawns an activity loop (:2745-2810). It sleeps the interval, calls inspector.inspect(pid), and publishes TerminalEvent::Activity only when has_running_subprocess or the child label changed. There is no pause for a quiet terminal, a childless shell, or a terminal no client is watching.

The sampler is shared, but the scans aren't

• NativeTerminalSubprocessInspector (:83-115) holds one NativeProcessSampler. The manager creates one inspector (:1776-1779), and production builds one TerminalManager (apps/server/src/production/runtime.rs:432-448). So all terminals share one sampler and one sysinfo::System.
• Each inspect still runs a full refresh: refresh_processes_specifics(ProcessesToUpdate::All, …) with CPU, memory, command and exe (apps/server/src/diagnostics/native.rs:334-349, reached through :66-71 / :101-112). The calls are serialized on the System mutex. N terminals mean N full scans per second, each followed by a walk of the result for one PID's children.
• This is separate from the diagnostics/profiler sampler (runtime.rs:269, the area of Every server boot runs a full host-wide process scan via sysinfo::System::new_all() #24).

Signals already available

• Output: the per-terminal output loop (:2615-2640).
• Input: TerminalManager::write (:3335).
• Attach: size_claim_attachments (:1730) tracks attached clients per session for size claims. Whether it reliably means "a client is watching" is unverified.

Who reads the result: has_running_subprocess and the child label in apps/server/src/terminal/model.rs:204, :321; the operational log summary (apps/server/src/terminal/operational_logs.rs:243-252); the web sidebar running-terminal indicator (apps/web/src/components/sidebar/workspaceCardLookups.ts).

Not measured. The cost of one full sysinfo refresh on BiBCode's supported platforms, and the CPU or wakeups with N idle terminals.

Proposed change

1. One scan per tick for all terminals. Replace the per-terminal inspect with a manager-level tick. It refreshes the process table once and answers every active terminal from that one snapshot (a parent → children map). N terminals cost one scan per tick.
2. Back off when quiet, per terminal:
   • After 2 consecutive unchanged observations with no child process and no input or output, the terminal is quiet and skips ticks.
   • Any input (write), output, resize or attach makes it active again and triggers a sample on the next tick.
   • A quiet terminal is still resampled every 10 ticks (about 10 s). That bounds how late the indicator can be for a child that starts and prints nothing.
   • A terminal with a running child keeps the 1 s poll so the indicator clears promptly when the child exits.
   • When no terminal needs a sample in a tick, skip the scan entirely.
3. Keep subprocess_poll_interval = ZERO meaning "off", and keep the TerminalSubprocessInspector seam testable with a fake.
4. No contract change: TerminalEvent::Activity and the summary fields keep their shape and meaning.

The worst-case delay for a silent child is about 10 s (10 ticks). Mauro approved this bound on 2026-11-01 instead of the proposed 30 s.

T3 Code reference (their claims, not measured by us)

• pingdotgg/t3code#21242, "back off terminal subprocess polling when idle" (merged 2026-10-20). They skip snapshots after two unchanged childless observations while quiet, resume on input or output, and resample every 30 quiet ticks. They report 11 → 2 process snapshots per 12 s of idle on Linux, and they call the up to ~30 s delay for a silent child a tradeoff that needs approval. They kept per-terminal polling (no shared scan).

Acceptance criteria

• Unit tests in apps/server/src/terminal/manager.rs (#[cfg(test)], with a counting fake TerminalSubprocessInspector or snapshot source and tokio::time::pause), hermetic:
  • idle_terminals_share_one_scan_per_tick: 3 active terminals → 1 snapshot per tick.
  • quiet_childless_terminal_stops_sampling_after_two_unchanged_ticks.
  • input_resumes_sampling_on_next_tick and output_resumes_sampling_on_next_tick.
  • quiet_terminal_resamples_every_10_ticks: a child that starts silently is reported within 10 ticks (about 10 s).
  • running_child_keeps_one_second_poll_and_clears_on_exit.
  • no_scan_when_no_terminal_needs_a_sample.
  • zero_interval_disables_polling (existing behavior kept).
  • Existing activity tests unchanged: Activity is published only on change, with the same sequence semantics.
• Integration: apps/server/src/production/server_terminal_input_tests.rs and the worktree runtime tests pass unchanged.
• Desktop e2e: vp run test:ui:desktop, including apps/desktop/e2e/specs/project-session-terminal.e2e.ts (the running-command indicator still turns on and off).
• Gates: cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, vp check (zero warnings), vp run typecheck, vp run check:contracts (no fixture changes expected), Release Smoke.
• Measurement in the PR: snapshot count and server CPU over 60 s with 1 and 5 idle terminals, before and after, on Linux and macOS (Windows if available).
• docs/testing/ runbooks are reviewed and updated or stated "remain accurate". Profiler coverage line in the delivery report: say whether the shared tick shows up as its own profiler span or under the existing terminal spans. Process command lines never reach profiler records or logs.

Living docs to update

• docs/architecture/overview.md:826-829 (add a terminal activity note next to the per-subscriber port-scan note): one shared scan per tick and the quiet backoff, including the ~10 s bound.

Not in scope

• The diagnostics/profiler process sampler (Every server boot runs a full host-wide process scan via sysinfo::System::new_all() #24).
• The port scan.
• Changing what counts as a "running subprocess".

Related

• Every server boot runs a full host-wide process scan via sysinfo::System::new_all() #24 (closed): "Every server boot runs a full host-wide process scan via sysinfo::System::new_all()". The same sysinfo cost, at boot.
• Unfocused desktop polls diagnostics every 2 s and gets 1 Hz unchanged pushes #168 (closed): "Unfocused desktop polls diagnostics every 2 s and gets 1 Hz unchanged pushes". The same kind of idle polling, on the client side.
• macOS x64: the live Codex terminal Activity dock never shows #192 (open): "macOS x64: the live Codex terminal Activity dock never shows". Possibly unrelated (it's about the Activity dock, not this poll); check it before starting so the changes don't conflict.

Open questions

• ~~Is a ~30 s worst-case delay acceptable for a silent child?~~ Decided 2026-11-01: use 10 ticks (about 10 s).
• Should a terminal with no attached client poll less often, or not at all? size_claim_attachments may be the signal, but that hasn't been verified.
• Should the shared scan refresh only the fields the terminal needs (parent and command)? Today the shared NativeProcessSampler also collects CPU and memory for diagnostics, so splitting it may need its own System.