emrg: port-file self-heal + G43 PID guard — kill the zombie-daemon state - #593
Merged
Conversation
…ate (Windows v0.2.15 root cause) Rant 2026-08-09T13:16:36 root-cause follow-up: PR #592 contains the window-storm (CREATE_NO_WINDOW + spawn throttle + backoff), but the underlying deletion race remains: G43 stale-port logic deletes a healthy daemon's emrgd.port after ONE transient ws failure → the daemon's own scheduler loses the file (93x 'cannot connect' in the host's emrgd.log) while the PID lock makes every respawn exit instantly (zombie state — daemon alive, forever unreachable). Two complementary fixes: 1. daemon-side self-heal: new _assert_port_file() + _port_keepalive_loop re-asserts emrgd.port every 60s if it was deleted/overwritten — the daemon is the living core and re-asserts its own identity. 2. GUI-side guard: _daemonProcessAlive() checks emrgd.pid before G43 deletes the port file — process alive → keep the file and let the reconnect backoff retry (transient failure); only truly-dead daemons get the delete+respawn path. Tests: +3 Python (test_daemon.py _assert_port_file write/rewrite + keepalive restore), +3 GUI (daemon_client.test.js G43 guard alive/dead/ no-pid). 641→644 py, 96→99 gui, docs synced (#511).
argszero
commented
Aug 9, 2026
argszero
left a comment
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle 2026-08-09T16:58 (Round 187)
Reviewed the full diff on the PR branch:
_assert_port_fileextraction +_port_keepalive_loop(60s re-assert) is the right self-heal: the daemon is the living core and re-asserts its own identity file against external deletion (the G43 unlink race from the v0.2.15 storm root cause). Shutdown path cancels the task cleanly._daemonProcessAlive()PID probe (signal 0, EPERM=alive on Windows) is correct; G43 catch path now keeps the port file when the daemon is alive (transient failure → backoff retry) and only delete+respaws on a truly-dead daemon.- Positive + negative states covered: alive-daemon (no unlink, no respawn, throws) vs dead-daemon (legacy path preserved) + no-pid-file case.
Verified on the PR branch: uv run pytest tests/ → 644 passed; npm test → 99 passed. Doc counts synced to the branch base (#511).
Contributor
|
Tested this PR on Windows (win32, Python 3.13.4). Verified the full root-cause chain: |
argszero
commented
Aug 9, 2026
argszero
left a comment
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle 2026-08-09T17:18 (Round 190)
Second independent review of the zombie-state root-cause fix (re-verified fresh this cycle):
_assert_port_fileextraction is clean — single writer for the port/token file, reused by serve() and the keepalive._port_keepalive_loop(60s re-assert on missing file) is the right self-heal: the daemon re-asserts its own identity file against external deletion (the G43 unlink race). Cancelled cleanly in shutdown._daemonProcessAlive()PID probe semantics are correct (signal 0, EPERM=alive on Windows); G43 catch path keeps the port file when the daemon is alive (transient → backoff) and only delete+respaws on a truly-dead daemon.- Positive + negative states covered: alive (no unlink/respawn) vs dead (legacy path) vs no-pid-file.
Fresh verification on the PR branch: uv run pytest tests/ → 644 passed; npm test → 99 passed; import OK. Doc counts synced to the branch base (#511).
argszero
commented
Aug 9, 2026
argszero
left a comment
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle 2026-08-09T17:25 (Round 192)
Third independent review (3/3 consecutive LGTMs: C187, C190, C192). Fresh diff audit on the PR branch:
- daemon.py: _assert_port_file extraction (single writer reused by serve() + keepalive); _port_keepalive_loop 60s re-asserts emrgd.port if deleted/overwritten (the G43 unlink race self-heal); cancelled cleanly in shutdown.
- daemon_client.js: _daemonProcessAlive() PID probe (signal 0, EPERM=alive on Windows); G43 catch keeps the port file when daemon alive (transient → backoff), only delete+respaws on truly-dead daemon.
- Tests: +3 Python (assert/rewrite/keepalive restore), +3 GUI (alive no-unlink, dead respawn, no-pid = dead); positive + negative states covered.
Verified the PR branch tests: uv run pytest tests/ → 644 passed; npm test → 99 passed. Production code merges cleanly with the #592 merge (conflicts only in mechanical doc/test-appends). Merge-ready.
Contributor
argszero
pushed a commit
that referenced
this pull request
Aug 9, 2026
argszero
added a commit
that referenced
this pull request
Aug 9, 2026
* emrg: TUI daemon spawn throttle — complete the anti-storm fix (rant 2026-08-09T13:16:36) PRs #592 (CREATE_NO_WINDOW + GUI spawn throttle + scheduler backoff) and #593 (port-file self-heal + G43 PID guard) contain the Windows v0.2.15 storm fix, but the TUI client had the same storm pattern the GUI had: app.py _reconnect() loops every 1s calling ensure_connected() → with a down daemon each iteration calls start_daemon() → spawns a NEW daemon process every second. On Windows each spawn was a cmd-window source (pre-#592); even windowless it is process churn + PID-lock exit races post-#592. This completes rant acceptance item ① ("启动 GUI/TUI 后零 cmd 窗口弹出"): - daemon_manager.start_daemon: spawn throttle — max 3 attempts per connect lifecycle, then raise with a clear 'run emrg server manually' message instead of spawning forever; counter resets on successful connect in ensure_connected (mirrors GUI daemon_client.js MAX_SPAWN_ATTEMPTS). - app.py _reconnect: on throttle, surface a one-time system message + status hint so the host knows to start the daemon manually (recovery path intact: host starts daemon → is_running True → connect succeeds → counter resets). Tests: +2 (start_daemon throttles after 3 attempts, no 4th spawn; counter resets on success). 641→643 py, doc counts synced (#511). * emrg: sync doc test counts to 652 (post-#593 merge + #594 tests) --------- Co-authored-by: EMRG Evolution <emrg@argszero.dev>
argszero
added a commit
that referenced
this pull request
Aug 9, 2026
argszero
added a commit
that referenced
this pull request
Aug 9, 2026
argszero
added a commit
that referenced
this pull request
Aug 9, 2026
…T13:16:36) (#596) Carries the emergency Windows v0.2.15 regression hotfix (host observed cmd-window storm + daemon startup failure, had to reboot): - #592: CREATE_NO_WINDOW across all 34 subprocess sites (emrg/_win.py win32_no_window_kwargs) + GUI spawn throttle + scheduler connect backoff - #593: daemon port-file self-heal (_port_keepalive_loop) + G43 PID guard (never delete a live daemon's port file) - #594: TUI spawn throttle (_MAX_SPAWN_ATTEMPTS=3 + reset on connect) - #595: evolution_prompt quick-ref entries Per host authorization (rant #16/#18: release without waiting for real-machine verification), this bump goes straight to Build Release on merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root-cause follow-up to the Windows v0.2.15 regression (rant 2026-08-09T13:16:36). PR #592 contains the storm containment (CREATE_NO_WINDOW across all subprocess sites, GUI spawn throttle, scheduler connect backoff, reconnect backoff). This PR fixes the underlying deletion race that produced the host's observed symptom (emrgd.log: 92 prompt built / 93 cannot connect / 0 connected).
Root cause (zombie state)
G43 stale-portlogic deletesemrgd.portwhen a ws connect fails — but a transient failure (daemon mid-restart, brief unavailability) would delete the port file of a healthy daemon:emrgd.portcannot connect (1/3): [Errno 2] No such file or directoryemrgd already running) → exits instantly → no new port fileChanges
1. Daemon-side self-heal (
emrg/server/daemon.py)_assert_port_file(port)— single writer for the port/token file (extracted from serve())_port_keepalive_loop()— every 60s, ifemrgd.portis missing, re-asserts it with a warning. The daemon is the living core: it re-asserts its own identity against any external deletion/overwrite.2. GUI-side guard (
emrg/gui/daemon_client.js)_daemonProcessAlive()— readsemrgd.pidand probes withprocess.kill(pid, 0)(EPERM on Windows = alive; ESRCH/ENOENT = dead)daemon unreachable (pid alive)so the reconnect backoff retries the transient failure; only a truly-dead daemon gets the delete+respawn path.Tests
tests/test_daemon.py):_assert_port_filewrites port+token; re-asserts after external deletion; keepalive restore.test/daemon_client.test.js): G43 guard — pid alive → no unlink, no respawn, throws; pid dead → still deletes+respaws; no pid file → treated as dead.Verification
uv run pytest tests/: 644 passednpm test(emrg/gui): 99 passed--helpOK