Skip to content

Decide if the daemon runs from a lock file, not from the PID - #456

Open
alex-clickhouse wants to merge 1 commit into
mainfrom
alex/daemon-lock-file
Open

alex-clickhouse wants to merge 1 commit into
mainfrom
alex/daemon-lock-file

Conversation

@alex-clickhouse

Copy link
Copy Markdown
Collaborator

On a systemd install, nerve start failed with "Nerve is already running" after a reboot. systemd restarted it every 10 seconds for two days.

Cause:

  • uvicorn sends SIGTERM to its own process again after a graceful shutdown. The process stops before the finally in start removes nerve.pid, so the file stays after each stop.
  • After the reboot, the system gave the old daemon PID to a root process. os.kill(pid, 0) raised PermissionError, and the check counts that as "running".

A reused PID can also belong to a process of the same user. Then nerve stop sends SIGTERM to that process.

Changes

  • Lock (nerve/pidfile.py): the daemon holds an exclusive flock on ~/.nerve/nerve.lock while it runs. The kernel releases the lock when the process stops for any reason, including SIGKILL and a reboot. The daemon is running only if the lock is held. The PID in nerve.pid is only for display and signals.
  • Start: start --foreground takes the lock before it starts the server. If two starts occur at the same time, only one gets the lock.
  • Restore (nerve/backup.py) uses the same check.
  • Upgrade: a daemon from an older version holds no lock. If nerve.lock does not exist, the check uses the PID as before. Thus nerve upgrade && nerve restart still stops the old daemon.
  • nerve.lock is never deleted. Status checks do not delete a stale nerve.pid, because it has no effect.

Tests

  • A reused PID with a lock that is not held: not running.
  • A daemon killed with SIGKILL releases the lock. The PID file stays and has no effect.
  • acquire fails while the lock is held and does not change the PID file.
  • No lock file: the check uses the PID.
  • Restore refuses while the lock is held.

The full suite passes locally. I also ran nerve status for each of these cases: a reused PID, a held lock, and a killed lock holder. I did not run a full daemon start and stop under systemd.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant