Skip to content

feat: durable request ledger with crash recovery and lock safety - #1

Open
dbdefault wants to merge 4 commits into
mainfrom
feat/ledger
Open

dbdefault wants to merge 4 commits into
mainfrom
feat/ledger

Conversation

@dbdefault

Copy link
Copy Markdown
Owner

Summary

Adds a durable, file-backed request ledger (src/ledger.mjs) and wires it into the idempotent request coordinator so that request outcomes survive restarts and crashes.

What it does

  • Append-only JSONL ledger with per-line sequence numbers and checksums; writes are durable (fsync) before a state is reported.
  • Request states: ACCEPTED, RUNNING, SUCCEEDED, FAILED, UNKNOWN. A missing receipt is never treated as success; ambiguous outcomes are recorded as UNKNOWN.
  • Idempotency by stable request ID; a repeated ID with a different payload is rejected as a conflict.
  • Crash recovery: on startup the ledger is replayed; a request left RUNNING is reconciled to UNKNOWN rather than re-executed.
  • Cross-process safety through a directory lock with an owner token and an mtime heartbeat, and atomic rename for stale-lock takeover. No network sockets and no child processes are used.
  • request-coordinator gains an optional durable mode; the 0.1.0 public API is unchanged.

Documentation

docs/LEDGER.md describes the file format, state machine, recovery rules, and known limits (including that a heartbeat stalled for longer than the stale threshold can let a live owner be taken over).

Tests

  • npm test → 30 pass / 0 fail in a fresh clone (Node 20+, zero dependencies).
  • Includes fault-injection style tests: crash during append, partial write, duplicate dispatch, concurrent processes, and lock takeover.

Review

  • Independent cross-review by a separate reviewer: PASS (23/0 at the time), and a follow-up delta review after merging main and tightening the stale-lock takeover: PASS (30/30, ledger tests stable over repeated runs).
  • Clean-room check: no private hostnames, paths, tokens, or code from other projects.

Merge target: main. No breaking changes.

Add Ledger (JSONL, fsync, crc, lock, compact) and optional durable
RequestCoordinator binding while keeping the in-memory v0.1.0 API.
Replace TCP/child_process locking with ledger.lock/ mkdir + mtime
heartbeat. Durable reconcile awaits ledger write before memory update.
Fail succeed persistence with LedgerWriteError instead of returning success.
Reclaim live owners only after 4x staleMs; ESRCH only after a missed
heartbeat. Verify owner token after rename so a racer cannot delete a
fresh lock. Document compromised-owner limits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant