Multi-agent: 20 concurrent writers on one tracker (no git) lose most writes to a fixed 3 s lock wait; append-only comments and parented creates serialize on exclusive locks #1516

Summary

Many agents work on the same tracker directory at once, with no git involved: a shared folder, a CI box or a host running several agent processes. In that setup most writes fail with Error: Lock conflict (exit 4) once more than about 10 writers are active. Every failure comes after a fixed ~3 s wait (locks.wait_ms: 3000), which does not grow with the number of queued writers. Parented creates all serialize on a single workspace-wide hierarchy-integrity lock. The recovery text then points agents at --force with a placeholder id.

Reproduction (pm 2026.10.19, Linux, plain directory, no git)

pm init --defaults --agent-guidance skip --prefix ng
ID=$(pm create --title "Shared goal" --type Task --json | jq -r .id)
for i in $(seq 1 20); do (
  export PM_AUTHOR=agent-$i
  pm create --title "Agent $i subtask" --type Task --parent $ID
  pm comments $ID --add "progress from agent $i"
  pm claim $ID
  pm update $ID --priority $((i%5))
) & done; wait

Every lock failure reports after waiting 3000–3446ms. With 8 concurrent writers, all 8 comments --add and all 8 notes --add succeed. Raising PM_LOCK_WAIT_MS=60000 lets 8 writers succeed in 34 s.

The host was heavily loaded during the runs (load average 47–81 on 4 cores), so one write took 1.9–5.5 s. On a quiet host a write takes about 0.3–0.5 s, so 20 queued writers still need roughly 6–10 s, more than the fixed 3 s wait. The failure mode is the same; only the threshold moves.

Why it matters for agents

• The failed writes are not optional: the agents' comments and subtasks are lost unless each agent implements its own retry loop. Agents read exit 4 plus "Lock conflict" as a hard failure.
• comments --add and notes --add are append-only and commutative, yet they take the item's exclusive lock and fail when it is busy.
• create --parent takes a workspace-wide hierarchy-integrity lock, so every agent creating children anywhere serializes on one lock, not just agents sharing the same parent.
• The recovery bundle shows pm update pm-a1b2 --status in_progress --force (a placeholder id, a different command) and says "use --force where supported if lock is stale". These locks were live, not stale. Steering contending agents toward --force is the wrong advice.

Proposals

1. Queue instead of failing on live locks: a fair FIFO wait whose deadline scales with the observed queue depth (or a much higher default), and a per-command --wait <ms> flag next to locks.wait_ms / PM_LOCK_WAIT_MS. Fail only when the lock is stale or the wait is explicitly bounded.
2. Append-only operations need no exclusive item lock (comments, notes, learnings, files/docs/tests --add, history events), or at most a very short critical section, so concurrent appends always land.
3. Scope the hierarchy lock to the parent (or to the affected subtree), not to the whole workspace.
4. Recovery text for a live lock: show the actual command and id, how many writers are ahead (if known) and how to wait longer. Mention --force only when the lock is stale (past ttl_seconds, or the owner process is dead).
5. A conformance test: N concurrent writer processes (N = 32) on one non-git tracker must lose 0 appends and 0 creates, with bounded latency.

This is about concurrent agents sharing one tracker without any version control; it does not involve branch merging.