Skip to content

BAD CODE: a deferred capture silently drops thoughts from recent and stats #35

Description

@flyingrobots

Observation

When capture followthrough exceeds its budget, the raw thought is committed and stays readable through inspect — the trapdoor working as designed. The read model does not survive it.

Reproduction, against main:

M=$(mktemp -d); git -C "$M" init -q
for i in A B C; do
  THINK_REPO_DIR=$M THINK_CAPTURE_FOLLOWTHROUGH_TIMEOUT_MS=120000 \
    node bin/think.js "healthy $i"
done
# recent -> 3 entries, stats total -> 3   (correct)

THINK_REPO_DIR=$M THINK_CAPTURE_FOLLOWTHROUGH_TIMEOUT_MS=1 \
  node bin/think.js "DEFERRED D"
# recent -> 1 entry (only DEFERRED D), stats total -> 1
#   A, B and C have vanished from both surfaces

THINK_REPO_DIR=$M THINK_CAPTURE_FOLLOWTHROUGH_TIMEOUT_MS=120000 \
  node bin/think.js "healthy E"
# recent -> E, C, B, A   stats total -> 4
#   D is now gone permanently; the total should be 5

A single deferral does three things:

  1. Hides every older capture while the deferred entry is newest.
  2. Loses the deferred capture permanently from both surfaces once a later healthy capture rebuilds the read model.
  3. Leaves recall inconsistent. In this reproduction remember returned zero matches; the archived mind in test/fixtures/cas/readme-smoke-mind.json returns one for its own deferred capture. That disagreement is itself the defect.

Not introduced by the followthrough-budget work

Confirmed on origin/main at 24e2613 by temporarily forcing the then-hardcoded CAPTURE_FOLLOWTHROUGH_TIMEOUT_MS to 1: same result. Exposing the budget as THINK_CAPTURE_FOLLOWTHROUGH_TIMEOUT_MS only made the state easy to reach deliberately.

Likely mechanism

src/store/read-model.js writes fastCaptureRecordsJson as a single-element array during the raw save; the full recentCaptureRecordsJson list is reconciled during followthrough. Abandon the followthrough and the single-record fast index is what recent and stats read. The later rebuild sources from the graph, where the deferred capture was never linked (canonicalThought.stored is false), so it is omitted.

Why this is bad

Deferral is reachable in normal use — a cold repository with Git fsmonitor enabled can exceed the 6s budget on its first write, which is what made an MCP acceptance assertion flaky. An agent that captures a decision, receives the documented "not a failure" warning, and moves on has silently dropped that thought out of every surface it would later search. For a memory product this is the worst available failure mode: silent, and indistinguishable from never having captured.

Suggested direction

Either write a complete fast index during the raw save so it is never a partial view, or have the read model treat a single-record fast index as unreconciled and fall back to a full scan. The rebuild should also include raw entries whose canonical thought was never stored.

References

  • Backlog card with the same repro: docs/method/backlog/bad-code/CORE_deferred-capture-corrupts-the-recent-read-model.md
  • Frozen regression fixture: test/acceptance/readme-smoke-mind-fixture.test.js pins the deferred state, which cannot be produced on demand

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions