[jadenfix]

Summary

skycap check <record_dir> [trajectory_ids...] reports whether each recorded trajectory stayed on one path, and why each fork happened, so a harness whose history should be append-only can be checked against a recorded run.

$ uv run skycap check ./record
tr_a  finished  linear
tr_b  finished  2 paths
  node 2 (under node 0, vs node 1): edited reply: the harness sent back the model's reply with reasoning_content dropped
1 of 2 trajectories linear; forks: 1 edited reply

• Causes: resample (another reply to the same history), edited reply (a model reply sent back with fields changed, e.g. stripped reasoning), re-rendered (token mode: the same message rendered to different tokens), different message (compaction, subagent, rewritten turn), tools or model (the same message with another tool set or model). In token mode it also counts unbridged calls.
• Exit status: 0 if every trajectory is linear, 1 if any forks, 2 if a record can't be read. A record that can't be read (including a document that isn't a JSON object, or whose fields have the wrong shape) is reported by its file name, and the rest are still checked.
• Cheap: reads only the record documents, never the token sidecars.
• Shared code: the graph rebuild moves out of record.load into record.add_nodes, which check reuses. check normalizes messages with MatchKey.fields, and test_check builds on test_graph's Harness and message helpers.

How a fork's cause is found

The graph is walked only at its fork points (graph.roots() and graph.branch_points()). For each child after the first, check picks the earlier sibling the request should have matched and diffs the two. The pick is one ranked min() over the earlier siblings: same match hash, then same message, then same role (for a harness-written node, the fewest changed fields), then a model reply over a harness-written one, then the latest.

The graph can't answer this pick by itself: a fork exists only because the request's message matched no child's hash, so after the hash and the message, content is the only signal left. Following the call history instead would be wrong for the main case. Resample A, resample B, then send back A with its reasoning stripped: the latest call is B, but the edit came from A, and only a content diff finds A.

Why not an index of siblings

An earlier revision kept five indexes (by match hash, message, role, model reply per role, latest) to make the pick O(1) for most nodes. It was removed because it didn't earn its code:

siblings under one node	model replies: index / min()	harness edits: index / min()
64	0.000s / 0.001s	0.003s / 0.003s
500	0.003s / 0.07s	0.15s / 0.19s
5,000	0.09s / 7.1s	15.3s / 18.5s

• It picked the same sibling in every test.
• It only helped with thousands of model replies under one node. skycap rejects n > 1 (one call is one sample) and a trajectory is one agent run, so that many separate calls on the same history isn't a real workload, and identical replies already dedupe into one node.
• Harness edits stayed quadratic with it anyway.
• check runs offline, after the run, never on the serving path.

If very wide forks ever show up, caching the latest model reply per role brings back most of the speed without the other four indexes.

Review comments

• Gemini: scan branch points only. Done.
• Gemini: count changes instead of formatting them. Superseded: the pick no longer runs on a hot path, so it uses _changes directly.
• Gemini: _a grammar. Done ("an unnamed", "a user").
• Gemini: .get("id") / .get("status"). Left as is: check_dir catches the KeyError and reports the record as unreadable under its file name, the same contract record.load has.
• Greptile P1: a non-object record crashes the run. Done: reported as unreadable. A record that is an object but whose fields have the wrong shape (e.g. capture a list) also raised AttributeError and stopped the run; it is now a read error too. Both are covered by test_an_unreadable_record_is_reported_and_exits_2.
• Greptile P2: index the siblings. Tried and reverted, see above.

Testing

• cd skycap && uv run pytest: 245 passed, 1 skipped (on the branch merged with current main)
• pre-commit run --files $(git ls-files skycap): ruff, black, gitleaks clean

---

Note

Low Risk
Offline read-only validation on record JSON; record.load behavior is preserved via add_nodes refactor with no serving-path changes.

Overview
Adds skycap check, a CLI that scans a --record-dir and reports whether each trajectory stayed on a single root-to-leaf path or forked, with a labeled cause per fork (resample, edited reply, re-rendered, different message, tools or model). It reads record documents only (no token sidecars), rebuilds the message graph via new shared record.add_nodes, walks branch points to compare each extra sibling against the best-matching earlier one, and prints a per-trajectory summary plus aggregate fork counts. In token mode it also reports unbridged calls. Exit codes: 0 all linear, 1 any fork, 2 unreadable record or usage error; optional trajectory id filters are supported. README documents the workflow; test_check covers fork kinds, token-mode behavior, CLI integration, and malformed records.

^{Reviewed by Cursor Bugbot for commit b16fc84. Bugbot is set up for automated code reviews on this repo. Configure here.}

[gemini-code-assist]

Code Review

This pull request introduces the skycap check command to verify if recorded trajectories remained linear and to identify the causes of any forks. The implementation includes the core verification logic, CLI integration, documentation, and comprehensive tests. The review feedback provides valuable improvements for the new check.py module, including optimizing fork detection using branch points, preventing potential KeyErrors when parsing document metadata, optimizing sibling comparison to avoid redundant string allocations, and correcting the grammatical logic in the _a helper function.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; the malformed-record crash is fixed and no new blocking issue was found.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Record directory] --> B[Read each record document]
  B --> C{Can the record be read?}
  C -->|No| D[Report unreadable and continue]
  C -->|Yes| E[Rebuild graph without token sidecars]
  E --> F[Count paths and explain forks]
  D --> G[Print reports and summary]
  F --> G
  G --> H[Exit 0 for linear, 1 for forks, 2 for unreadable]

Loading

_{Reviews (2) · Last reviewed commit: "[skycap] test_check: fold the subagent r..." · Reviewed by Greptile}

[jadenfix]

Hey @kouroshHakha, thanks again for reviewing #2425! This adds skycap check from #2318. CI is green and all review threads are addressed. Could you take a look when you have a chance?

[jadenfix]

@greptile re-review pls - should have fixed the malformed record crash

[kouroshHakha]

Thanks for this, and for the careful write-up (the sibling-index benchmark especially).

We currently have a lot of duplicate work in skycap-viewer and I think the right place for this inspection is actually there instead of adding it to skycap:

• The viewer already covers most of it: the forked and unbridged health flags, the Tree view with forks marked, and the Forks view that compares two branches from the last message they share. For CI, skycap-viewer summary <dir> --json reports forked trajectories and branch points, so a harness can be gated on forked == 0.
• We can keep the focus of skycap package here on the capture itself.