[luvs01]

Summary

Recover the unfinished CPU/RAM/disk-efficiency audit, independently based on fork dev 306a88e08b387a7d3d2ee51bf04e72530285e5b4.

An unterminated JSONL tail cannot advance indexedOffset. Repeated history refreshes therefore reread the same bytes, allocate read/fragment buffers and rewrite five SQLite metadata fields even when nothing changed.

• Retain one small successful partial-tail observation per live database handle in a WeakMap, never the ledger bytes.
• Bind reuse to the path, exact bigint device/inode, birth time, size, nanosecond modification/change timestamps, next complete-record offset and worker end bound.
• Check expected worker identity before reuse; publish an observation only after successful ingest with an unchanged post-read revision.
• Clear observations on schema/source-meta resets. A replaced/changed file, different path/bound, new handle or failed read does not reuse it.
• Keep the indexer's schema, identity and PRAGMA quick_check gates unchanged. Canonical history is not rewritten, partial records stay unindexed until the newline arrives, and no record-size limit is weakened.
• Add six regressions to the existing registered history-worker suite. The revision uses bigint stat timestamps; the persisted modification time remains numeric milliseconds (derived from nanoseconds).

Verification

Original ingest, schema and test bytes were checked against their Git blob SHA before editing.

Executed locally: six new test bodies against the actual transpiled ingest and schema modules, real temporary files and real SQLite through Node 22.16.0's DatabaseSync, with a Bun database API wrapper. The selector encoder, file-identity helper and test assertions were substituted; the file-identity substitute uses real bigint fstat. No live user home or provider was accessed.

• Baseline unchanged-tail test failed as expected: three repeat calls made three ledger reads.
• Patched six cases passed: unchanged-tail zero reads/zero SQLite total_changes delta plus newline completion; same-size edit/replacement/path; schema reset/new handle; worker end bound/expected identity; failed-read retry; source growth during reading.
• TypeScript transpilation found no syntax errors; it is not repository typechecking.

Not executed/claimed: native Bun SQLite behavior, existing worker/integration suite, repository typecheck, platform suite or whole-process percentage improvements. Exact-head hosted CI is required; draft pending that evidence.

Native validation:

bun run test --parallel=2 ./tests/usage/request-history-rebuild-worker.test.ts ./tests/usage/request-history-index.test.ts
bun run typecheck
bun run structure:check
bun run privacy:scan

No merge, deployment, live restart, workflow change or policy relaxation.

Co-authored-by: luvs01 27862058+luvs01@users.noreply.github.com

Summary by CodeRabbit

• Performance

  • History rebuilds skip re-reading unchanged portions of source files, which can reduce unnecessary work when processing incomplete file tails.

• Reliability

  • File changes, replacements, and failed reads are detected so history processing can retry or read the updated source.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

No actionable comments were generated in the recent review.

ℹ️ Recent review info

⚙️ Run configuration

• Source mode: Auto
• Repository: luvs01/opencodex
• Branch: perf/history-unchanged-partial-tail
• File: .coderabbit.yaml
• Commit: 00c1e34
• Layers: Repository YAML
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: c3017a8d-cb07-4b0e-9d6c-4dc014fc221c

📥 Commits

Reviewing files that changed from the base of the PR and between 306a88e and 00c1e34.

📒 Files selected for processing: 2

• src/routing/history/ingest.ts
• tests/usage/request-history-rebuild-worker.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Walkthrough

ingestSourceTail now caches observations of incomplete file tails per database handle. It skips a read when the cached path, file revision, indexed offset, and configured end offset match. Added tests cover reuse, invalidation, file changes, bounds, and failed reads.

Changes: Tail Ingestion Cache

Layer / File(s)	Summary
Tail cache state and invalidation src/routing/history/ingest.ts	A per-database cache stores file revision details and the next indexed offset. Schema resets and source metadata updates clear the cached observation.
Tail reuse, validation, and tests src/routing/history/ingest.ts, tests/usage/request-history-rebuild-worker.test.ts	Ingestion uses bigint file metadata to check cached observations and only caches an observation if the file revision remains unchanged. Tests cover reuse, invalidation, file changes, bounds, identity checks, and read failures.

Priority: Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Suggested reviewers: lidge-jun

Merge Risk: Minimal · up to 00c1e

No merge-blocking issue was established for the tail-observation cache. Normal checks remain appropriate before merging.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title accurately and concisely describes the primary change: skipping unchanged partial-tail reads and SQLite metadata rewrites in history processing.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

✅ Deterministic PR hygiene checks passed.

[luvs01]

Recovery verification update: GitHub Actions Cross-platform CI run 40033666102 completed successfully for this PR head 00c1e34bb8b03a8bd585d6062dd4bea559f517a6. React Doctor (40033666065) and Catalog async contracts (40033666082) also succeeded.

This supersedes the description's pending-hosted-validation note for this exact head. The no-repeat-read result refers to ledger byte reads and SQLite metadata changes for an unchanged partial tail, not elimination of file-open/stat operations. Integrity checks remain enabled. Local testing remains the documented Node/SQLite compatibility harness; skipped platform jobs are not counted as executed passes. The PR remains draft and unmerged; no deployment or live configuration was changed.

[luvs01]

No description provided.

[luvs01]

Closing this tracking PR after carrying the validated change into the upstream review queue. Commit c6261d671b709580ceea3ac2830303a3b7a27717 preserves both change-owned file blobs from 00c1e34bb8b03a8bd585d6062dd4bea559f517a6 and integrates current dev security dependencies. Bun 1.4.4 Windows validation passed 42 cases/1,688 assertions, with scoped static checks and independent source review passing. The branch is preserved; this is not a merge or release claim.

[github-actions]

⏳ DRAFT

• UI screenshot required.

What to do

• Add a screenshot of the UI change to the PR description.

This pull request was already a draft. Its draft status will be preserved after every issue above is resolved.