Skip to content

Shard messagesByThread into per-thread files: one message re-serializes all 227 threads #225

New issue
New issue
Closed
Closed
Shard messagesByThread into per-thread files: one message re-serializes all 227 threads#225
Labels
plan:donePlanboard status

Description

@currentbits
currentbits
opened on Aug 17, 2026

Split out of #124. The debounce backoff landed there mitigates this; this is the structural fix.

Measurement

~/Library/Application Support/Solenta/coder-store.json on the daily driver is 54MB, and JSON.stringify(store, null, 2) on it takes ~58ms on the main-process event loop. Composition:

messagesByThread   52.5MB   <-- 97%
workLogByThread     0.3MB
threads             0.2MB
projects/usage/settings/spend/automations/templates  ~0.0MB

Within messagesByThread: 227 buckets, 0 empty, 0 orphaned (no live thread), median 190KB, largest 1.32MB / 1065 messages.

This is not a leak — the caps work

Worth stating plainly, because it rules out the obvious first guess: #89's per-thread transcript caps are doing their job. No runaway bucket, no orphaned buckets from deleted threads, largest thread is a reasonable 1.3MB. Nothing here needs garbage collecting.

The problem is purely that total size is linear in thread count: 227 threads x ~190KB = 52.5MB. Capping each thread cannot bound the total, so this gets worse with every thread and no per-thread cap will ever fix it.

The actual inefficiency

The store is one file, so a flush is all-or-nothing. When one message arrives on one thread, we re-serialize the message history of all 227 threads to persist it. That is ~0.2ms of useful work wrapped in ~58ms of waste.

Proposed fix

messagesByThread is already a map keyed by thread id, so it shards almost for free: one file per thread, e.g. messages/<threadId>.json, and a flush writes only the dirty threads. Cost per flush drops from stringifying 52.5MB to stringifying ~190KB — roughly 58ms to ~0.2ms, and it stops scaling with thread count.

workLogByThread (0.3MB) is keyed the same way and could follow, but at 0.3MB it is not worth doing on its own — only shard it if the same code path makes it nearly free.

Notes for whoever picks this up:

  • Needs a migration that splits the existing 54MB file and is safe to interrupt half-way.
  • The saveNow() exit-hook path is sync on purpose (process.on(exit) cannot await) and must stay sync — sharding means it writes N files instead of one, so keep the dirty-set small and only write what changed.
  • Once this lands, the debounce backoff added in App beachballs when multiple threads run concurrently #124 can likely revert to a flat 250ms, which also shrinks the unsaved-data window back down from 2s.

Why not other options

  • Worker thread for the stringify — does not help. Structured-cloning the store to the worker costs about what stringifying it costs.
  • Dropping the null, 2 indent — ~2-3x on a problem that wants ~250x, and it costs the human-readable/diffable store file.

Activity

  1. currentbits
    added
    plan:todoPlanboard status
    on Aug 17, 2026

  2. currentbits commented on Aug 19, 2026

    @currentbits
    currentbits
    on Aug 19, 2026
    OwnerAuthor

    Fresh numbers, same machine, ~1 month later — the linear-in-thread-count prediction held exactly:

    coder-store.json            143.2 MB   (was 54 MB)      + 139.5 MB .bak
threads                     445        (was 227)
messagesByThread            134.35 MB  = 99.5% of payload
  archived threads          126.3 MB   (430 threads)
  live threads              8.1 MB
  tool-role messages        96% of message bytes
JSON.stringify per flush    162 ms sync on main   (was ~58 ms)
boot JSON.parse             375 ms

    Flush debounce is 250 ms backing off to 2 s (store.js:273-276), so under streaming the main process burns 162 ms per flush re-serializing archived transcripts. The ponytail: comment at store.js:1186 set its own trigger — 'per-thread files if the store stays >50MB' — and we are at 2.9x.

    I filed #560 for this before finding this issue; closing it as the dupe. One thing worth keeping from it: the split also shrinks the whole-store .bak (139.5 MB today) to ~700 KB, and makes transcript export (#394) and archived-transcript deletion per-file operations.

  3. currentbits commented on Aug 19, 2026

    @currentbits
    currentbits
    on Aug 19, 2026
    OwnerAuthor

    Live profiler evidence from the running app (pid 32564, sample 32564 5), to size the win:

    • A libuv worker thread spent 3,690 of 4,126 samples — 3.7 of 5 seconds — inside a single rename() of the store tmp file (uv__fs_post → rename). The flush pipeline is saturated renaming 143 MB payloads.
    • Watching coder-store.json mtime for 30 s with the app near-idle (3 idle claude children): 2 full rewrites = 286 MB written in 30 s, ~9.5 MB/s sustained. At the 2 s debounce cap under active streaming that is ~71 MB/s of SSD writes — hundreds of GB per working day, all to persist a few KB of new tokens per flush.
    • Main process RSS is 1.0 GB; the deserialized 143 MB blob plus lastPushByThread copies are the obvious bulk. The split shrinks that too.

    Filed the two neighbours this profiling surfaced separately: #563 (boot sweep du storm) and #564 (renderer mounts whole transcripts).

  4. currentbits commented on Aug 20, 2026

    @currentbits
    currentbits
    on Aug 20, 2026
    OwnerAuthor

    One day later (2026-08-20), same machine. The curve is steepening, not flattening:

    2026-07 (#225 filed) 2026-08-19 2026-08-20
    store 54 MB 143 MB 170 MB
    threads 227 445 545
    JSON.stringify ~58 ms 162 ms 310 ms
    JSON.parse at boot — 375 ms 624 ms
    .bak sibling — 139 MB 168 MB

    Composition is now 163.8 MB archived transcripts vs 4.8 MB live — 97% of every flush is threads nobody will open again. 310 ms of synchronous main-process stringify, up to every 2 s under streaming, to persist a few KB.

    Worth noting against #559/#563, which both shipped this week: worktree disk went 96 GB → 29 GB, so the disk half of the dogfooding findings is fixed. This is the one still compounding, and it is now the largest single main-process stall in the app.

  5. currentbits commented on Aug 21, 2026

    @currentbits
    currentbits
    on Aug 21, 2026
    OwnerAuthor

    Round 31 live measurements (2026-08-21, store now 180 MB, was 143 MB on 08-19 — growing ~18 MB/day):

    While a single background worker streams:

    • The store was rewritten ~6 times in a 30 s window (mtime deltas: 3–6 s apart). Each flush is a full JSON.stringify(this.data) of 180 MB on the main thread (store.js:1175) plus a 180 MB tmp write + rename. That is ~1 GB of SSD writes per 30 s (~36 MB/s sustained) for appending a few KB of transcript.
    • Main process burned 7.2 CPU-seconds in that 30 s (~24 % of a core) — a 5 s sample shows nearly all busy main-thread time inside timer-driven V8 string building/memmove, i.e. the stringify. The renderer was at 0.2 % the whole time.
    • Main RSS 1.2 GB (peak 1.8 GB): the whole store lives as JS objects and every flush allocates a transient 180 MB payload string on top.

    This is now the dominant CPU, battery, memory and SSD-wear cost of the app; everything else measured (idle tickers, renderer, pushDetail diffing) is healthy.

    Interim one-liner worth taking before the shard migration: raise SAVE_DEBOUNCE_MAX_MS (store.js:277) from 2 000 to ~15–30 000. The crash-loss window is only the transcript tail, which the provider CLIs' own session logs can restore on resume; a 15 s cap cuts flush CPU/disk by ~5–10× immediately.

  6. currentbits commented on Aug 21, 2026

    @currentbits
    currentbits
    on Aug 21, 2026
    OwnerAuthor

    Interim mitigation landed on main as 755f0a2: SAVE_DEBOUNCE_MAX_MS raised 2 s → 15 s, cutting the measured streaming flush churn (~6 rewrites / 30 s, ~1 GB disk writes, ~24 % of a core) by roughly 7×. Same commit also removed the per-sidebar-click rewrite (#636) and the boot .bak byte-copy (#637). This issue stays open for the real fix: sharding messagesByThread.

  7. currentbits commented on Aug 21, 2026

    @currentbits
    currentbits
    on Aug 21, 2026
    OwnerAuthor

    Fresh live measurement (2026-08-21 evening, nightly build with the #636/#637/15s-debounce interim fixes, commit 755f0a2):

    Idle is fixed. 0 store rewrites in 30 s with no active runs; main/renderer/GPU all idle under sample(1). Clicking threads and boot no longer flush.

    Streaming is still the whole cost. With one thread actively streaming:

    • 4 full-store flushes in 45 s = 733 MB written (~16 MB/s sustained SSD write)
    • each flush = JSON.stringify of the now 192 MB store on the main thread (store.js:1193), blocking ~1-2 s each
    • store growth continues: 143 MB (08-19) → 180 MB (08-21 AM) → 192 MB (08-21 PM), ~18-20 MB/day; messagesByThread is 181.4 MB of it across 617 threads, no single blob messages (largest thread 2.3 MB / 1073 msgs)

    Also in scope for the sharding: the quit path does an unconditional full-store sync saveNow() (electron/runner.js:5638), so every quit rewrites all 192 MB and blocks on it. With per-thread shards, quit would write only dirty shards.

    The 15 s debounce + backoff bought headroom, but the per-flush cost still scales with total store size, which is growing ~10%/week. Sharding remains the real fix.

  8. currentbits
    added 2 commits that reference this issue on Aug 21, 2026

    perf: cheaper store boot and flush cadence (#637, #225 interim)

    6b7db99

    perf: stop rewriting the whole store on click and boot (#636, #637, #225

    755f0a2
  9. currentbits
    added
    plan:doingPlanboard status
    and removed
    plan:todoPlanboard status
    on Aug 22, 2026

  10. currentbits commented on Aug 22, 2026

    @currentbits
    currentbits
    on Aug 22, 2026
    OwnerAuthor

    Implemented in this worktree: transcripts now live in messages/<threadId>.json next to coder-store.json. A flush stringifies only dirty shards plus the small envelope.

    • Load splits a legacy blob without JSON.parse of tool payloads; existing shard files win so a half-finished migrate is safe to resume.
    • saveNow() stays sync and only writes what changed (plus reclaim of an aborted in-flight snapshot).
    • Debounce backoff cap is gone: flat 250ms again.
    • workLogByThread stays in the envelope (still ~0.5MB).

    Envelope on disk has messagesByThread: {}. Grep/forensics of coder-store.json for transcripts will miss them; they are under messages/.

  11. currentbits commented on Aug 24, 2026

    @currentbits
    currentbits
    on Aug 24, 2026
    OwnerAuthor

    Implemented and Grok-reviewed in thread worktree 19029a3a-c692-4a24-8c28-2491a5173d8e.

    • messagesByThread now persists as lazy per-thread messages/<encoded-id>.json shards; coder-store.json keeps an empty transcript envelope.
    • Flushes write only dirty/deleted shards plus the small envelope; saveNow() remains synchronous and stale async snapshots cannot overwrite it.
    • Legacy migration is resumable and fail-closed, existing shards win, pre-migration backup/recovery is preserved, and the debounce is flat 250 ms again.
    • Added migration, corruption, path safety, deletion, partial-I/O, and async race coverage.

    Verification:

    The issue remains open / plan:doing until these uncommitted worktree changes are committed and landed.

  12. currentbits
    added
    plan:donePlanboard status
    and removed
    plan:doingPlanboard status
    on Aug 24, 2026

  13. currentbits commented on Aug 24, 2026

    @currentbits
    currentbits
    on Aug 24, 2026
    OwnerAuthor

    Implementation completed and Grok-reviewed in thread 19029a3a-c692-4a24-8c28-2491a5173d8e. Focused tests pass 88/88 and the production build passes.

  14. currentbits
    closed this as completedon Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    plan:donePlanboard status

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions