Skip to content

Smaller chat messages in the store: 10.7 KB down to 3.5 KB - #2077

Closed
joepio wants to merge 5 commits into
developfrom
claude/project-thread-kci585
Closed

joepio wants to merge 5 commits into
developfrom
claude/project-thread-kci585

Conversation

@joepio

@joepio joepio commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Requested by Joep · project thread

Before: a chat message with the text "hallo" took about 10.7 KB in the server store (measured on 50 messages, after #1967 (removed duplicate index rows)). The same signed data sat in four places (stored resource, genesis commit row, envelope, Loro snapshot) and the two atom indexes wrote ten rows each.

After: 3.5 KB per message (-67%), measured by the new ignored test measure_chat_message_bytes.

per message before after
resources (message + genesis commit row) 2.7 KB 1.7 KB
Loro snapshot 1.5 KB 0.3 KB
envelope 2.4 KB 0
PropValSub 1.9 KB 0.7 KB
ValPropSub 1.9 KB 0.4 KB
search 0.3 KB 0.3 KB

How, in four commits:

  1. Compression. CompressedKv wraps the store's KvStore; resources, Loro snapshots and envelopes are deflated. A compressed value starts with 0x00 plus a codec byte, which no earlier value in these trees can start with, so old rows are read as they are. Pure Rust (miniz_oxide), so it builds for WASM.
  2. One copy of the genesis commit. Its envelope is no longer written; envelopes() rebuilds it from the commit row (same JSON, asserted in a test). Under all-envelopes retention the first later commit writes it out.
  3. Leaner atom indexes. A creation's commit atoms are not indexed (the commit is found by its id; rights, parent and destroy commits still are). The value index (ValPropSub) only gets values that can be references. Joep agreed to the two behavior changes: a query on a plain text value without a property no longer finds new resources (references still do), and a creation's commit is no longer found through the subject index. The two tests that pinned the old behavior are adjusted.
  4. The snapshot is a delta on the genesis commit. The commit row already keeps the signed loroUpdate; the snapshot keeps only what came after it, and a read puts the two back together. It stores the whole snapshot when the commit row is missing (a resource that arrived by sync) or when the rebuild does not match the original version vector.

Old stores keep working: nothing is rewritten on upgrade, old rows are read as they are, and old index rows go when their resource changes. A downgrade to a build without this change cannot read compressed rows. Reads of a delta snapshot cost two extra Loro imports; if that shows up in collection queries, a small cache is the next step. The compaction test fixture used repeated text and now uses noise.

planning/chat-message-storage.md has the measurement and what is left (shorter index keys, search rows, and short property keys inside the signed Loro document, which is a protocol change).

A chat message took about 10.7 KB on the server for a few bytes of text. Three trees held a third of that as base64 and msgpack that repeats itself. Values in them are now deflated, under a marker byte that no earlier row can start with, so rows an older store wrote are read as they are. A 50-message chat drops from 10.7 KB to 8.3 KB per message.

Adds an ignored test that prints the bytes per tree for a chat.
@joepio joepio added the performance Speed improvements label Oct 5, 2026 — with Claude
@joepio joepio self-assigned this Oct 5, 2026
@joepio joepio added the performance Speed improvements label Oct 5, 2026
A resource's genesis commit already has a row of its own in the resource tree, so its envelope held the same signed bytes again: about 2.4 KB for every chat message. The envelope is no longer written for a genesis commit. Reading a resource's envelopes rebuilds it from that row while nothing newer has replaced it. Under the all-envelopes retention the first later commit writes the genesis envelope out before it, so history is unchanged.

A chat message now takes 6.8 KB instead of 10.7 KB.
@joepio joepio changed the title Smaller chat messages in the store: compress the large rows Smaller chat messages in the store: compress rows, keep the genesis once Oct 5, 2026
A chat message wrote ten rows into each atom index. Five were its creation commit's atoms, which nothing queries (the commit is found by its id), and the value index also held descriptions and timestamps. The commit of a creation is no longer indexed, and the value index only gets values that can be references. Rights, parent and destroy commits are still indexed by subject.

A query on a plain text value without a property no longer finds new resources; references still do. Old rows go when their resource changes.

A chat message now takes 4.2 KB instead of 10.7 KB. Adds planning/chat-message-storage.md with the measurement and what is left.
@joepio joepio changed the title Smaller chat messages in the store: compress rows, keep the genesis once Smaller chat messages in the store: 10.7 KB down to 4.2 KB Oct 5, 2026
The genesis commit row already keeps the signed loroUpdate, and the snapshot of a resource nobody has edited is that same data plus a few server-written changes. The snapshot now keeps only what comes after the genesis (about 250 bytes for a chat message instead of 1 KB) and a read puts the two back together. It falls back to the whole snapshot when the commit row is missing, as for a resource that arrived by sync, or when the rebuild does not match.

A chat message now takes 3.5 KB instead of 10.7 KB.
@joepio joepio changed the title Smaller chat messages in the store: 10.7 KB down to 4.2 KB Smaller chat messages in the store: 10.7 KB down to 3.5 KB Oct 5, 2026
@joepio
joepio marked this pull request as ready for review October 5, 2026 18:19

joepio commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

The failed Downstream check is not a code failure: cargo check --locked in atomic-saas stops because its Cargo.lock does not list the new miniz_oxide dependency of atomic_lib. The paired lock update is in ontola/atomic-saas#193 (Cargo.lock for this PR), on a branch of the same name. The check needs a re-dispatch of main.yml on this branch once that PR is open, which is the one re-run I cannot do from here.


Generated by Claude Code

joepio added a commit that referenced this pull request Oct 6, 2026
One tested commit for the open PRs of 5 October: link device to account (#1948), one profile form (#1953), faster branch runs (#1956), end-to-end encrypted conversations (#1971), passkey ceremony header (#1972), Flutter account package (#2035), cheap canvas undo (#2053), atomic.place tab title (#2056), table resource cell (#2057), small table fixes (#2058), storage map (#2061), Sync page cloud ladder (#2072), several served-domain suffixes (#2073), one sign-in (#2074), smaller chat messages (#2077), bring-along e2e fix (#2078), inline row data loss (#2079), Retry-After (#2080), #2070, #2071, #2066, #2067, #2068, #1969, #1887, #1897 and #1920.

Compressed stored values from #2077 stay readable, but a downgrade cannot read compressed rows.

Co-authored-by: Joep Meindertsma <joep@ontola.io>
@joepio joepio closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Speed improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants