Skip to content

Per-table change list: rows changed since a cursor, including deletions (tombstones) #1850

Open

Description

@michielbdejong

A host API that returns the rows of a table changed since an opaque cursor, including deleted rows (tombstones), with a new cursor. Plugins use it for correct two-way sync, and it backs the durable afterCommit hook.

Suggested shape, open for design:

  • an opaque per-table sequence cursor;
  • each change entry has the row subject, change kind (created, updated or deleted) and the row's Loro version (each row is its own LoroDoc), so a plugin can compute field-level diffs against its baseline;
  • paginated and bounded;
  • tombstones kept long enough for sync, with a documented retention;
  • rights-checked like reads.

It overlaps with ontola/atomic-plugins#167 section 5 (query cursors and tombstones); coordinate there so it's built once.

From Michiel's decisions on plugin two-way sync (ontola/atomic-plugins#177, questions 4–7). The pattern is "hook for speed, change list for correctness".

Activity

  1. michielbdejong commented on Sep 25, 2026

    @michielbdejong
    ContributorAuthor

    This backs the durable afterCommit hook in #1851.

  2. michielbdejong commented on Sep 29, 2026

    @michielbdejong
    ContributorAuthor

    Picking this up (Claude, atomic-server worker). A design and implementation PR against develop will follow.

  3. michielbdejong commented on Sep 29, 2026

    @michielbdejong
    ContributorAuthor

    The per-table change list is up for review in #1885 (against develop).

    API

    • HTTP: GET /changes?table=<subject>&since=<cursor>&limit=<n> returns {changes: [{subject, kind: "created"|"updated"|"deleted", version: {peerId: counter}, at}], cursor, hasMore}.
      • Page size: 100 by default, at most 500.
      • Without since, it lists every current row.
    • Refusals: 410 {error: "CURSOR_EXPIRED"} means resync without since. 400 INVALID_CURSOR and 400 NOT_A_TABLE are the other typed answers. A caller who can't read the table gets the usual 401/404, which names no row.
    • Rust: atomic_lib::change_log::table_changes(&db, &table, since, limit, &for_agent).
    • @tomic/lib: store.getTableChanges(table, {since, limit}). A 410 is thrown as TableChangesCursorExpiredError.

    Semantics

    • What a table is: a resource with class Table and a classtype. Its rows are its children of that class. Moving a row out of the table, or removing its class, counts as deleted there.
    • Ordering: the cursor is opaque and per table. Entries are strictly ordered, and only the latest entry per row is kept, so treat created and updated alike (upsert).
    • What is covered: local commits, remote commits, sync-replicated resources and sync removals all land in the log, in the same transaction as the state they describe.
    • Retention: tombstones are kept for 30 days (--table-change-retention-days). An older cursor gets CURSOR_EXPIRED.
    • Backfill: tables that existed before the log list all their current rows as created on the first request.

    Scope: it is per table, not per folder. If your protocol plugins need per-folder or per-query cursors, say so on #1850 and I'll see whether the same log can back them. No plugin host call (ctx) yet: #1851 defines the plugin side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestlib rustatomic-lib (rust)pluginShould probably be an Atomic Pluginserveratomic-server

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions