feat(desktop): implement 30178 team catalog backend - #5112
Conversation
16a1ff3 to
e5e5c2a
Compare
wesbillman
left a comment
There was a problem hiding this comment.
P1 — the tombstone can be older than the retained catalog head, so deletion silently leaves the team public.
prepare_team_publication_at and reconcile intentionally assign replacement heads max(now, prior.created_at + 1), which permits a retained 30178 head to be future-dated (commands/teams/pending.rs:159-183; persona_events.rs:112-127). But tombstone_team_catalog_coordinate signs the kind:5 at plain wall-clock now before it opens or reads the retention database (managed_agents/team_catalog.rs:728-756). The relay only deletes coordinate versions whose created_at is at or before the tombstone (crates/buzz-relay/src/handlers/side_effects.rs:2170-2181).
Reproduction shape: retain a shared 30178 head with created_at = now + 60, then delete the team or trigger either unrebuildable-head retraction path. The tombstone is accepted and flushed but applies to zero relay rows. Locally, this helper has already purged the retained 30178 row and queued only that ineffective kind:5 (team_catalog.rs:756-772), so successful flush removes the sole retry witness. The supposedly deleted/retracted team remains publicly discoverable indefinitely. All direct-delete, edit-retraction, and boot-reconcile paths converge on this helper.
Please load the retained head timestamp inside the same BEGIN IMMEDIATE transaction, sign the tombstone with monotonic_created_at(Some(head.created_at)) (falling back to None only when no head exists), then purge and retain atomically. Add a future-dated retained-head regression asserting the tombstone timestamp strictly dominates the head; call-path coverage for direct delete and both retraction paths would prevent wiring regressions.
728ae20 to
9a086df
Compare
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Reviewed exact head 9a086df0167a34482d427a35d2d524315275e0c3 with the Royal Court. Requesting changes for one merge-blocking lifecycle defect.
P1: future-dated tombstones can be rejected, then become permanently unpublishable
The new deletion protocol signs the 30178 kind:5 tombstone strictly after the retained head and atomically purges that head (desktop/src-tauri/src/managed_agents/team_catalog.rs:724-786). That fixes replacement ordering locally, but it conflicts with the relay contract: ingest rejects every event outside ±900 seconds (crates/buzz-relay/src/handlers/ingest.rs:2005-2011). Pending flush republishes ordinary events, including kind:5, byte-for-byte; only identity archive requests receive a fresh signature/timestamp (desktop/src-tauri/src/managed_agents/persona_events.rs:308-337).
The added regression demonstrates the failure shape while accidentally blessing it: it seeds a head at now + 86,400, requires an even later tombstone, and never sends that tombstone through relay ingest (desktop/src-tauri/src/commands/teams/pending/tests.rs:720-788). That tombstone is rejected until it enters the relay's 15-minute future window. If Desktop does not flush successfully during the resulting narrow acceptance interval, the unchanged event becomes stale and is rejected forever. Because the 30178 head was already purged atomically, the durable witness needed to regenerate a usable deletion is gone while the public catalog coordinate can remain live.
The same incompatible pattern exists for the 30176 team tombstone (desktop/src-tauri/src/commands/teams/mod.rs:297-339), so fixing only the catalog helper would leave sibling deletion broken.
Please implement a relay-compatible durable recovery protocol for both coordinates that preserves domination of the target head without relying on publishing an arbitrarily future-dated event. Validate it through the actual relay timestamp gate, including delayed/offline retry past the original acceptance window. A local assertion that tombstone.created_at > head.created_at is not sufficient.
Non-blocking completeness note: the auto-retraction paths emit team-catalog-auto-retracted, but a scoped search under desktop/src found no listener. If user notification is part of the intended behavior, wire the frontend consumer or adjust the claim.
No additional actionable defects survived consolidation. Existing CI was used for broad validation; I did not duplicate CI-equivalent suites locally.
a66f7fa to
6a57da7
Compare
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Re-reviewed exact head 6a57da79961be71055724ca328a1018f2ad8e5b9 with the Royal Court. The prior P1 is resolved, and no new blocking findings survived consolidation.
flush_pending_events_at now treats a retained kind:5 timestamp as the domination floor: it re-signs at max(now, floor) when that timestamp is relay-acceptable, leaves a farther-future tombstone pending without submitting it, and defers a same-coordinate replacement for the sweep (desktop/src-tauri/src/managed_agents/persona_events.rs:311-360). Compare-and-clear still uses the untouched retained row (persona_events.rs:364-372), preserving the concurrent-update fence.
Regression coverage drives both kind:30176 and kind:30178 tombstones through the submission path against the relay’s ±900-second timestamp contract, including in-window domination, zero-POST future deferral, and delayed/offline stale-floor re-signing (desktop/src-tauri/src/commands/teams/pending/tests/gate.rs:157-402). The production 30-second flush loop provides the durable retry path (desktop/src-tauri/src/lib.rs:570-594).
All current PR checks pass. Existing CI provided broad validation; reviewers did not duplicate CI-equivalent suites locally.
Verdict: no blocking findings at this head. Wes retains final approval authority.
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Re-reviewed exact head 6a57da79961be71055724ca328a1018f2ad8e5b9 against base bbd20fae75ecc3bd7a83cc12a65379fac22a2b79.
[P1] Catalog adoption does not enqueue owner-device sync heads
add_verified_team commits the adopted personas and team, regenerates Nest, and returns (desktop/src-tauri/src/commands/teams/adopt/apply.rs:109-136), but the adoption path never calls retain_persona_pending or retain_team_pending. The analogous team-snapshot import enqueues each member and the team after its writes (desktop/src-tauri/src/commands/team_snapshot.rs:751-761).
Consequently, a successful catalog adoption has no pending kind:30175/30176 heads for another device owned by the same user. The only apparent repair is the later startup migration (desktop/src-tauri/src/event_sync.rs:16-31,57-77,202-229), leaving a window where a crash or device loss before restart can lose the only adopted copy despite the command reporting success.
Please enqueue every newly copied or reactivated non-built-in persona and the adopted team after the atomic store commit. Add command-path regression coverage asserting pending 30175 and 30176 rows, including idempotent replay.
The independent schema/protocol and relay/authorization reviews found no additional blocker. Exact-head required checks are green, but they do not cover this successful-adoption retention contract.
6a57da7 to
da22f55
Compare
Implements the backend for 30178 team catalog sharing on the community catalog. No UI callers yet — this PR is the backend half of a two-PR split; desktop PR #3995 (stacked here) adds the TS parse layer, hooks, CommunityCatalogDialog, and e2e. Changes: - team_catalog.rs: 30178 projection builder + size contracts (32 KiB ceiling, per-field bounds, avatar downscaling for raster data URLs); build_team_catalog_event/content, team_catalog_content_from_event, tombstone_team_catalog_coordinate - commands/teams/pending.rs: publish/unshare/tombstone commands with relay-scoped share state; refresh_or_retract_shared_head_at for immediate retraction on member edits that exceed the size contract; persona-edit refresh guard so unrelated personas are never embedded - commands/teams/adopt/: add_team_from_catalog with backend head verification + byte-level rollback across both stores; plan_add with full member provenance (owner, d-tag, member-key, projection-hash); builtin reuse via hint matching; commit_stores atomic writer. The commit and the retention enqueue are sequenced inside commit_and_enqueue, the sole route to a durable adoption commit: after the store write succeeds it enqueues retention heads for every member copy the add wrote or reactivated plus the adopted team, so a crash before the next boot reconcile cannot lose the only adopted copy; reused built-ins and replays write nothing and enqueue nothing, and a failed commit enqueues nothing. - event_sync.rs: reconcile_team_catalog_heads_at startup reconcile; republish changed heads, tombstone unrebuildable ones, skip unshared; multi-head continuation so a single pass handles all shared teams - persona_events.rs: domination-aware flush for kind:5 tombstones. A tombstone is signed strictly past the future-dated head it retracts, so its retained created_at is the domination floor. The relay ingest gate rejects events beyond ±900s of server time, so a byte-frozen future-dated replay can age out of the acceptance window and strand the head live forever. Flush now re-dates to now when the floor has passed, publishes at the floor when it is within the window, and leaves the tombstone pending to converge when the floor is further ahead than the window — never emitting an event the relay rejects. - Rust tests: team_catalog/tests.rs, adopt/tests.rs + adopt/tests/ retention.rs (adoption drives commit_and_enqueue through a spy commit + temp-dir scope: commits-then-enqueues, commit-failure enqueues nothing, idempotent replay skips both, reused-builtin/reactivation provenance), pending/tests.rs (incl. gate tests driving both the 30176 and 30178 tombstones through a stub relay that enforces the real ±900s ingest gate, covering the delayed/offline-retry case), event_sync_team_catalog_tests.rs - 27 shared JSON parity fixtures (team_catalog_content/) - Existing TeamRecord + AgentDefinition types extended with sharing fields (team_catalog_source, catalog_source, shared flag, etc.) - All commands registered in lib.rs; dormant until UI PR merges Stack: this PR -> #3995 Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
da22f55 to
43026a7
Compare
Implements the Rust/backend half of 30178 team catalog sharing on the community catalog. No TS callers yet — this is PR 1 of 2; #3995 (stacked here) adds the parse layer, hooks,
CommunityCatalogDialog, and e2e tests.What this adds
Projection builder (
team_catalog.rs):build_team_catalog_event/contentproduces a 30178 event from a team + member definitions. Size contracts: 192 KiB total ceiling, per-field bounds (name 256 B, text 4 KiB, system-prompt 16 KiB, avatar URL 32 KiB for projection fields; https URLs additionally validated at 2,048 bytes). Avatar handling: oversized raster data URLs downscaled to fit; oversized built-in avatars silently omitted; oversized https URLs rejected with a named error.Share/unshare/tombstone (
commands/teams/pending.rs):prepare_team_publication_atretains a signed 30178 head for the flush loop; share state is relay-scoped (one community can share while another does not).refresh_or_retract_shared_head_atrebuilds or tombstones the head immediately on team/member edit — no stale publication until the next boot. Both the 30178 catalog and 30176 team tombstones are signed with acreated_atthat strictly dominates the retained head's (read inside the delete transaction), so a future-dated head cannot survive its own deletion under the relay'screated_at <=soft-delete gate.Domination-aware flush (
managed_agents/persona_events.rs): a strictly-dominating tombstone can be signed past the relay's ingest acceptance window (MAX_TIMESTAMP_DRIFT_SECS, ±900s from server time). Republishing such a byte-frozen event verbatim from the pending queue lets it age out of the window and be rejected forever, stranding the head live. The flush loop is now domination-aware for every retained kind:5 (covering both 30176 and 30178). For a pending tombstone with floorf(= the retained row's owncreated_at):f <= now→ re-date and re-sign atnow(mirrors the existing archive-request re-sign branch;mark_syncedstays keyed to the untouched retained row, so a re-date can't mask a concurrent edit),now < f <= now + 900→ publish verbatim atf, inside the window,f > now + 900→ skip the sweep; the event stays pending and its replacement keeps deferring (viafailed_tombstones) so out-of-order retraction remains impossible, converging as the wall clock advances.No path emits an event the gate rejects, and a boundary reject self-heals through the submit-error requeue. Durable across offline gaps of any length.
Atomic adopt (
commands/teams/adopt/):add_team_from_catalogre-fetches and signature-verifies the head from the relay, then plans and commits a multi-entity add across two store writes with byte-exact rollback on any failure (crash window between writes explicitly retained).plan_addresolves full member provenance (owner, d-tag, member-key, projection-hash), reuses exact-match built-in personas, and reactivates deactivated copies on re-add.commit_storessnapshots both stores before writing and restores them on failure. The commit and the retention enqueue are sequenced insidecommit_and_enqueue, the sole route to a durable adoption commit: once the store write succeeds it enqueues a pending 30175 for every member copy the add wrote or reactivated and a pending 30176 for the team, so a crash before the next boot reconcile cannot lose the only adopted copy. A byte-identical reused built-in and an idempotent replay write nothing and enqueue nothing; a failed commit propagates and enqueues nothing. Enqueue is best-effort per row (the boot reconcile is the backstop). The frontend refreshes via theuseAddTeamFromCatalogMutationquery invalidation in #3995, so noagents-data-changedemit is needed here.Startup reconcile (
event_sync.rs):reconcile_team_catalog_heads_atwalks all retained 30178 heads at boot: republishes heads whose content changed, tombstones heads whose team or member was deleted, skips unshared heads and unchanged content. Multi-team continuation — all shared teams processed in one pass.Types:
TeamRecordandAgentDefinitionextended withshared,catalog_source,team_catalog_sourcefields. All commands registered inlib.rs.Tests
team_catalog/tests.rs: projection, size contracts, member-key stability, tombstone rollback, fixture matrixadopt/tests.rs: head verification, store planning, provenance, rollbackadopt/tests/retention.rs: adoption drivescommit_and_enqueuethrough a spy commit + a real temp-dir retention scope and asserts persisted pending rows — commits-then-enqueues (30175 per minted member + 30176 team), a failed commit enqueues nothing, an idempotent replay skips both the commit and the enqueue, a reused built-in retains only the team, and a reactivated copy is re-retained. Deleting the enqueue inside the seam turns these red — the wiring, not just the helper, is protectedpending/tests.rs: share/unshare/tombstone lifecycle, edit refresh/retract, tombstone timestamp domination, typed outcomespending/tests/gate.rs: flush driven through a stub relay that logs everyPOST /eventswith its accept/reject status and enforces the real ±900s ingest gate, for both 30176 and 30178 — within-window publish+dominate; beyond-window stays-pending with zero POSTs (the gate never receives a rejectable event); and the delayed/offline-retry case where a tombstone signed strictly past a then-future head has aged more than 900s into the past, so flush must re-date tonowto publish. The reversal check — restore the byte-frozen replay inpersona_events.rsand the delayed-retry and zero-POST assertions go red — is what proves the suite discriminates the fix from the rejected implementationteams/tests.rs: 30176 tombstone timestamp domination and no-head fallbackevent_sync_team_catalog_tests.rs: reconcile scenarios including multi-head continuationtests/fixtures/team_catalog_content/) consumed by both this PR's Rust tests and feat(desktop): add team sharing to community catalog #3995's TS testsStack: this PR → #3995