You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
chatgpt: an upstream text change to an emitted message is never stored (messages is append_only) #605
connectors/chatgpt/manifest.json:151 declares messages as append_only (main 5584b0e, v0.3.13). The connector relies on that in index.ts:3063. When a later run re-emits the same message id with changed content, the Personal Server keeps the first stored version and discards the new one. In vana-com/personal-server-ts at 8a3f1b81, ingest-plan.ts:112-119 returns unchanged with flag: "append_only_conflict", pdpp-records-sqlite-store.ts:1007-1022 writes nothing to current state or history, and the route answers HTTP 200 (pdpp-records.ts:1383-1397).
The server does report the conflict, per input, in that 200 response: results[i] carries outcome: "unchanged" with flag: "append_only_conflict", and a true replay has no flag. The aggregate unchanged count merges the two, and nothing durable records the conflict. Whether ingest clients read the per-input flag is not verified here. The run finishes cleanly and the stored text stays at the first capture.
ChatGPT messages carry no update_time or status, so the connector can't tell an edit from a render change. We have not observed a changed re-emit in real data, so this is latent today.
Proposal: declare messages as mutable_state in a 0.4.0 connector, with id, the primary key and the cursor unchanged. Emit content_sha256 as a text identity only: SHA-256 of a versioned, lossless preimage of the unescaped extracted text (for example UTF-8 of JSON.stringify(raw), tagged raw-json-string-v1, with null kept distinct from ""). A message counts as changed when that hash or its attachment, tree, branch or metadata fields differ. The server then upserts and keeps the old version in pdpp_record_changes. The first 0.4.0 resend of a 0.3.x row, which has no hash, must compare against a view derived from the stored row so the new field does not record a fake edit.
Not verified: how the provider changes a message on an edit needs a live capture.
Design pass on the proposal. The upgrade-resend result below was read at an unreleased head (6c7c90c), which changes the owed and cursor accounting, so it must be re-checked against the version that ships before 0.4.0. The proposal above is revised to the narrower identity it describes.
At that head, a completed install with its cursor at rest resends no historical messages on upgrade, provided 0.4.0 leaves the cursor and enumeration version alone. A conversation that does advance resends every selected message in it, unchanged siblings included, so the server's no-op comparison carries the load.
A hash of the text alone is not stable for an unchanged message. extractFallback stringifies unknown content (parsers.ts:270-276), so a key-order change alters the hash, and text past the 5000-character cut in extractContentParts (:425-433) is never seen. Attachment-only and tree changes do not reach the text at all. The natural hash point is extractMessageWithBlob (parsers.ts:629-655), which sees the text before escaping and before the inline/blob storage choice.
Bumping the version does not repair conflicts already behind the cursor. That needs a separate, resumable messages repair scan. The append-only parent freeze at index.ts:3081-3100 and :3126-3133 goes away with the change.
The design includes 14 test specs; none has run. No build starts until the current release ships.
Correction to the body: "We have not observed a changed re-emit in real data" is no longer true. One ordinary run of connector 0.3.10 on an instance with real history reported one append_only_conflict on messages. The host kept the first stored version, as described above. Which field changed is not known. So the defect is not only latent.
Activity
tnunamak commentedon Nov 4, 2026
Design pass on the proposal. The upgrade-resend result below was read at an unreleased head (6c7c90c), which changes the owed and cursor accounting, so it must be re-checked against the version that ships before 0.4.0. The proposal above is revised to the narrower identity it describes.
At that head, a completed install with its cursor at rest resends no historical messages on upgrade, provided 0.4.0 leaves the cursor and enumeration version alone. A conversation that does advance resends every selected message in it, unchanged siblings included, so the server's no-op comparison carries the load.
A hash of the text alone is not stable for an unchanged message.
extractFallbackstringifies unknown content (parsers.ts:270-276), so a key-order change alters the hash, and text past the 5000-character cut inextractContentParts(:425-433) is never seen. Attachment-only and tree changes do not reach the text at all. The natural hash point isextractMessageWithBlob(parsers.ts:629-655), which sees the text before escaping and before the inline/blob storage choice.Bumping the version does not repair conflicts already behind the cursor. That needs a separate, resumable messages repair scan. The append-only parent freeze at
index.ts:3081-3100and:3126-3133goes away with the change.The design includes 14 test specs; none has run. No build starts until the current release ships.
tnunamak commentedon Nov 4, 2026
Correction to the body: "We have not observed a changed re-emit in real data" is no longer true. One ordinary run of connector 0.3.10 on an instance with real history reported one
append_only_conflictonmessages. The host kept the first stored version, as described above. Which field changed is not known. So the defect is not only latent.