Skip to content

storage: migrate artifact metadata to SQLite - #1694

Merged
likun666661 merged 1 commit into
apache:mainfrom
likun666661:agent/artifact-metadata-sqlite
Jul 31, 2026
Merged

likun666661 merged 1 commit into
apache:mainfrom
likun666661:agent/artifact-metadata-sqlite

Conversation

@likun666661

Copy link
Copy Markdown
Member

Summary

  • add the artifact schema and metadata repository to the shared runtime.sqlite operational authority
  • cut over legacy artifacts/metadata.jsonl once under the artifact writer lock, with fingerprinted journal validation and fail-closed old-writer detection
  • keep payload bytes as files while reusing the durable staging/link and purge-intent recovery protocol against SQLite metadata transactions
  • wire CLI, Desktop, Runtime Host, and Headless artifact writers to the SQLite-backed store and close their database leases
  • teach Harbor trajectory image collection to read artifact_records from runtime.sqlite, with legacy JSONL fallback

Crash and migration guarantees

  • interrupted legacy import rolls back and resumes idempotently
  • payload linked without a committed metadata row is removed during recovery
  • committed metadata preserves the payload and removes publication staging residue
  • interrupted purge completes payload and SQLite metadata cleanup from the durable purge intent
  • a changed legacy JSONL source after completed cutover fails closed
  • a new workspace does not create metadata.jsonl

Verification

  • npm --workspace @maka/storage test — 849 passed, 1 skipped
  • npm --workspace @maka/runtime-host test — 414 passed
  • Storage, Runtime Host, Headless, CLI typechecks
  • Desktop main build
  • Biome checks and Python py_compile
  • direct SQLite trajectory metadata smoke

The full local Harbor adapter file also encountered two unrelated environment-sensitive Python 3.9 failures in the Kimi/Codex adapter tests; the modified trajectory path was validated separately.

Part of #1649. The selected-session bundle / backup relational-closure work remains the next planned PR (Stage 5).

@likun666661
likun666661 force-pushed the agent/artifact-metadata-sqlite branch from 39177e9 to 2d4ce8b Compare July 31, 2026 12:37
@likun666661
likun666661 marked this pull request as ready for review July 31, 2026 12:37
@likun666661
likun666661 merged commit 7ce84bb into apache:main Jul 31, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant