Skip to content

[memory→tinymemory] tinymemory-tools / integrations: run slots, migration service + claim, Backfiller, Importer::resume_plan #290

Closed as not planned

Description

@senamakel

Blocked by #280.

The drivers are upstream already (#266). The generic glue around them still lives in the host:

  • run_slots: a RunSlots (a RUNNING set plus an RAII guard) and a start-gate trait. It replaces four hand-rolled copies in backfill.rs, import.rs, layout_migration/service.rs, and the import/migration START_GATE interlock.
  • layout_migration::service::MigrationService: the run-slot registry, RunGuard, the tick policy (finished → paused → running → start) and the spawn-and-record-failure wrapper. From layout_migration/service.rs (154).
  • layout_migration::claim: a default LegacyClaim (hard-link marker, digest of the normalised endpoint, normalize) behind an fs-claim feature. From layout_migration/claim.rs (143).
  • backfill::Backfiller:
    • Takes from backfill.rs (359): ThreadPlan / plan(), the last_batch visibility wait, the accepted-then-visible write policy, the stale-Running→Error fix and BackfillState.
    • Ports: ThreadSource (threads and messages) and CounterStore.
    • Derive the stored counts from the engine (list by BACKFILL_TAG and thread) where it can, and fall back to the store only where it can't.
  • Importer::resume_plan(file, billing_ok, paused):
    • Takes from import.rs (389): the resume state machine (resumable = Running, or Error with paused_for_credits and a billing allow).
    • Also adds a store-only record_resume_failure. The host branch that writes ImportFile by hand duplicates Importer.
    • TokioPacer is a trivial seam implementation. Keep it in the host.

Done when: the new API is merged with its tests (moved host tests travel with the code), cargo test --workspace and cargo clippy --workspace --all-targets -- -D warnings pass, and a tag is cut so OpenHuman can bump the gitlink. tinymemory-tools stays tokio-free (spawn/clock are traits); schemars only behind an optional schema feature.

Agent brief

  • Files to consider: host memory/{backfill,import}.rs, memory/layout_migration/{claim,service}.rs; upstream tinymemory-tools/src/{backfill,layout_migration}/, tinymemory-integrations/src/import/
  • Research: the gaps listed above
  • Workflow: an implementing-an-epic step; follow the migrate-to-submodule skill (upstream PR → tag → OpenHuman gitlink bump + delete host copy).
  • Subagent: tdd-implementer.

Parent: tinyhumansai/openhuman#8000

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority: p3Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.rustPull requests that update rust code

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions