Skip to content

[memory→tinymemory] tinymemory-tools / integrations: run slots, migration service + claim, Backfiller, Importer::resume_plan #11

Closed

Description

@senamakel

Moved from tinyhumansai#290, which is closed in favour of this issue. Part of tinyhumansai/openhuman#8000.

Blocked by #1.

The drivers are upstream already (tinyhumansai#266). The generic glue around them still lives in the host:

  • run_slots: a RunSlots (a RUNNING set plus an RAII guard) and a start-gate trait. It replaces four hand-rolled copies in backfill.rs, import.rs, layout_migration/service.rs, and the import/migration START_GATE interlock.
  • layout_migration::service::MigrationService: the run-slot registry, RunGuard, the tick policy (finished → paused → running → start) and the spawn-and-record-failure wrapper. From layout_migration/service.rs (154).
  • layout_migration::claim: a default LegacyClaim (hard-link marker, digest of the normalised endpoint, normalize) behind an fs-claim feature. From layout_migration/claim.rs (143).
  • backfill::Backfiller:
    • Takes from backfill.rs (359): ThreadPlan / plan(), the last_batch visibility wait, the accepted-then-visible write policy, the stale-Running→Error fix and BackfillState.
    • Ports: ThreadSource (threads and messages) and CounterStore.
    • Derive the stored counts from the engine (list by BACKFILL_TAG and thread) where it can, and fall back to the store only where it can't.
  • Importer::resume_plan(file, billing_ok, paused):
    • Takes from import.rs (389): the resume state machine (resumable = Running, or Error with paused_for_credits and a billing allow).
    • Also adds a store-only record_resume_failure. The host branch that writes ImportFile by hand duplicates Importer.
    • TokioPacer is a trivial seam implementation. Keep it in the host.

Done when: the new API is merged with its tests (moved host tests travel with the code), cargo test --workspace and cargo clippy --workspace --all-targets -- -D warnings pass, and a tag is cut so OpenHuman can bump the gitlink. tinymemory-tools stays tokio-free (spawn/clock are traits); schemars only behind an optional schema feature.

Agent brief

  • Files to consider: host memory/{backfill,import}.rs, memory/layout_migration/{claim,service}.rs; upstream tinymemory-tools/src/{backfill,layout_migration}/, tinymemory-integrations/src/import/
  • Research: the gaps listed above
  • Workflow: an implementing-an-epic step; follow the migrate-to-submodule skill (upstream PR → tag → OpenHuman gitlink bump + delete host copy).
  • Subagent: tdd-implementer.

Parent: tinyhumansai/openhuman#8000

Activity

  1. added
    enhancementNew feature or request
    rustPull requests that update rust code
    priority: p3Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.
    on Oct 16, 2026
  2. senamakel commented on Oct 18, 2026

    @senamakel
    OwnerAuthor

    Implementation is ready in #25 at 3f17272d460e2e0d6cada34b333eb941ccbe39ad. Required local workspace gates pass, with 93.10% line coverage using the exact CI exclusions.

    Hosted CI finished with 32 successful checks and two failed CortexDB live checks. Both failed before tests because cortexdb/cortexdb:v0.10.4 cannot be pulled (manifest unknown). This is the same external artifact blocker recorded in #6. The older official cached v0.9.9 image was already tested there and fails the required cached-pack erasure contract, so it is not a replacement.

    Keep this issue and PR open until a compatible image or registry access is available and every hosted check passes on the final head. No tag or merge has been made, and no test expectation or image pin was weakened. Independent host tinyhumansai#88 work continues.

  3. senamakel commented on Oct 18, 2026

    @senamakel
    OwnerAuthor

    Merged in #25 (976883d). Annotated adoption tag epic-8000-11 points to this merged commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority: p3Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect.rustPull requests that update rust code

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions