Repository navigation
feat(storage): adaptive and workflow stores on the v2 schema - #16
Merged
senamakel merged 28 commits intoOct 17, 2026
Merged
Conversation
Cover encoding and decoding of stored values to guard against regressions in the persistence codec. Auto-committed-on: dragonfly
Reworked the codec helpers to reduce duplication in the encode and decode paths. Behaviour is unchanged. Auto-committed-on: dragonfly
Add tests covering the driver store migration path to verify schema upgrades apply cleanly and existing data is preserved. Auto-committed-on: dragonfly
Add tests covering the driver store migration path, verifying that existing data is preserved and the schema is brought up to date. This locks in the migration behaviour so future schema changes can be made with confidence. Auto-committed-on: dragonfly
The store module in tinyflows-drivers is no longer referenced anywhere, so it has been deleted to keep the crate free of dead code. Auto-committed-on: dragonfly
Split the schema migration logic into smaller helper functions so each migration step is easier to read and test. Behaviour is unchanged. Auto-committed-on: dragonfly
Adds a migration routine to the driver store so existing on-disk data can be brought up to the current schema. This keeps older stores usable after the schema changes instead of requiring a manual reset. Auto-committed-on: dragonfly
Split the definition store logic into smaller helper functions to make the read and write paths easier to follow. Behaviour is unchanged. Auto-committed-on: dragonfly
The history store module was no longer referenced by any driver, so it has been deleted to keep the crate free of dead code. Auto-committed-on: dragonfly
Adds store helpers for persisting proposals and runs so the driver layer can record and retrieve workflow state. This gives the runtime a place to keep proposal and run data without duplicating storage logic in callers. Auto-committed-on: dragonfly
Add shared test helpers for the driver store so tests can build and inspect store state without duplicating setup code. Auto-committed-on: dragonfly
Note writes were being applied to the in-memory copy only, so edits were lost on the next read. The updated note is now written back to the store. Auto-committed-on: dragonfly
The lock guard now releases the underlying lock when it is dropped, so locks are freed even when a caller returns early or panics instead of releasing them explicitly. Auto-committed-on: dragonfly
Introduce an import path in the storage driver so stored data can be loaded back in. This wires the new module into the storage layer and exposes the entry point callers need. Auto-committed-on: dragonfly
wf_* becomes flows.workflows, flows.workflow_{revisions,runs,notes,proposals,quarantine}
with snake_case object bodies and *_ms times, v1 -> v2 steps, and the decision lock
on tsd.leases.
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The migration tests were extracted from the driver file into their own module so the driver stays focused on runtime behaviour. No test coverage or assertions were changed. Auto-committed-on: dragonfly
Adds tests covering the conversion of drivers during migration, verifying that existing driver state is carried over correctly. This guards against regressions in the migration path. Auto-committed-on: dragonfly
Moved the migration driver out of the drivers module into a dedicated file so the adaptive crate's driver layout stays consistent as more drivers are added. No behaviour change. Auto-committed-on: dragonfly
The drivers module was no longer referenced anywhere in the crate, so it has been dropped to keep the source tree free of dead code. Auto-committed-on: dragonfly
Introduce a ledger driver under the adaptive drivers tree and wire it into the drivers module so it can be registered alongside the existing drivers. Auto-committed-on: dragonfly
Reworked the ledger driver trait implementation to reduce duplication and clarify the control flow. Behaviour is unchanged. Auto-committed-on: dragonfly
Introduce ledger, vault, and migration drivers for the adaptive crate, along with their trait implementations and test coverage. This extends the driver layer so adaptive workflows can persist and migrate state through the new backends. Auto-committed-on: dragonfly
Reformat the adaptive storage drivers and their tests with rustfmt, wrapping long call chains and reordering imports. No behaviour changes. Auto-committed-on: dragonfly
Adds an in-memory implementation of the driver store trait, giving tests and local runs a store backend that needs no external dependencies. Auto-committed-on: dragonfly
…a v2 adaptive_* becomes flows.adaptive_* with scope_key as a Key::of part, snake_case flat bodies and *_ms times; id sequences move to tsd.counters with an importer for the v1 sequences; v1 -> v2 steps and extend_storage_schema for the shared flows owner. Co-authored-by: Medulla <medulla@tinyhumans.ai>
Delete the trait implementation in the store module since it is no longer referenced anywhere. Auto-committed-on: dragonfly
Narrow the store module's imports to only the types it actually uses, dropping the re-exports that were no longer referenced. Auto-committed-on: dragonfly
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the two library-only stores deferred by #15 onto storage schema v2. Reference: tinyhumansai/openhuman#8058 (epic #7973).
What changed
tinyflows_drivers::storage_schema()(ownerflows, declared once) now also declares the workflow store (featurestore) with its steps.tinyflows_adaptive::drivers::extend_storage_schema(schema)addsflows.adaptive_*and its steps to the same owner; the host passes one schema through both.import_legacy_adaptive_countersis the host importer fortsd.counters.*_msinstants, records as objects (the v1 JSON strings are parsed into the body). Wire/model types are unchanged; the conversion is a separate storage codec (store/codec.rs,drivers/convert.rs).tsd.leaseslease (ownerflows, keyworkflow_decision/<id>); adaptive id sequences aretsd.counters(adaptive_rows,adaptive_lessons).save_proposal_if_fingerprintfences the definition withatomic_batch_checked+ReadCheck(capabilityValidatedBatch) instead of rewriting it, since a batched put declares no body version and would reset the definition's recorded schema. Consequence: the proposal put in that batch carries schema 0 (undeclared) until rewritten.store/mod.rstrait impl moved tostore/trait_impl.rs(500-line rule).sha2dropped from adaptive'sstorage-drivers;chrono/tracingadded there.v1 -> v2 mapping
wf_definitionsflows.workflows(Key::of([id]))wf_revisionsflows.workflow_revisions(workflow_id,superseded_ms, workflow fields flat)wf_runsflows.workflow_runs(snake_case,started_ms/finished_ms,executor.process_started_ms)wf_notesflows.workflow_notes(noteslist)wf_proposalsflows.workflow_proposalswf_quarantineflows.workflow_quarantine(verbatim)wf_lockstsd.leases(not carried: 2 minute lease)adaptive_rows,_lessons,_evidence,_scores,_variants,_episodes,_steps,_workflowsflows.adaptive_*(scope_keyaKey::ofpart where per bucket and a body field for queries;at/started_at/updated_at->*_ms)adaptive_counterstsd.countersownerflowsviaimport_legacy_adaptive_countersThe
workflow_prefix is used becauseWorkflowRecord/RunRecorddiffer in shape from the catalog'sFlow/run (flows.definitions,flows.runs).Steps added
workflows_v2,workflow_revisions_v2,workflow_runs_v2,workflow_notes_v2,workflow_proposals_v2,workflow_quarantine_v2,adaptive_{rows,lessons,evidence,scores,variants,episodes,steps,workflows}_v2. An unparseable v1 record is carried verbatim, not dropped. All tested with v1-shaped fixtures and read back through the stores.Gates
cargo fmt --all -- --check: okcargo clippy -p tinyflows-drivers -p tinyflows-adaptive -p tinyflows-import --all-targets --all-features -- -D warnings: okcargo test -p tinyflows-drivers -p tinyflows-adaptive -p tinyflows-import --all-features: all pass (drivers 361, adaptive 280 lib + integration)cargo clippy -p tinyflows-driversalone (without adaptive's feature unification) reports a dead-code error incrates/tinyflows/src/engine/api.rs(run_preflight_with_observer); untouched file, pre-existing.