Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
9a28ba6
build(deps): bump tinystoragedrivers to v1.0.0
senamakel Oct 16, 2026
aeacfe1
chore(deps): bump tinystoragedrivers to v1.0.0
senamakel Oct 16, 2026
38b5d98
chore(drivers): remove unused checkpoint key and layout helpers
senamakel Oct 16, 2026
5616214
chore(drivers): remove unused checkpoint module
senamakel Oct 16, 2026
13df6db
chore(drivers): add state store module
senamakel Oct 16, 2026
96c1dba
feat(tinyflows-drivers): add driver crate
senamakel Oct 16, 2026
6e34a29
test(drivers): cover v2 layout and namespace encoding
senamakel Oct 16, 2026
919fe0f
feat(checkpoint): add checkpoint layout and kv import
senamakel Oct 16, 2026
59a36a2
feat(drivers): add state store and storage driver traits
senamakel Oct 16, 2026
95a0f2b
chore(drivers): remove unused catalog module
senamakel Oct 16, 2026
5a5731c
feat(catalog): add driver catalog definitions
senamakel Oct 16, 2026
c2cfd55
refactor(catalog): extract revision helpers into a dedicated module
senamakel Oct 16, 2026
87be784
chore(catalog): remove unused catalog modules
senamakel Oct 16, 2026
f5cc052
feat(catalog): add kv store driver
senamakel Oct 16, 2026
b3f2960
refactor(catalog): extract migration logic into its own module
senamakel Oct 16, 2026
5b07e6d
test(catalog): inline scoped storage helper in migrate tests
senamakel Oct 16, 2026
27840fa
test(catalog): add coverage for drafts, kv, revisions and runs
senamakel Oct 16, 2026
3549f65
feat(drivers): flow catalog and state on the storage schema v2
senamakel Oct 16, 2026
24715bf
refactor(schedule): simplify codec serialization
senamakel Oct 16, 2026
f3e134c
refactor(schedule): simplify codec serialization
senamakel Oct 16, 2026
a1759db
refactor(schedule): extract schedule driver into its own module
senamakel Oct 16, 2026
3b0f9c4
chore: I don't see a diff in your message — the "Diff:" section is em…
senamakel Oct 16, 2026
87472e3
chore(schedule): remove unused jobs module
senamakel Oct 16, 2026
5fa3a6c
feat(catalog): add driver catalog and id helpers
senamakel Oct 16, 2026
2c21244
feat(drivers): add schedule jobs driver
senamakel Oct 16, 2026
3765d2a
chore: I don't see a diff in your message — the "Diff:" section is em…
senamakel Oct 16, 2026
8786719
chore(schedule): remove unused claims module
senamakel Oct 16, 2026
7cde9b5
feat(schedule): add job scheduling policy and removal support
senamakel Oct 16, 2026
1558680
chore(schedule): add migration module scaffold
senamakel Oct 16, 2026
5db384d
feat(schedule): add admission control and import for schedules
senamakel Oct 16, 2026
81feeec
test(schedule): add unit tests for admission, codec, and sweep
senamakel Oct 16, 2026
0bf5848
test(schedule): add coverage for migration, sweep, and run queries
senamakel Oct 16, 2026
5eb9197
test(schedule): add coverage for admission, claims, jobs, migrate and…
senamakel Oct 16, 2026
a0422f6
feat(drivers): cron jobs and runs on the storage schema v2
senamakel Oct 16, 2026
9b0fcd5
docs(storage): clarify v2 schema ownership and drop store extension
senamakel Oct 16, 2026
19d9a47
chore: remove unused imports
senamakel Oct 16, 2026
e65ca51
test(import): canonicalize timestamps in fixture comparisons
senamakel Oct 16, 2026
828d093
fix(drivers): compose the flows schema without the v1 workflow store;…
senamakel Oct 16, 2026
27e7b45
feat(schedule): add admission control for schedule documents
senamakel Oct 16, 2026
78a8d0f
fix(schedule): match unsupported error kind variant
senamakel Oct 16, 2026
311ee70
chore(schedule): rename cron_jobs collection to cron.jobs
senamakel Oct 16, 2026
cbb87fa
fix(schedule): ensure unique creation timestamps for cron jobs
senamakel Oct 16, 2026
976a598
build(tinyflows-drivers): add tokio to schedule feature
senamakel Oct 16, 2026
f42e39f
fix(schedule): track job creation times from the store
senamakel Oct 16, 2026
a1c1157
test(module): update cron job collection name in artifact tests
senamakel Oct 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
151 changes: 146 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions crates/tinyflows-adaptive/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ thiserror = "2"
# `storage-drivers` only: the storage ports (no database client), named by git
# tag so a host that vendors tinystoragedrivers once can `[patch]` it; SHA-256
# for over-long document ids; tokio's `OnceCell` for one-time collection setup.
tinystoragedrivers-core = { git = "https://github.com/tinyhumansai/tinystoragedrivers", tag = "v0.11.0", optional = true }
tinystoragedrivers-core = { git = "https://github.com/tinyhumansai/tinystoragedrivers", tag = "v1.0.0", optional = true }
sha2 = { version = "0.11", optional = true }
tokio = { version = "1", features = ["sync"], optional = true }

Expand All @@ -37,7 +37,7 @@ tinyflows = { path = "../tinyflows", features = ["store", "mock"] }
# The engine's `WorkflowStore` on an in-memory storage backend, for tests and
# the eval example.
tinyflows-drivers = { path = "../tinyflows-drivers", default-features = false, features = ["store"] }
tinystoragedrivers-core = { git = "https://github.com/tinyhumansai/tinystoragedrivers", tag = "v0.11.0" }
tinystoragedrivers-core = { git = "https://github.com/tinyhumansai/tinystoragedrivers", tag = "v1.0.0" }
tokio = { version = "1", features = ["macros", "rt", "rt-multi-thread", "sync", "time"] }

[lints.rust]
Expand Down
4 changes: 2 additions & 2 deletions crates/tinyflows-drivers/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ catalog = [
# `tinyflows-schedule`, so a host that runs scheduled jobs but no flows can
# build with `default-features = false, features = ["schedule"]` and pull
# neither the engine nor its expression stack.
schedule = ["dep:tinyflows-schedule", "dep:anyhow", "dep:chrono", "dep:uuid"]
schedule = ["dep:tinyflows-schedule", "dep:anyhow", "dep:chrono", "dep:uuid", "dep:tokio"]
# `store`: `DriverWorkflowStore`, the engine's `WorkflowStore` on documents (saved
# workflows, revisions, runs, the notes journal, proposals). Its synchronous
# trait runs on the storage crates' `Blocking` bridge.
Expand All @@ -53,7 +53,7 @@ store = [
test-fixtures = ["catalog"]

[dependencies]
tinystoragedrivers-core = { git = "https://github.com/tinyhumansai/tinystoragedrivers", tag = "v0.11.0" }
tinystoragedrivers-core = { git = "https://github.com/tinyhumansai/tinystoragedrivers", tag = "v1.0.0" }
serde = { workspace = true }
serde_json = { workspace = true }
tracing = { workspace = true }
Expand Down
33 changes: 22 additions & 11 deletions crates/tinyflows-drivers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ of `tinyflows` itself.

| Type | Implements | Layout |
| --- | --- | --- |
| `DriverStateStore` | `tinyflows::caps::StateStore` | one document per key in `flows_state` |
| `DriverCheckpointer<State>` | `tinyflows::graph::Checkpointer<State>` | `flows_graph_checkpoints`, `_threads`, `_writes` |
| `schedule::CronDocuments` | the cron job and run store | `cron_jobs`, `cron_runs`, `cron_counters` |
| `catalog::FlowCatalogDocuments` | saved flows, revisions, runs, suggestions, drafts | `flows_*` |
| `DriverStateStore` | `tinyflows::caps::StateStore` | one `tsd.kv` entry per key (owner `flows`, namespace `state`) |
| `DriverCheckpointer<State>` | `tinyflows::graph::Checkpointer<State>` | `graph.checkpoints`, `graph.threads`, `graph.writes` (namespace `flows`; declared by `tinyagents-graph`) |
| `schedule::CronDocuments` | the cron job and run store | `cron.jobs`, `cron.runs`, `cron.admission`; run numbers in `tsd.counters`, claims in `tsd.leases` |
| `catalog::FlowCatalogDocuments` | saved flows, revisions, runs, suggestions, drafts | `flows.*` |
| `store::DriverWorkflowStore` | `tinyflows::store::WorkflowStore` | `wf_*` |

```rust
Expand All @@ -42,6 +42,17 @@ A host that runs scheduled jobs but no flows builds with
`default-features = false, features = ["schedule"]` and pulls neither the
engine nor its expression stack.

## Storage schema v2

Everything here stores on the uniform schema of `tinystoragedrivers` v1.0.0.
`storage_schema()` (owner `flows`) and `cron_storage_schema()` (owner `cron`)
declare the collections and carry the v1 -> v2 steps; the host composes them
with `Registry::compose` and runs `Registry::migrate` per scope, then calls
`import_legacy_kv` and `import_legacy_cron_counters` once per scope (a document
step cannot write `tsd.*`). The graph checkpointer reads and writes the
`graph.*` documents of `tinyagents-graph` under namespace `flows`. The
library-only `store::DriverWorkflowStore` (`wf_*`) is still on its v1 names.

## The flow catalog (`catalog`)

`FlowCatalogDocuments` is the catalog and the authoring drafts: one `async fn`
Expand All @@ -60,12 +71,12 @@ let state = Arc::new(FlowStateDocuments::new(catalog.clone(), flow.id.clone()));

| Collection | One document per |
| --- | --- |
| `flows_definitions` | flow |
| `flows_revisions` | superseded graph (newest 20 kept) |
| `flows_runs` | run, with its steps |
| `flows_kv` | `(namespace, key)` of flow state |
| `flows_suggestions` | discovery suggestion |
| `flows_drafts` | authoring draft |
| `flows.definitions` | flow |
| `flows.revisions` | superseded graph (newest 20 kept) |
| `flows.runs` | run, with its steps |
| `tsd.kv` (owner `flows`, namespace `kv:<namespace>`) | `(namespace, key)` of flow state |
| `flows.suggestions` | discovery suggestion |
| `flows.drafts` | authoring draft |

- Every guarded SQL `UPDATE … WHERE status = …` (finish, resume, interrupt,
TTL expiry) is a compare-and-swap on the run, so a transition happens at
Expand Down Expand Up @@ -96,7 +107,7 @@ let state = Arc::new(FlowStateDocuments::new(catalog.clone(), flow.id.clone()));
- Graphs, steps, drafts and state values are JSON strings, optional fields
are omitted rather than `null`, and ordering uses epoch-nanosecond fields.

`FlowStateDocuments` is one flow's namespaced state over the same `flows_kv`
`FlowStateDocuments` is one flow's namespaced state over the same `tsd.kv`
documents as `kv_get` / `kv_set` / `kv_delete`. It is the engine's
`StateStore` and the host's synchronous `DedupKv`; the latter runs on the
storage crates' `Blocking` bridge (one per process, or the host's own via
Expand Down
Loading
Loading