diff --git a/AGENTS.md b/AGENTS.md index 65d26def724..506e6b9bd2a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -783,6 +783,7 @@ Two columns because there are two sets (see above): **Contrib** is `[features] d | `voice` | OFF | ON | the `openhuman::voice` family (incl. `voice::audio_toolkit`) — STT/TTS providers, dictation server, always-on listening, podcast audio + email | `hound`, `lettre` | | `inference` | OFF | ON | the `cpal` audio-device stack: microphone capture for voice, plus `desktop::accessibility::permissions`' mic-permission probe. Implied by `voice`. Off ⇒ the probe reports `Unknown`. **The name is historical** — it used to gate the bundled whisper.cpp STT engine, which no longer exists (see the scope note below); do not rename it, it is forwarded by name from the shell manifest and asserted by `INFERENCE_COMPILED_IN` | `cpal` | | `web3` | OFF | ON | the `openhuman::web3` family (`web3`, `web3::wallet`, `web3::x402`) — crypto wallet (multi-chain sign/broadcast), swaps/bridges/dapp calls, x402 machine payments | `bitcoin`, `curve25519-dalek` | +| `memory` | ON | ON | the `openhuman::memory` family — recall/ingest/tree/goals/sources/sync, its ~13 RPC namespaces, the ~30 memory agent tools, the `MemoryGuard` policy decorator, the driver binding, and **the TinyMemory module the binding loads**. Implied by `memory-git`. Off ⇒ the `memory*` controllers are unregistered, every memory agent tool is absent rather than degraded, the memory subscribers are not installed, and no memory `cdylib` is downloaded or `dlopen`ed. Recall inside a turn returns nothing — the behaviour `NullMemoryProvider` already produces at runtime | **none.** `tinymemory-api` is the contract and stays; every crate the family touches (`rusqlite`, `reqwest`, `serde`) has other parents. Surface-only, like `media` — do not "fix" the empty list | | `media` | ON | ON | `openhuman::media::generation` (the `media_generate_*` agent tools) + `openhuman::media::image` scaffold | none (surface-only) | | `documents` | OFF | ON | the `generate_document` / `generate_presentation` agent tools and PDF text extraction during multimodal ingest. **The synthesis is not in this build** — all three run in the `tinydocs` TinyBus module (see below), so this gate turns on the tools and the host policy around them: the artifact pipeline, the deadlines, image resolution under the security policy. The dependency is `tinydocs-bus`, the wire contract crate, and nothing else from that repository. Implies `modules`. Off ⇒ both tools absent from the tool list rather than degraded, and PDF ingest degrades a file to a reference instead of extracted text | **39 crates**, and they leave `Cargo.lock` entirely: `docx-rs`, `ppt-rs`, `pdf-extract` plus `lopdf`, `syntect`, `pulldown-cmark`, `xml-rs`, `quick-xml`, `zip 0.6`, `zstd`, `bzip2`, `encoding_rs`, `euclid`, `ttf-parser`, the CFF/Type1/CMap parsers, … Product profile 505 → 448 names | | `modules` | ON | ON | `openhuman::modules` — the dynamic module host: the loader that admits a compiled `cdylib` through tinybus's ABI descriptor, manifest, dependency and SHA-256 gates, the compiled-in registry of modules this build trusts, and the `modules` RPC namespace. Implied by `documents`. Off ⇒ `modules.*` is unknown-method and nothing can load a native module | none in the product profile (`ureq`, `flate2`, `tar`, `zip 2`, `tempfile`, `toml` are already there) — **but see the kernel-floor note**: this feature exists so `tinybus/modules` is not enabled on the dependency itself, which would put a `dlopen` loader into the kernel profile where `tinybus` is always-on | @@ -1051,6 +1052,75 @@ The general shape of the warning still holds for *other* pairs: two crates with near-identical types are a real hazard, and a "free carve-out" is only free once the compiler says so. Probe before assuming, in either direction. +#### The `memory` gate + +Facade + stub in the `voice` shape, with `skills`' type carve-out taken further +than anywhere else in the tree — **six** carve-outs rather than one. That is the +interesting part, and the part to copy. + +**`pub mod memory;` is always compiled.** The behavioural submodules are +`#[cfg(feature = "memory")]` and `src/openhuman/memory/stub.rs` covers the +disabled build. What stays compiled in **both** directions, with exactly one +definition each and no stub twin: + +| Ungated | What | Why it is not behaviour | +| --- | --- | --- | +| `memory::api` | the `tinymemory-api` re-export | contract vocabulary in always-on agent-harness signatures | +| `memory::host` | `MemoryHostConfig for Config` + the bus event sink | impls of API-crate traits on OpenHuman's own `Config`. ~20 always-compiled callers pass `&Config` where `dyn MemoryHostConfig` is expected; gating it makes `Config` stop satisfying a bound unrelated to the family | +| `memory::ranking` | `hybrid_score`, `WeightProfile`, `cosine_similarity`, `mmr_select` | arithmetic over values the caller already holds. Both search tools fetch through the guard *first*, so routing this back over the bus buys a serialization hop and nothing else | +| `memory::citation` | `MemoryCitation`, `CROSS_CHAT_HEADER` | inert serde type in the agent's `last_turn_citations` field and the web-chat delivery path | +| `memory::namespaces` | `CONVERSATION_RAW_NAMESPACE`, `REFLECTIONS_NAMESPACE` | gated producer, always-compiled reader; a `&'static str` cannot fail without a driver | +| `flows::namespace` | `flow_namespace`, `FLOW_MEMORY_NAMESPACE_PREFIX` | a `format!`; `flows_delete` and the run-digest subscriber need it with memory off | + +**The rule these six share, and the reason to prefer it over a stub:** a stub +twin of an inert type is a *second definition*, free to drift, and the disabled +build cannot catch drift in a type it also defines. Carve the type out ungated; +stub only behaviour. `flow_namespace` carries a stated security invariant — +"the only place in the codebase that may construct this namespace string" — +and the move **kept** that true precisely because there is one definition, not +two. + +**Stubs answer in two shapes, deliberately.** Registration sites want +*absence*: the `all_*_registered_controllers` aggregators return empty vectors, +so `core/all.rs` needs no `#[cfg]` for them, and `memory.*` becomes +unknown-method rather than a known method that fails. Write paths return a +build-fact error, because silently discarding a write is the one failure mode +that looks like success. + +**Three mis-housings were retargeted, not stubbed** — the same move the +`channels` gate made for its three `tinychannels` imports. The Composio +provider catalogue, the conversation store `threads` uses, and the +`rpc_models` request types were always-compiled code reaching the engine +*through* a `memory::…` shim; they name `tinymemory_core` directly now. That +removed ~37 always-on → gated edges with no stub at all. **Watch the trap:** +those shims re-export the engine *and* define host code beside it, so a blanket +path substitution also catches the host half — +`register_conversation_persistence_subscriber` and +`slack::all_slack_memory_registered_controllers` both went to the engine crate, +where they do not exist. List what a shim adds beyond its glob before +retargeting it. + +**CLI arms are kept, not deleted.** `openhuman memory` and `tree-summarizer` +answer with the gate name; falling through to generic namespace resolution +would say "unknown namespace: memory", which reads like a typo rather than a +build fact. Same reasoning as the `mcp` and `tui` arms. + +**`CoreContext::current_memory_capabilities` answers CLOSED with the gate off** +— the one place it does. The open default exists because a capability set is +unknowable until a driver answers; with the family compiled out the state is +not "unknown", it is "absent". + +**Sheds no third-party crate.** `tinymemory-api` is the contract and stays, and +every crate the family touches (`rusqlite`, `reqwest`, `serde`) has other +parents. Surface-only, like `media` — the `memory = []` list is intentional, do +not "fix" it with `dep:` entries. Implied by `memory-git`. + +**Errors surface in waves.** Name resolution first, then trait bounds — the +`MemoryHostConfig` wave only appeared once resolution was clean, so a run that +looks nearly done can still be hiding a structural problem. Run +`cargo check --no-default-features --features "media,skills,flows,mcp,channels,medulla,http-server,scheduler-gate,file-logging,modules"` +(that is `default` minus `memory`) before pushing anything touching this surface. + #### The `tui` gate The tabbed terminal UI (`openhuman`, or explicitly `openhuman tui` / alias `chat`) lives in `src/openhuman/tui/` and follows the **`mcp`/`voice` facade+stub** pattern: `pub mod tui;` is always compiled; the behavioural submodules (`app`, `render`, `state`, `terminal`, `runner`) are `#[cfg(feature = "tui")]`; and `#[cfg(not(feature = "tui"))] mod stub;` re-exposes the one symbol an always-compiled caller reaches — `run_from_cli` — with a build-fact error body (`"tui feature disabled at compile time … --features tui"`). Bare-command auto-launch requires terminal stdin/stdout and `HostKind::Cli`; Docker, CI, pipes, and `--no-tui` retain the non-TUI CLI path. diff --git a/Cargo.toml b/Cargo.toml index 51df265ae11..68119792131 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -757,7 +757,7 @@ proptest = "1" # (`modules`: Contrib=ON, Product=ON) and with `scripts/ci/product-features.txt`, # which already lists it. This does NOT move the kernel floor: that profile is # `--no-default-features --features flows` and never reads this list. -default = ["media", "skills", "flows", "mcp", "channels", "medulla", "http-server", "scheduler-gate", "file-logging", "modules"] +default = ["media", "memory", "skills", "flows", "mcp", "channels", "medulla", "http-server", "scheduler-gate", "file-logging", "modules"] # HTTP + Socket.IO server transport (#5048): the `/rpc` JSON-RPC endpoint and # its auth middleware/CORS layer (`core::jsonrpc`, `core::auth`), the `/v1` # OpenAI-compatible router (`inference::http`), the ad-hoc static-dir file @@ -804,7 +804,7 @@ default = ["media", "skills", "flows", "mcp", "channels", "medulla", "http-serve # `None`; and the three `ops` entry points always-on code calls return a # build-fact error, so a post-sync snapshot or a subconscious diff is logged and # skipped rather than silently reported as "nothing changed". -memory-git = ["tinycortex/git-diff", "tinycortex/wiki-git", "tinymemory-core/memory-git"] +memory-git = ["memory", "tinycortex/git-diff", "tinycortex/wiki-git", "tinymemory-core/memory-git"] http-server = ["dep:axum", "dep:socketioxide"] # Local audio-device access: the `cpal` capture stack behind voice recording # and the accessibility microphone-permission probe. Default-ON. Slim / @@ -1005,6 +1005,45 @@ contacts = ["tinymemory-core/contacts"] # controllers / stores / subscribers tagged `Media` (agent tools only), and # `openhuman::image` is currently unwired scaffold (added #2997). media = [] + +# The `openhuman::memory` family: the recall/ingest/tree/goals/sources/sync +# domains, their ~13 RPC namespaces, the ~30 memory agent tools, the guard, the +# driver binding, and the TinyMemory module the binding loads. Default-ON, +# product-ON — this is a core product surface, and the gate exists for the +# embedding host that wants a workflow or tool runtime with no memory at all +# (`DomainSet::kernel()` already models that at runtime; this is the +# compile-time axis it composes with). +# +# FACADE + STUB, not a leaf gate. `pub mod memory;` is always compiled: ~230 +# production references across ~101 files outside the family reach into it, and +# a handful of always-on callers (the agent harness, flows, channels, cron) +# name concrete items rather than a registration aggregator. The behavioural +# submodules are `#[cfg(feature = "memory")]` and +# `src/openhuman/memory/stub.rs` re-exposes that surface with null / disabled +# bodies. Same shape as `voice`, with `skills`' refinement: inert types stay +# ungated so both builds share ONE definition. +# +# TYPE CARVE-OUT: `memory::api` (a re-export of `tinymemory-api`) and +# `memory::ranking` are compiled in BOTH builds. The first is the contract +# vocabulary — `MemoryEntry`, `RecallOpts`, `MemoryCategory` appear in +# always-on agent-harness signatures — and the second is host-side arithmetic +# with no driver behind it. Neither needs a stub twin, and a twin is exactly +# what would be free to drift. +# +# What OFF means: the `memory*` controllers are unregistered (unknown-method +# over `/rpc`, absent from `/schema`), every memory agent tool is ABSENT from +# the tool list rather than degraded to an error, the memory subscribers are +# not installed, and — the point of the gate for an embedder — the TinyMemory +# module is neither pinned nor loaded, so no `cdylib` is downloaded or +# `dlopen`ed. Recall inside an agent turn returns nothing, which is the +# behaviour `NullMemoryProvider` already produces at runtime. +# +# Sheds no third-party crate on its own: `tinymemory-api` is the contract and +# stays, and every crate the family touches (rusqlite, reqwest, serde) has +# other parents. The gate buys the tool surface, the prompt bloat, the RPC +# surface and the module download — not binary size. Do not "fix" the empty +# list by adding `dep:` entries. +memory = [] # Flows domains: the `flows::` automation surface (saved tinyflows graphs — # create/run/schedule + the workflow_builder / flow_discovery agents), the # `tinyflows::` adapter seam, and the `rhai_workflows::` language-workflow tool. diff --git a/app/src-tauri/Cargo.toml b/app/src-tauri/Cargo.toml index a17c81d1b0c..4a90b5a2156 100644 --- a/app/src-tauri/Cargo.toml +++ b/app/src-tauri/Cargo.toml @@ -163,6 +163,10 @@ uuid = { version = "1", features = ["v4"] } openhuman_core = { path = "../..", package = "openhuman", default-features = false, features = [ "channels", "media", + # The memory family and the TinyMemory module it loads. Forwarded because + # the shell sets `default-features = false`, so a gate absent from this + # list is compiled out of the shipped app silently (#4901). + "memory", "inference", "voice", "web3", diff --git a/docs/specs/2026-08-23-memory-bus-only-remaining-surface.md b/docs/specs/2026-08-23-memory-bus-only-remaining-surface.md new file mode 100644 index 00000000000..43e5b97d791 --- /dev/null +++ b/docs/specs/2026-08-23-memory-bus-only-remaining-surface.md @@ -0,0 +1,296 @@ +# Retiring `tinycortex` from OpenHuman — the remaining surface + +*Status: inventory, 2026-08-23. Companion to +[`2026-08-13-memory-module-port.md`](2026-08-13-memory-module-port.md).* + +## What this document is for + +The goal is that OpenHuman links **only `tinymemory-api`** and reaches memory +over the TinyMemory TinyBus module, so `tinycortex` and `tinymemory-core` both +leave the build and `vendor/tinycortex` stops being a submodule of this repo. + +Two rounds of that work have landed. This document is the honest remainder: what +is left, why each item is left, and which repository has to move first. It exists +because the previous round's note — recorded in +`memory/direct_engine_refs_tests.rs` — concluded that *"what blocks the migration +is release lag, not seam width"*. **That was true of the five families v1.2.0 +shipped and is not true in general.** For the rest, the seam genuinely does not +exist, and re-deriving that finding costs a day. + +## Where the number stands + +Measured on this branch, production files only (`*_tests.rs`, `tests.rs`, +`test_support/` and `src/bin/` excluded): + +| | files | references | +| --- | ---: | ---: | +| name `tinymemory_core::` | 98 | ~360 | +| name `tinycortex::` / `tinycortex_api::` directly | 41 | ~130 | + +The two overlap: `tinymemory_core::tinycortex::…` is a re-export, so a file can +appear in both. + +## The rule that decides where each item goes + +A capability belongs **on the bus** when it needs the store. It belongs **in the +host** when it is arithmetic, policy, or vocabulary the caller already holds. +Getting this backwards in either direction is expensive: + +- Routing host arithmetic over the bus buys a serialization hop and nothing else. +- Leaving store access host-side keeps a second, unpoliced door into the + subsystem — the argument `direct_engine_refs_tests` opens with. + +Two items have already moved the host way rather than the bus way, and they are +the pattern to copy: `util::redact` (a six-line SHA-256 log redactor) and +`memory::ranking` (`hybrid_score`, `WeightProfile`, `cosine_similarity`, +`mmr_select`). Both are ported verbatim with parity tests pinned to **literal** +expected values, never to a call back into the engine — a test written the other +way stops compiling on exactly the day it would matter. + +## A. Needs new bus surface — upstream `tinymemory` first + +Each of these is a `tinymemory` PR, then a release, then a `modules::registry` +re-pin, and only then a host change. Adding the trait method alone produces a +driver that answers `Unsupported`: the failure moves from compile time to run +time, which is strictly worse than the direct call it replaced. + +### A1. `MemoryQueue` — the ingest job queue (~36 refs) + +The largest single blocker, and the one that gates the ingest RPC surface. +Methods in use: `enqueue`, `store::count_by_status`, +`store::count_failed_unrecoverable`, `store::requeue_failed`, `drain_until_idle`, +`wake_workers`, `backfill_in_progress`, `ensure_reembed_backfill`, +`requeue_failed_after_provider_change`, `start`, plus the `NewJob` / `JobStatus` +/ `ExtractChunkPayload` / `FlushStalePayload` payload types. + +Production callers: `memory/tree/tree/rpc.rs`, `memory/read_rpc/admin.rs`, +`memory/sync_events_bridge.rs`, `security/credentials/ops.rs`, +`config/ops/model.rs`, `config/migrations/migrate_legacy_embedding_provider.rs`, +`inference/embeddings/rpc.rs`, `core/runtime/services.rs`. + +Note the shape: several callers *enqueue* work and then *poll* for drain. A +family that only enqueues leaves the polling half behind. + +### A2. `MemoryConversations` — threads and messages (~25 refs) + +`get_messages`, `list_threads`, `purge_threads`, `append_message`, +`CreateConversationThread`, `ConversationThread`, `ConversationMessage`. Today +`memory/conversations/mod.rs` is a `pub use tinymemory_core::conversations::*;` +shim, and `threads/ops.rs` and the channel adapters reach through it. + +### A3. Source **registry** — widen `MemorySourceSink` (~20 refs) + +`MemorySourceSink` today has exactly two methods, `accept_source_items` and +`forget_source`. The registry half — `list_sources`, `get_source`, upsert, +enable/disable — has no bus representation at all, so `memory/tools/diff.rs`, +`memory/sources/rpc.rs` and `memory/sources/reconcile.rs` cannot move. + +The same module also needs the coding-session ingest surface +(`coding_session_status`, `ingest_coding_sessions`, `CodingSessionIngestRequest` +/ `Response`, `CodingSessionSourceStatus`), the sync audit log (`read_audit_log`, +`SyncAuditEntry`), and `estimate_cost_usd`. + +### A4. Composio sync pipelines (~25 refs) + +`run_composio_connection`, `load_composio_sync_state`, +`HOST_SYNC_STATE_NAMESPACE`, `HostSyncAdapter`, `sync::sync_status::*`, +`sync::composio::providers::{ProviderContext, ComposioUsageHandle}`. This is a +pipeline, not a getter — it needs a design pass on what crosses the bus and what +stays a host callback, not just a trait method. + +### A5. `MemoryChat` — the chat host seam (~17 refs) + +`chat::{ChatPrompt, ChatProvider, StaticChatProvider, build_chat_provider, +build_chat_runtime, test_override}` and `chat_host::{ChatHost, set_chat_host}`. +Consumed by the whole `agent/harness/archivist/` family. Note this is a seam the +**host implements for the engine**, so its bus direction is the reverse of the +families above — the same shape as `host::nlp` and `host::embedding_host`, which +already live in `tinymemory-api`. + +### A6. Preferences (8 refs) + +`load_general_preferences`, `recall_situational_preferences`, +`STANDING_PREFS_LIMIT`, `USER_PREF_GENERAL_NAMESPACE`. Read on the hot path in +`agent/harness/session/turn/{core,context}.rs`. + +### A7. Recency recall on `MemoryRetrieval` (2 handlers) + +Carried over verbatim from the previous round, because the obvious migration is +wrong and someone will try it again: + +- `recall_namespace_scored` resolves to + `query_namespace_hits_excluding_session(ns, query, limit, exclude)` — the + **query-ranked** path. +- `memory.recall_context` and `memory.recall_memories` call + `recall_namespace_memories(ns, limit)` — a distinct **recency** path. + +Passing an empty query to the scored method does not degrade to recency; it runs +the ranking path with nothing to rank against. The two share a prefix +(`load_documents_for_scope` + `kv_records_for_scope`) and diverge after it, so +the swap compiles, returns plausible hits, and quietly changes what the user +gets. The ask is a `RecallNamespaceRecent`-shaped method. + +### A8. Smaller, one method each + +- `MemoryDiff::snapshot_count_for_source` — `memory/tools/diff.rs` opens a + `Ledger` directly for a count today. +- Persona compilation — `compile_flavoured_root`, `flavoured_root_abs_path`, + `PersonaFacet`, `tree::store::get_tree_by_scope`. `MemoryProfile` covers + *facets* (`ProfileFacet`) but not rendering a flavoured root, and + `PersonaFacet` is a different type. +- `ingest_pipeline::{ingest_chat, ingest_document, ingest_email, + ingest_document_with_scope}` and the `ingest::canonicalize::{chat, document, + email}` input types. `MemoryIngest` has `ingest_document` and `ingest_chat` + over a generic `IngestItem`; there is **no `ingest_email`**, and the + canonicaliser input shapes are the engine's. + +### A9. The chunk readers need the writer, and the writer needs a wider `IngestOutcome` + +**Six revisions.** The earlier five are kept below in one line each, because +every one of them reads as correct and the next person will land on one. + +The symptom throughout: migrating `memory_tree_list_chunks` / `get_chunk` onto +`MemoryChunks` compiles cleanly and breaks four `ingest_document` round-trips, +which read back 0 chunks. + +| # | Claim | Verdict | +| --- | --- | --- | +| 1 | "the `config` argument selects the store" | true, not the cause | +| 2 | "so honour the argument" — the historical `guard_for_config` experiment over `binding::for_workspace` | built it; still 0 chunks | +| 3 | "the handlers need an injection seam" | necessary, not sufficient | +| 4 | "the writer migration is a behaviour change (chunking moves into the driver)" | **false** — same pipeline both ways | +| 5 | "the blocker is email" | true but not the first blocker | +| 6 | **the blocker is `IngestOutcome`** | **this one** | + +**Why the zero happens at all.** `MemorySubsystemConfig::default()` names driver +`"tinycortex"`, `binding::admit` aliases it to the `tinymemory` module, no +artifact exists in a unit test, so the binding falls back as documented and the +guard wraps the **null driver** while fixtures write through `ingest_pipeline` +to the engine's SQLite. + +**Revision 4 was wrong and the correction matters**, because the expensive +version of this story would deter the work. Migrating the writer does *not* +relocate chunking or entity scoring: +`TinycortexProvider::ingest_document` calls +`tinymemory_core::ingest_pipeline::ingest_document_with_scope` — the same +pipeline the host calls today. Its `title: String::new()` drops +`DocumentInput.title`, but `canonicalise` reads `title` only in its emptiness +guard and deliberately keeps it out of the content, so the only behavioural +delta there is a document with an empty body and a non-empty title. `provider` +is inert. + +**The blocker is that `IngestOutcome` is narrower than `IngestResult`.** The RPC +returns `IngestResult` (= tinycortex's `IngestSummary`) with six fields; the bus +carries three. + +| `IngestResult` | `IngestOutcome` | Survives the bus? | +| --- | --- | --- | +| `chunks_written` | `written` | yes | +| `chunk_ids` | `ids` | yes | +| `source_id` | — | caller already knows it | +| `chunks_dropped` | folded into `skipped` | **only when not already-ingested** | +| `already_ingested` | folded into `skipped` | **no — becomes ambiguous** | +| `extract_jobs_enqueued` | — | **no** | + +The adapter collapses two distinct facts into one number: +`skipped = if already_ingested { 1 } else { chunks_dropped }`. That is the exact +ambiguity `already_ingested` exists to prevent — `wipe_all_clears_ingest_gate` +in `read_rpc_tests.rs` pins a shipped bug where a wiped source could never +re-ingest because the gate stayed claimed, and the symptom was "0 chunks +written" that only `already_ingested` distinguished from a normal no-op. +Migrating the writer today would put that back on the wire as a public RPC +response narrowing. + +**So A9 needs three upstream additions**, filed together as +[tinymemory#88](https://github.com/tinyhumansai/tinymemory/issues/88) because +neither is useful alone for this consumer: + +1. `IngestOutcome` gains `already_ingested` and `extract_jobs_enqueued`, and + keeps `skipped` meaning dropped-chunks only. +2. An email ingest method — `ingest_rpc` dispatches Chat/Email/Document and + `MemoryIngest` has only `ingest_document` and `ingest_chat`. Already listed + here as the missing `ingest_email`. +3. `MemoryIngest::ingest_document` must carry `DocumentInput.title` (or reject + title-only documents explicitly). The current adapter sends an empty title, + so migrating before this is fixed would silently skip a valid title-only + request. + +There is no current per-call `guard_for_config` escape hatch. The remaining +handler work must instead be proven with same-workspace tests against a real +memory module, so the reader and writer cross the same bound driver. + +**What is already done, and is in this PR.** The test-side half no longer +blocks anything: `memory::guard::in_memory::InMemoryChunks` + +`guarded_in_memory_chunks()` is a storing provider implementing `MemoryIngest` +and `MemoryChunks`, so a write and a read that both cross a real `MemoryGuard` +reach the same store. `InMemoryProvider` could not serve this — it implements +the mandatory three only, so `as_chunks()` is `None` and the guard answers +`Unsupported`. Three tests pin the round trip, the `source_id` dedupe, and +scope-before-limit narrowing. When the two upstream additions land, the handler +migration is a seam plus these fixtures, not a research problem. + +**What stays true regardless:** do not make those four tests pass by binding a +global client first. They are the only thing that goes red when a reader moves +to the guard while the writer stays on the store. + +### A10. Richer chunk listing for `read_rpc` — a design pass, not a method + +`memory/read_rpc/chunks.rs` builds **raw SQL against the engine's schema**: +`mem_tree_chunks` joined to `mem_tree_entity_index`, with a `total` count +alongside the page. `ChunkQuery` has no entity filter and `list_chunks` returns +no total, so this cannot move as-is. Either the query type grows those two +things, or the read surface changes shape. + +## B. Stays host-side — do not "fix" these into bus calls + +- **`store::safety::{sanitize_text, has_likely_secret, sanitize_json, + canonical_identifier}`** (11 production sites, most of them *outside* the + memory tree — the approval store, artifact offload, tool results). These are + `regex`-based and `tinymemory-api` forbids `regex`; the crate's manifest + carries that invariant with a `cargo tree` guard. Repointing them at + `tinycortex` to shrink the ratchet would move the number without moving the + substance. They should be ported into the host the way `util::redact` was. +- **`memory::ranking`** — already done. Both search tools fetch through the + guard first, so what remains is arithmetic over values the caller holds. +- **`memory::source_scope`** — host policy, not a seam gap. It crosses the bus + as a `SourceScope` *value* on every scoped method, because a task-local set by + the host cannot be read inside the module's `cdylib`, which has its own + statics. +- **`global::{client, client_if_ready, init, active_workspace_dir}`** — the + per-workspace singleton. Some of the 30 references are genuinely host-side + workspace resolution and should stay; the rest should resolve through + `binding` / `active_memory_guard` instead. This is host cleanup, not upstream + work. + +## C. Order of operations + +1. **A1 (queue) and A2 (conversations)** first — they are the biggest, and they + unblock the ingest and thread surfaces that everything else reads from. +2. **A3 + A4 together** — the source registry and the sync pipelines are one + subsystem in practice; splitting them means two releases to get one working + path. +3. **A5 + A6** — both are agent-harness hot-path reads. +4. **A9 needs one `tinymemory` change carrying two additions** + ([tinymemory#88](https://github.com/tinyhumansai/tinymemory/issues/88)) — a + wider `IngestOutcome` (`already_ingested`, `extract_jobs_enqueued`) and an + email ingest method. v1.2.0 already serves the four methods themselves, so this is + a contract widening rather than new plumbing. The host-side test fixtures + are already in place (`guarded_in_memory_chunks`), so once it lands the + migration is writer-plus-readers in one change. +5. **A7, A8, A10** — small or design-bound; can ride any release. +6. Only then: delete the `tinycortex` and `tinymemory-core` dependencies, drop + `vendor/tinycortex`, and write the shed back into + `scripts/kernel-floor.limits`. + +Each release step is a `tinymemory` tag plus a `modules::registry` re-pin with +digests taken **verbatim from the release's `checksum.toml`**, never recomputed +from a local build. + +## D. The two ratchets are the to-do list + +`memory/direct_engine_refs_tests.rs` (engine references) and +`memory/bypass_allowlist_tests.rs` (unguarded driver reach) each enumerate the +current callers with a verdict and a reason. **Both may shrink and must never +grow**, and each has a staleness half: a migrated file must be *deleted* from the +list, not left behind as a comment. Keeping them green is what makes this +converge instead of drifting. diff --git a/scripts/ci/product-features.txt b/scripts/ci/product-features.txt index 872f3743866..ebf50b75592 100644 --- a/scripts/ci/product-features.txt +++ b/scripts/ci/product-features.txt @@ -31,6 +31,11 @@ channels # The media_generate_* agent tools. media +# The whole memory family: recall/ingest/tree/goals/sources/sync, their RPC +# namespaces, the memory agent tools, the guard, and the TinyMemory module the +# driver binding loads. Without it the product has no memory at all. +memory + # Local audio-device access (cpal): voice recording plus the accessibility # microphone-permission probe. `voice` requires it. inference diff --git a/src/core/all.rs b/src/core/all.rs index e52d54c702c..08789b461e3 100644 --- a/src/core/all.rs +++ b/src/core/all.rs @@ -638,6 +638,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::inference::embeddings::all_embeddings_registered_controllers(), ); // People resolution and interaction scoring + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -719,6 +720,7 @@ fn build_registered_controllers() -> Vec { // registers as nine tagged pushes rather than one (M5.2). Order matches // `memory::schemas::all_registered_controllers`, which // `registered_controller_order_is_pinned_to_the_capability_partition_snapshot` pins. + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -791,6 +793,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::memory::all_memory_tool_memory_registered_controllers(), ); // Long-term goals list (editable list + turn-based enrichment agent) + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -804,6 +807,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::threads::goals::all_thread_goals_registered_controllers(), ); // Memory tree ingestion layer (#707 — canonicalised chunks with provenance) + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -818,6 +822,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::memory::tree::all_memory_tree_registered_controllers(), ); // Memory tree retrieval layer (#710 — LLM-callable read tools over the tree) + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -825,6 +830,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::memory::tree::all_retrieval_registered_controllers(), ); // Slack → memory-tree ingestion engine (per-message ingest, no bucketing) + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -832,9 +838,10 @@ fn build_registered_controllers() -> Vec { // driver that cannot accept synced source items should lose the whole // source-sync surface coherently, not half of it. Some(Capability::Sources), - crate::openhuman::integrations::composio::providers::slack::all_slack_memory_registered_controllers(), + crate::openhuman::memory::sync::composio::providers::slack::all_slack_memory_registered_controllers(), ); // Per-connection memory sync status, controls, and progress (#1136) + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -843,6 +850,7 @@ fn build_registered_controllers() -> Vec { ), ); // Memory sources — user-configured data connectors registry + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -850,6 +858,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::memory::sources::all_memory_sources_registered_controllers(), ); // Memory diff — snapshot-based change tracking for memory sources + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -929,6 +938,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::platform::update::all_update_registered_controllers(), ); // Hierarchical knowledge summarization + #[cfg(feature = "memory")] push_cap( &mut controllers, DomainGroup::Memory, @@ -936,6 +946,7 @@ fn build_registered_controllers() -> Vec { crate::openhuman::memory::tree::all_tree_summarizer_registered_controllers(), ); // Self-learning and user context enrichment + #[cfg(feature = "memory")] push( &mut controllers, DomainGroup::Agent, diff --git a/src/core/cli.rs b/src/core/cli.rs index f807c4ceb63..35d8144c6c9 100644 --- a/src/core/cli.rs +++ b/src/core/cli.rs @@ -92,12 +92,22 @@ pub fn run_from_cli_args(args: &[String]) -> Result<()> { // diagnostic rather than a misleading "unknown namespace" error. "tui" | "chat" => run_tui_from_cli(&args[1..]), "call" => run_call_command(&args[1..]), + #[cfg(feature = "memory")] "tree-summarizer" => { crate::openhuman::memory::tree::tree_runtime::cli::run_tree_summarizer_command( &args[1..], ) } + #[cfg(feature = "memory")] "memory" => crate::core::memory_cli::run_memory_command(&args[1..]), + // Kept rather than deleted so the failure names the build, not the + // user: falling through to generic namespace resolution would answer + // "unknown namespace: memory", which reads like a typo. Same shape as + // the `mcp` and `tui` arms above. + #[cfg(not(feature = "memory"))] + "memory" | "tree-summarizer" => Err(anyhow::anyhow!( + "memory feature disabled at compile time — rebuild with `--features memory`" + )), "agent" => { log::debug!( "[cli] dispatching to agent subcommand, args={:?}", diff --git a/src/core/cli_capability.rs b/src/core/cli_capability.rs index 4dc9a42cf8a..5873739431f 100644 --- a/src/core/cli_capability.rs +++ b/src/core/cli_capability.rs @@ -147,7 +147,14 @@ pub async fn bound_memory_driver() -> Option<(String, DriverClass, Capabilities) return None; } }; - bound_memory_driver_for(&config.workspace_dir, &config.subsystems.memory) + #[cfg(feature = "memory")] + { + bound_memory_driver_for(&config.workspace_dir, &config.subsystems.memory) + } + #[cfg(not(feature = "memory"))] + { + None + } } /// [`bound_memory_driver`] against an already-loaded config. @@ -161,6 +168,7 @@ pub async fn bound_memory_driver() -> Option<(String, DriverClass, Capabilities) /// Nothing here touches memory *data*: only the driver id, its class, and the /// advertised capability set — exactly what `memory.provider_status` already /// reports over RPC. +#[cfg(feature = "memory")] pub fn bound_memory_driver_for( workspace_dir: &std::path::Path, cfg: &crate::openhuman::config::schema::MemorySubsystemConfig, @@ -191,6 +199,14 @@ pub fn bound_memory_driver_for( } } +#[cfg(not(feature = "memory"))] +pub fn bound_memory_driver_for( + _workspace_dir: &std::path::Path, + _cfg: &crate::openhuman::config::schema::MemorySubsystemConfig, +) -> Option<(String, DriverClass, Capabilities)> { + None +} + /// Blocking gate for the synchronous CLI call sites. /// /// `required == None` short-circuits before any runtime is built or any config diff --git a/src/core/jsonrpc.rs b/src/core/jsonrpc.rs index 20278a7b658..d1768045c35 100644 --- a/src/core/jsonrpc.rs +++ b/src/core/jsonrpc.rs @@ -217,9 +217,7 @@ pub async fn rpc_handler(State(state): State, Json(req): Json bool { msg == crate::openhuman::web3::wallet::WALLET_NOT_CONFIGURED_MESSAGE } +/// Whether `msg` is a memory ingest-payload schema-validation failure. +/// +/// A thin `#[cfg]` shim over +/// [`memory::tree::tree::rpc::is_invalid_ingest_payload_message`], which lives +/// behind the `memory` gate. It exists as a function rather than a `#[cfg]` in +/// the `else if` chain at the call site because that chain decides Sentry +/// severity and is easier to read whole. +/// +/// With the family compiled out no handler can emit such a message, so the +/// answer is `false` — not "unknown". Anything that *did* start with that +/// prefix in this build would not be an ingest-payload failure, and keeping +/// the loud default is the right way round. +#[cfg(feature = "memory")] +fn memory_ingest_payload_message(msg: &str) -> bool { + crate::openhuman::memory::tree::tree::rpc::is_invalid_ingest_payload_message(msg) +} + +/// See the `memory`-enabled twin above. +#[cfg(not(feature = "memory"))] +fn memory_ingest_payload_message(_msg: &str) -> bool { + false +} + /// Internal method invocation logic. /// /// It first attempts to match the method name against the static controller @@ -2110,6 +2131,7 @@ fn register_domain_subscribers( { log::warn!("[composio][history] failed to initialize trigger archive: {error}"); } + #[cfg(feature = "memory")] crate::openhuman::integrations::composio::register_composio_trigger_subscriber(); crate::openhuman::integrations::task_sources::bus::register_task_sources_subscriber(); } @@ -2139,6 +2161,7 @@ fn register_domain_subscribers( // channel-less users (#5003). `agent::learning` is an Agent-family // domain; it sat on the Platform boot path only because `learning` // used to be a top-level directory. Idempotent. + #[cfg(feature = "memory")] crate::openhuman::agent::learning::startup::register_learning_subscribers( workspace_dir.clone(), ); @@ -2207,6 +2230,7 @@ fn register_domain_subscribers( // block as the trigger subscriber above — that guard only returns // `true` once per process, so a second, separate call here would // never register. + #[cfg(feature = "memory")] if let Some(handle) = crate::core::bus::BUS.subscribe(Arc::new( crate::openhuman::flows::bus::FlowRunDigestSubscriber::new(Arc::new( config.clone(), @@ -2249,6 +2273,7 @@ fn register_domain_subscribers( // Memory: conversation-persistence + sync-stage bridge. if plan.memory { + #[cfg(feature = "memory")] if group_first_time(DomainGroup::Memory) { crate::openhuman::memory::conversations::register_conversation_persistence_subscriber( workspace_dir.clone(), diff --git a/src/core/mod.rs b/src/core/mod.rs index f4cdbc9747f..6f3b4146150 100644 --- a/src/core/mod.rs +++ b/src/core/mod.rs @@ -24,6 +24,7 @@ pub mod jsonrpc; pub mod legacy_aliases; pub mod log_redaction; pub mod logging; +#[cfg(feature = "memory")] pub mod memory_cli; pub mod observability; pub mod rpc_log; diff --git a/src/core/runtime/context.rs b/src/core/runtime/context.rs index c780b22fb07..522ac2b4249 100644 --- a/src/core/runtime/context.rs +++ b/src/core/runtime/context.rs @@ -720,7 +720,7 @@ pub async fn init_stores( } } -#[cfg(test)] +#[cfg(all(test, feature = "memory"))] mod tests { use super::*; use std::path::PathBuf; diff --git a/src/core/subsystem/schemas.rs b/src/core/subsystem/schemas.rs index d1225c7f36b..c429e594702 100644 --- a/src/core/subsystem/schemas.rs +++ b/src/core/subsystem/schemas.rs @@ -72,7 +72,33 @@ pub fn schemas(function: &str) -> ControllerSchema { /// Memory is the only occupant today. This is also what /// [`crate::core::subsystems_cli`] renders as a table. pub async fn subsystems_status() -> Vec { - vec![crate::openhuman::memory::rpc::memory_subsystem_status().await] + #[cfg(feature = "memory")] + { + vec![crate::openhuman::memory::rpc::memory_subsystem_status().await] + } + // Memory is the table's only occupant, so with the family compiled out the + // table is empty — not a row reporting a broken subsystem, which is what a + // disabled-error stub here would have produced. + #[cfg(not(feature = "memory"))] + { + vec![crate::core::subsystem::SubsystemStatus { + slot: crate::core::subsystem::SubsystemSlot::Memory + .as_str() + .to_string(), + driver: "disabled".to_string(), + class: crate::core::subsystem::DriverClass::Null + .as_str() + .to_string(), + health: "down".to_string(), + health_reason: Some("the `memory` feature is disabled in this build".to_string()), + contract_version: crate::core::subsystem::format_contract_version( + crate::openhuman::memory::api::CONTRACT_VERSION, + ), + capabilities: Vec::new(), + fell_back_from: None, + last_error: Some("the `memory` feature is disabled in this build".to_string()), + }] + } } fn handle_status(_params: Map) -> ControllerFuture { @@ -122,5 +148,10 @@ mod tests { assert_eq!(rows[0]["slot"], "memory"); assert!(rows[0]["contract_version"].is_string()); assert!(rows[0]["capabilities"].is_array()); + #[cfg(not(feature = "memory"))] + { + assert_eq!(rows[0]["driver"], "disabled"); + assert_eq!(rows[0]["health"], "down"); + } } } diff --git a/src/openhuman/agent/debug/mod.rs b/src/openhuman/agent/debug/mod.rs index 0a07cf720e8..4d971949ec8 100644 --- a/src/openhuman/agent/debug/mod.rs +++ b/src/openhuman/agent/debug/mod.rs @@ -234,6 +234,7 @@ async fn load_dump_config( // `dump-all` invocation aborts before rendering a single prompt. // Idempotent, so calling it per invocation is safe. Same rationale as // `memory_cli` / `subconscious_cli`. + #[cfg(feature = "memory")] crate::openhuman::memory::host_impls::install_memory_host_seams(std::sync::Arc::new( config.clone(), )); diff --git a/src/openhuman/agent/harness/memory_context.rs b/src/openhuman/agent/harness/memory_context.rs index e7e9202e32f..e9c6ff3aa7c 100644 --- a/src/openhuman/agent/harness/memory_context.rs +++ b/src/openhuman/agent/harness/memory_context.rs @@ -159,7 +159,7 @@ pub(crate) async fn build_context( // primary JSONL path, and the orchestrator prompt's // "Capability questions" section that names this header stays // in sync. See CROSS_CHAT_HEADER's doc for the rationale. - context.push_str(crate::openhuman::memory::agent::memory_loader::CROSS_CHAT_HEADER); + context.push_str(crate::openhuman::memory::citation::CROSS_CHAT_HEADER); for entry in &cross { let prov = entry .session_id @@ -377,9 +377,7 @@ mod tests { let context = build_context(&mem, "what database should I use?", 0.4).await; assert!( - context.contains( - crate::openhuman::memory::agent::memory_loader::CROSS_CHAT_HEADER.trim_end() - ), + context.contains(crate::openhuman::memory::citation::CROSS_CHAT_HEADER.trim_end()), "expected cross-chat header, got:\n{context}" ); assert!( @@ -445,9 +443,7 @@ mod tests { let mem = MockMemory::new(Vec::new(), Vec::new(), false); let context = build_context(&mem, "Postgres", 0.4).await; assert!( - !context.contains( - crate::openhuman::memory::agent::memory_loader::CROSS_CHAT_HEADER.trim_end() - ), + !context.contains(crate::openhuman::memory::citation::CROSS_CHAT_HEADER.trim_end()), "no cross-chat hits must produce no header, got:\n{context}" ); } diff --git a/src/openhuman/agent/harness/mod.rs b/src/openhuman/agent/harness/mod.rs index 516b8a26cb6..e5efe82107a 100644 --- a/src/openhuman/agent/harness/mod.rs +++ b/src/openhuman/agent/harness/mod.rs @@ -22,6 +22,7 @@ //! `crate::openhuman::agent::tinyagents`); there is no in-house interrupt fence. pub mod agent_graph; +#[cfg(feature = "memory")] pub mod archivist; pub mod artifact_offload; pub(crate) mod builtin_definitions; diff --git a/src/openhuman/agent/harness/session/builder/factory.rs b/src/openhuman/agent/harness/session/builder/factory.rs index befbf188c07..8f2aa092778 100644 --- a/src/openhuman/agent/harness/session/builder/factory.rs +++ b/src/openhuman/agent/harness/session/builder/factory.rs @@ -1,6 +1,7 @@ //! `Agent::from_config` factory methods and the internal //! `build_session_agent_inner` constructor. +#[cfg(feature = "memory")] use super::helpers::prefetch_tool_memory_rules_blocking; use super::should_synthesize_delegation_tools; use crate::openhuman::agent::context::prompt::SystemPromptBuilder; @@ -14,6 +15,7 @@ use crate::openhuman::agent::harness::session::types::Agent; use crate::openhuman::agent::host_runtime; use crate::openhuman::config::Config; use crate::openhuman::inference::provider; +#[cfg(feature = "memory")] use crate::openhuman::memory::tool_memory::capture::ToolMemoryCaptureHook; use crate::openhuman::memory::Memory; use crate::openhuman::security::SecurityPolicy; @@ -578,6 +580,7 @@ impl Agent { }, None => SystemPromptBuilder::with_defaults(), }; + #[cfg(feature = "memory")] if config.learning.enabled { // Insert the privileged reflection block ahead of the // generic `user_memory` section when one is already @@ -612,6 +615,7 @@ impl Agent { // reach every session prompt. The `fetch_learned_context` gate is // widened by `explicit_preferences_enabled` on the Agent (see // `session/turn.rs`) so the data is actually fetched and populated. + #[cfg(feature = "memory")] if config.learning.explicit_preferences_enabled && !config.learning.enabled { prompt_builder = prompt_builder.add_section(Box::new( crate::openhuman::agent::learning::UserProfileSection::new(memory.clone()), @@ -658,6 +662,7 @@ impl Agent { let mut post_turn_hooks: Vec> = Vec::new(); if config.learning.enabled { + #[cfg(feature = "memory")] if config.learning.reflection_enabled { // The reflection hook needs an owned `Arc`; reuse the // shared base config (a refcount bump) rather than a second deep @@ -693,6 +698,7 @@ impl Agent { ); } + #[cfg(feature = "memory")] if config.learning.user_profile_enabled { post_turn_hooks.push(Arc::new( crate::openhuman::agent::learning::UserProfileHook::new( @@ -703,6 +709,7 @@ impl Agent { log::info!("[learning] user_profile hook registered"); } + #[cfg(feature = "memory")] if config.learning.tool_tracking_enabled { post_turn_hooks.push(Arc::new( crate::openhuman::agent::learning::ToolTrackerHook::new( @@ -713,6 +720,7 @@ impl Agent { log::info!("[learning] tool_tracker hook registered"); } + #[cfg(feature = "memory")] if config.learning.tool_memory_capture_enabled { post_turn_hooks.push(Arc::new(ToolMemoryCaptureHook::new(memory.clone(), true))); log::info!("[learning] tool_memory_capture hook registered"); @@ -740,6 +748,7 @@ impl Agent { // the inference stack (`reflection`, `stability_detector`) is disabled. // Gated only on `config.learning.episodic_capture_enabled` (default: true) // using the explicit SQLite resource returned by the session factory. + #[cfg(feature = "memory")] let archivist_hook_arc: Option< Arc, > = if config.learning.episodic_capture_enabled { @@ -997,6 +1006,7 @@ impl Agent { let allowed_by_filter = visible.is_empty() || visible.contains(*name); registered && allowed_by_filter }); + #[cfg(feature = "memory")] if has_retrieval { prompt_builder = prompt_builder.add_section(Box::new( crate::openhuman::agent::learning::MemoryAccessSection, @@ -1033,6 +1043,7 @@ impl Agent { // and users who opt out of learning expect no stored rules to surface) // or when the runtime cannot host a synchronous bridge (single-threaded // test harnesses). + #[cfg(feature = "memory")] if config.learning.enabled && config.learning.tool_memory_capture_enabled { let agent_tool_names: Vec = tools.iter().map(|t| t.name().to_string()).collect(); @@ -1267,7 +1278,10 @@ impl Agent { if let Some(ps) = payload_summarizer { builder = builder.payload_summarizer(ps); } - builder = builder.archivist_hook(archivist_hook_arc); + #[cfg(feature = "memory")] + { + builder = builder.archivist_hook(archivist_hook_arc); + } let mut agent = builder.build()?; let connected_integrations_initialized = prewarmed_integrations.is_some(); agent.connected_integrations = prewarmed_integrations.unwrap_or_default(); diff --git a/src/openhuman/agent/harness/session/builder/helpers.rs b/src/openhuman/agent/harness/session/builder/helpers.rs index 7152479aa08..732d895e558 100644 --- a/src/openhuman/agent/harness/session/builder/helpers.rs +++ b/src/openhuman/agent/harness/session/builder/helpers.rs @@ -1,5 +1,6 @@ //! Utility helpers used during agent construction. +#[cfg(feature = "memory")] use crate::openhuman::memory::tool_memory::{tool_memory_store, ToolMemoryRule}; use crate::openhuman::memory::Memory; use std::sync::Arc; @@ -20,6 +21,7 @@ use std::sync::Arc; /// Critical / High rules captured later in the session are still /// available via the `memory_tool_rules_for_prompt` RPC; this prefetch /// merely seeds the rules that exist at session start. +#[cfg(feature = "memory")] pub(super) fn prefetch_tool_memory_rules_blocking( memory: Arc, tool_names: &[String], diff --git a/src/openhuman/agent/harness/session/builder/setters.rs b/src/openhuman/agent/harness/session/builder/setters.rs index 8d095966fdd..fabcb17adbb 100644 --- a/src/openhuman/agent/harness/session/builder/setters.rs +++ b/src/openhuman/agent/harness/session/builder/setters.rs @@ -56,6 +56,7 @@ impl AgentBuilder { tokenjuice_compression: crate::openhuman::inference::tokenjuice::AgentTokenjuiceCompression::Full, tool_policy: None, + #[cfg(feature = "memory")] archivist_hook: None, } } @@ -423,6 +424,7 @@ impl AgentBuilder { /// SQLite connection is available. Callers that construct an `Agent` /// directly (tests, CLI) can leave this `None` — flush is a no-op /// when the hook is absent. + #[cfg(feature = "memory")] pub fn archivist_hook( mut self, hook: Option>, @@ -689,6 +691,7 @@ impl AgentBuilder { announced_skills: std::collections::HashSet::new(), pending_skill_announcement: Vec::new(), pending_skill_retraction: Vec::new(), + #[cfg(feature = "memory")] archivist_hook: self.archivist_hook, synthesized_tool_names: std::collections::HashSet::new(), pending_synthesized_tools_mask: std::collections::HashSet::new(), diff --git a/src/openhuman/agent/harness/session/runtime.rs b/src/openhuman/agent/harness/session/runtime.rs index b43c6c7c7e2..246b303998d 100644 --- a/src/openhuman/agent/harness/session/runtime.rs +++ b/src/openhuman/agent/harness/session/runtime.rs @@ -139,6 +139,7 @@ impl Agent { /// the session already holds, so constructing one is a refcount bump, and /// storing it would create a second handle that could drift from /// `self.memory` if the backend were ever swapped. + #[cfg(feature = "memory")] pub fn host_agent_memory( &self, ) -> crate::openhuman::agent::tinyagents::host::OpenHumanAgentMemory { @@ -632,7 +633,7 @@ impl Agent { /// wait. pub async fn take_last_turn_citations( &mut self, - ) -> Vec { + ) -> Vec { if let Some(handle) = self.pending_citations.take() { match handle.await { Ok(citations) => self.last_turn_citations = citations, diff --git a/src/openhuman/agent/harness/session/turn/context.rs b/src/openhuman/agent/harness/session/turn/context.rs index 9281bb440ba..e96cba2d314 100644 --- a/src/openhuman/agent/harness/session/turn/context.rs +++ b/src/openhuman/agent/harness/session/turn/context.rs @@ -222,9 +222,9 @@ impl Agent { let reflection_entries = self .memory .list( - Some(crate::openhuman::agent::learning::reflection::REFLECTIONS_NAMESPACE), + Some(crate::openhuman::memory::namespaces::REFLECTIONS_NAMESPACE), Some(&MemoryCategory::Custom( - crate::openhuman::agent::learning::reflection::REFLECTIONS_NAMESPACE.into(), + crate::openhuman::memory::namespaces::REFLECTIONS_NAMESPACE.into(), )), None, ) diff --git a/src/openhuman/agent/harness/session/turn/core.rs b/src/openhuman/agent/harness/session/turn/core.rs index 9d793fa1a86..dd27351fa18 100644 --- a/src/openhuman/agent/harness/session/turn/core.rs +++ b/src/openhuman/agent/harness/session/turn/core.rs @@ -15,6 +15,7 @@ use crate::openhuman::agent::harness::fork_context::ParentExecutionContext; use crate::openhuman::agent::hooks::{self, TurnContext}; use crate::openhuman::agent::messages::{ChatMessage, ConversationMessage}; use crate::openhuman::agent::progress::AgentProgress; +#[cfg(feature = "memory")] use crate::openhuman::memory::agent::memory_loader::collect_recall_citations; use crate::openhuman::memory::MemoryCategory; use crate::openhuman::util::truncate_with_ellipsis; @@ -401,7 +402,7 @@ impl Agent { tokio::spawn(async move { match memory .store( - crate::openhuman::agent::learning::transcript_ingest::CONVERSATION_RAW_NAMESPACE, + crate::openhuman::memory::namespaces::CONVERSATION_RAW_NAMESPACE, &autosave_key, &user_msg, MemoryCategory::Conversation, @@ -431,32 +432,38 @@ impl Agent { // turn it belonged to. previous.abort(); } - let citation_memory = self.memory.clone(); - let citation_query = user_message.to_string(); - self.pending_citations = Some(tokio::spawn(async move { - match collect_recall_citations( - citation_memory.as_ref(), - &citation_query, - MEMORY_CITATION_LIMIT, - MEMORY_CITATION_MIN_RELEVANCE, - ) - .await - { - Ok(citations) => { - log::debug!( - "[agent_loop] memory citations collected count={}", - citations.len() - ); - citations - } - Err(_err) => { - // Recall errors may include the user-authored query. Keep - // warning logs free of raw external content. - log::warn!("[agent_loop] memory citation collection failed"); - Vec::new() + // UI-only, and a full recall: with the family compiled out there is + // nothing to cite, so no task is spawned and `last_turn_citations` + // simply stays empty — the same state a turn with no hits reaches. + #[cfg(feature = "memory")] + { + let citation_memory = self.memory.clone(); + let citation_query = user_message.to_string(); + self.pending_citations = Some(tokio::spawn(async move { + match collect_recall_citations( + citation_memory.as_ref(), + &citation_query, + MEMORY_CITATION_LIMIT, + MEMORY_CITATION_MIN_RELEVANCE, + ) + .await + { + Ok(citations) => { + log::debug!( + "[agent_loop] memory citations collected count={}", + citations.len() + ); + citations + } + Err(_err) => { + // Recall errors may include the user-authored query. Keep + // warning logs free of raw external content. + log::warn!("[agent_loop] memory citation collection failed"); + Vec::new() + } } - } - })); + })); + } // No per-turn memory-context block is assembled here any more. // // `memory_loader.load_context()` used to prepend `[User working @@ -1332,7 +1339,7 @@ impl Agent { let _ = self .memory .store( - crate::openhuman::agent::learning::transcript_ingest::CONVERSATION_RAW_NAMESPACE, + crate::openhuman::memory::namespaces::CONVERSATION_RAW_NAMESPACE, &autosave_key, &summary, MemoryCategory::Daily, diff --git a/src/openhuman/agent/harness/session/turn/mod.rs b/src/openhuman/agent/harness/session/turn/mod.rs index d4dd82978ab..c2d467a6092 100644 --- a/src/openhuman/agent/harness/session/turn/mod.rs +++ b/src/openhuman/agent/harness/session/turn/mod.rs @@ -163,6 +163,7 @@ pub(super) fn collect_tree_root_summaries( per_namespace_cap: usize, total_cap: usize, ) -> Vec { + #[cfg(feature = "memory")] let rows = if memory_subdir == "memory" { crate::openhuman::memory::tree::tree_runtime::store::collect_root_summaries_with_caps( workspace_dir, @@ -176,6 +177,17 @@ pub(super) fn collect_tree_root_summaries( total_cap, ) }; + // Without the family there is no `memory-*` tree on disk to read, so every + // subdir takes the profile-tree path — which reads plain markdown and does + // not need a driver. + #[cfg(not(feature = "memory"))] + let rows = { + collect_profile_tree_root_summaries( + &workspace_dir.join(memory_subdir), + per_namespace_cap, + total_cap, + ) + }; rows.into_iter() .map(|(namespace, body, updated_at)| { crate::openhuman::agent::context::prompt::NamespaceSummary { diff --git a/src/openhuman/agent/harness/session/turn/session_io.rs b/src/openhuman/agent/harness/session/turn/session_io.rs index 626f7029379..3df28355971 100644 --- a/src/openhuman/agent/harness/session/turn/session_io.rs +++ b/src/openhuman/agent/harness/session/turn/session_io.rs @@ -817,6 +817,7 @@ impl Agent { // This honours the doc-comment guarantee on `flush_open_segment` in // `archivist.rs`. No deadlock risk: no mutex guard is held across // this await point. + #[cfg(feature = "memory")] if let Some(ref archivist) = self.archivist_hook { let session_id = self.event_session_id.clone(); log::debug!( @@ -902,33 +903,36 @@ impl Agent { /// /// Fire-and-forget: failures are logged, never propagated. pub(in super::super) fn spawn_transcript_ingestion(&self) { - let Some(path) = self.session_transcript_path.clone() else { - log::debug!("[transcript_ingest] no session transcript path yet — skipping spawn"); - return; - }; - let memory = std::sync::Arc::clone(&self.memory); - - tokio::spawn(async move { - match crate::openhuman::agent::learning::transcript_ingest::ingest_transcript_path( - memory.as_ref(), - &path, - ) - .await - { - Ok(report) => tracing::info!( - transcript = %path.display(), - extracted = report.extracted, - stored = report.stored, - deduped = report.deduped, - reflections_stored = report.reflections_stored, - "[transcript_ingest] background ingest complete" - ), - Err(err) => tracing::warn!( - transcript = %path.display(), - error = %err, - "[transcript_ingest] background ingest failed — will retry next threshold window" - ), - } - }); + #[cfg(feature = "memory")] + { + let Some(path) = self.session_transcript_path.clone() else { + log::debug!("[transcript_ingest] no session transcript path yet — skipping spawn"); + return; + }; + let memory = std::sync::Arc::clone(&self.memory); + + tokio::spawn(async move { + match crate::openhuman::agent::learning::transcript_ingest::ingest_transcript_path( + memory.as_ref(), + &path, + ) + .await + { + Ok(report) => tracing::info!( + transcript = %path.display(), + extracted = report.extracted, + stored = report.stored, + deduped = report.deduped, + reflections_stored = report.reflections_stored, + "[transcript_ingest] background ingest complete" + ), + Err(err) => tracing::warn!( + transcript = %path.display(), + error = %err, + "[transcript_ingest] background ingest failed — will retry next threshold window" + ), + } + }); + } } } diff --git a/src/openhuman/agent/harness/session/types.rs b/src/openhuman/agent/harness/session/types.rs index 1b27c0d0f3f..4a7ba7da178 100644 --- a/src/openhuman/agent/harness/session/types.rs +++ b/src/openhuman/agent/harness/session/types.rs @@ -9,6 +9,7 @@ use crate::openhuman::agent::context::prompt::SystemPromptBuilder; use crate::openhuman::agent::context::ContextManager; use crate::openhuman::agent::dispatcher::ToolDispatcher; +#[cfg(feature = "memory")] use crate::openhuman::agent::harness::archivist::ArchivistHook; use crate::openhuman::agent::harness::definition::TriggerMemoryAgent; use crate::openhuman::agent::hooks::PostTurnHook; @@ -86,8 +87,7 @@ pub struct Agent { pub(super) last_memory_context: Option, /// Citation metadata collected from memory recall for the most recent turn. /// Consumed by web-channel delivery to render source chips in the UI. - pub(super) last_turn_citations: - Vec, + pub(super) last_turn_citations: Vec, /// In-flight citation recall for the current turn. /// /// Citations are UI-only — they render source chips and never enter the @@ -100,11 +100,8 @@ pub struct Agent { /// joined only when a consumer actually asks for the citations — which /// happens after the turn returns. The contract is unchanged: callers still /// get the citations for the turn they just ran. - pub(super) pending_citations: Option< - tokio::task::JoinHandle< - Vec, - >, - >, + pub(super) pending_citations: + Option>>, /// Holistic token/cost/context accounting for the most recent turn (parent + /// any sub-agents spawned during it). Consumed by web-channel delivery to /// surface session token/cost/context meters in the UI footer. `None` until @@ -383,6 +380,7 @@ pub struct Agent { /// `flush_open_segment` at session-memory-extraction time (the /// closest available signal to "session is ending") to finalize the /// trailing open segment with an LLM recap + embedding. + #[cfg(feature = "memory")] pub(super) archivist_hook: Option>, /// Names of every tool currently in [`Agent::tools`] that was /// produced by [`crate::openhuman::tools::orchestrator_tools::collect_orchestrator_tools`] @@ -502,6 +500,7 @@ pub struct AgentBuilder { /// Optional reference to the production `ArchivistHook`. Set when /// `config.learning.episodic_capture_enabled` is true. Used to call /// `flush_open_segment` at the closest available session-end signal. + #[cfg(feature = "memory")] pub(super) archivist_hook: Option>, } diff --git a/src/openhuman/agent/harness/subagent_runner/ops/runner.rs b/src/openhuman/agent/harness/subagent_runner/ops/runner.rs index f9188ddd769..6b909a2712f 100644 --- a/src/openhuman/agent/harness/subagent_runner/ops/runner.rs +++ b/src/openhuman/agent/harness/subagent_runner/ops/runner.rs @@ -1741,7 +1741,7 @@ async fn run_typed_mode( artifact_paths: Vec::new(), }) } -#[cfg(test)] +#[cfg(all(test, feature = "memory"))] mod fast_path_tests { use super::{ apply_max_result_chars, format_deterministic_memory_hits, parse_memory_fast_path_enabled, diff --git a/src/openhuman/agent/learning/reflection.rs b/src/openhuman/agent/learning/reflection.rs index 329b4a4978b..24af0293307 100644 --- a/src/openhuman/agent/learning/reflection.rs +++ b/src/openhuman/agent/learning/reflection.rs @@ -20,7 +20,7 @@ use std::time::{SystemTime, UNIX_EPOCH}; /// Distinct from `learning_observations` (agent-extracted) and /// `user_profile` (preference facts) — these are sentences the user /// authored about themselves that should steer future agent behaviour. -pub const REFLECTIONS_NAMESPACE: &str = "learning_reflections"; +pub use crate::openhuman::memory::namespaces::REFLECTIONS_NAMESPACE; /// Structured output expected from the reflection LLM call. #[derive(Debug, Clone, Serialize, Deserialize, Default)] diff --git a/src/openhuman/agent/learning/transcript_ingest/types.rs b/src/openhuman/agent/learning/transcript_ingest/types.rs index ccf871e97c8..378cd304db3 100644 --- a/src/openhuman/agent/learning/transcript_ingest/types.rs +++ b/src/openhuman/agent/learning/transcript_ingest/types.rs @@ -29,7 +29,7 @@ pub const CONVERSATION_REFLECTIONS_NAMESPACE: &str = "conversation_reflections"; /// durable facts rather than raw turns. These are the unprocessed messages: /// still queryable on demand by naming this namespace, but no longer swept /// into every default-namespace read. -pub const CONVERSATION_RAW_NAMESPACE: &str = "conversation_raw"; +pub use crate::openhuman::memory::namespaces::CONVERSATION_RAW_NAMESPACE; /// Importance tier — controls which memories are surfaced into a fresh /// chat by default. Only `High` candidates make it into the prompt block; diff --git a/src/openhuman/agent/mod.rs b/src/openhuman/agent/mod.rs index 29ebd5e857d..997fc131356 100644 --- a/src/openhuman/agent/mod.rs +++ b/src/openhuman/agent/mod.rs @@ -31,6 +31,12 @@ pub mod harness; pub mod harness_init; pub mod hooks; pub mod host_runtime; +// The persona/profile learning domain is memory-backed end to end: `FacetCache` +// wraps the driver's `MemoryProfile` family, `transcript_ingest` writes into +// the memory tree, and `tool_tracker` reads tool-effectiveness rules out of +// tool memory. With the family compiled out there is no store behind any of +// it, so the domain is absent rather than inert. +#[cfg(feature = "memory")] pub mod learning; pub mod library; pub(crate) mod message_convert; diff --git a/src/openhuman/agent/prompts/builder.rs b/src/openhuman/agent/prompts/builder.rs index 701f28e540d..fcb7c2de5c8 100644 --- a/src/openhuman/agent/prompts/builder.rs +++ b/src/openhuman/agent/prompts/builder.rs @@ -245,6 +245,7 @@ impl SystemPromptBuilder { /// this method. /// /// No-op when `rules` is empty. + #[cfg(feature = "memory")] pub fn with_tool_memory_rules( mut self, rules: Vec, diff --git a/src/openhuman/agent/registry/agents/loader.rs b/src/openhuman/agent/registry/agents/loader.rs index c67801a96b3..c53dcbcbddc 100644 --- a/src/openhuman/agent/registry/agents/loader.rs +++ b/src/openhuman/agent/registry/agents/loader.rs @@ -283,6 +283,11 @@ pub const BUILTINS: &[BuiltinAgent] = &[ prompt_fn: crate::openhuman::skills::runtime::agent::skill_executor::prompt::build, graph_fn: None, }, + // Gated with `memory` for the same reason as the `flows` agents below: a + // slim build must not advertise an agent whose entire tool belt is absent, + // so the entry — and its `include_str!` — is stripped rather than left to + // resolve to nothing at run time. + #[cfg(feature = "memory")] BuiltinAgent { id: "agent_memory", toml: include_str!("../../../memory/agent/agent/agent.toml"), diff --git a/src/openhuman/agent/registry/agents/orchestrator/prompt.rs b/src/openhuman/agent/registry/agents/orchestrator/prompt.rs index ca80a781dd6..803c6450286 100644 --- a/src/openhuman/agent/registry/agents/orchestrator/prompt.rs +++ b/src/openhuman/agent/registry/agents/orchestrator/prompt.rs @@ -336,7 +336,7 @@ fn render_delegation_guide( // CROSS_CHAT_HEADER (single source of truth) — drift would silently // detune the rule. let cross_chat_header_for_prompt = - crate::openhuman::memory::agent::memory_loader::CROSS_CHAT_HEADER.trim_end(); + crate::openhuman::memory::citation::CROSS_CHAT_HEADER.trim_end(); let _ = write!( out, "\n### Capability questions about connected toolkits\n\n\ diff --git a/src/openhuman/agent/tinyagents/host/mod.rs b/src/openhuman/agent/tinyagents/host/mod.rs index 3b40d006422..a597af5e0b1 100644 --- a/src/openhuman/agent/tinyagents/host/mod.rs +++ b/src/openhuman/agent/tinyagents/host/mod.rs @@ -33,22 +33,26 @@ //! reachable — those are honest gaps, not stubs pretending to work, and each //! names what is missing. +#[cfg(feature = "memory")] pub mod agent_memory; pub mod budget_gate; pub mod context_composer; pub mod definition_registry; pub mod experience_store; +#[cfg(feature = "memory")] pub mod learning_sink; pub mod model_resolver; pub mod progress_sink; pub mod security_gate; pub mod tool_outcome_classifier; +#[cfg(feature = "memory")] pub use agent_memory::OpenHumanAgentMemory; pub use budget_gate::OpenHumanBudgetGate; pub use context_composer::OpenHumanContextComposer; pub use definition_registry::OpenHumanDefinitionRegistry; pub use experience_store::OpenHumanExperienceStore; +#[cfg(feature = "memory")] pub use learning_sink::OpenHumanLearningSink; pub use model_resolver::OpenHumanModelResolver; pub use progress_sink::OpenHumanProgressSink; diff --git a/src/openhuman/agent/tools.rs b/src/openhuman/agent/tools.rs index cc399a060e6..3f86d91a28e 100644 --- a/src/openhuman/agent/tools.rs +++ b/src/openhuman/agent/tools.rs @@ -2,12 +2,14 @@ mod ask_clarification; mod delegate; mod delegate_to_personality; mod plan_exit; +#[cfg(feature = "memory")] pub mod remember_preference; // Pure `skill_runtime` client (spawn + await a workflow run) — compiled out // with the `skills` gate so the tool list OMITS these rather than degrading // them to a disabled-error. #[cfg(feature = "skills")] mod run_workflow; +#[cfg(feature = "memory")] pub mod save_preference; mod todo; mod update_task; @@ -16,11 +18,13 @@ pub use ask_clarification::AskClarificationTool; pub use delegate::DelegateTool; pub use delegate_to_personality::DelegateToPersonalityTool; pub use plan_exit::{PlanExitTool, PLAN_EXIT_MARKER}; +#[cfg(feature = "memory")] pub use remember_preference::RememberPreferenceTool; #[cfg(feature = "skills")] pub use run_workflow::{ AwaitWorkflowTool, RunWorkflowTool, AWAIT_WORKFLOW_TOOL_NAME, RUN_WORKFLOW_TOOL_NAME, }; +#[cfg(feature = "memory")] pub use save_preference::SavePreferenceTool; pub use todo::TodoTool; pub use update_task::UpdateTaskTool; diff --git a/src/openhuman/channels/context.rs b/src/openhuman/channels/context.rs index c4bc4c96a06..3fe41693fd5 100644 --- a/src/openhuman/channels/context.rs +++ b/src/openhuman/channels/context.rs @@ -33,6 +33,7 @@ pub(crate) struct ChannelRuntimeContext { /// Production contexts carry `config` and construct crate-native sources. pub(crate) turn_model_source: Option, pub(crate) default_provider: Arc, + #[cfg(feature = "memory")] pub(crate) memory: Arc, pub(crate) tools_registry: Arc>>, pub(crate) system_prompt: Arc, @@ -109,6 +110,7 @@ pub(crate) fn is_context_window_overflow_error(err: &anyhow::Error) -> bool { use tinymemory_api::provider::MemoryRecall as _; +#[cfg(feature = "memory")] pub(crate) async fn build_memory_context( mem: &crate::openhuman::memory::guard::MemoryGuard, user_msg: &str, @@ -171,7 +173,7 @@ pub(crate) async fn build_memory_context( context } -#[cfg(test)] +#[cfg(all(test, feature = "memory"))] mod tests { use super::*; use crate::openhuman::channels::traits; diff --git a/src/openhuman/channels/runtime/dispatch/processor.rs b/src/openhuman/channels/runtime/dispatch/processor.rs index cde3b6db8f5..71f3654aa7f 100644 --- a/src/openhuman/channels/runtime/dispatch/processor.rs +++ b/src/openhuman/channels/runtime/dispatch/processor.rs @@ -15,9 +15,11 @@ use crate::core::events::DomainEvent; use crate::openhuman::agent::bus::{AgentTurnRequest, AgentTurnResponse, AGENT_RUN_TURN_METHOD}; use crate::openhuman::agent::messages::ChatMessage; use crate::openhuman::agent::progress::AgentProgress; +#[cfg(feature = "memory")] +use crate::openhuman::channels::context::build_memory_context; use crate::openhuman::channels::context::{ - build_memory_context, compact_sender_history, conversation_history_key, - conversation_memory_key, is_context_window_overflow_error, ChannelRuntimeContext, + compact_sender_history, conversation_history_key, conversation_memory_key, + is_context_window_overflow_error, ChannelRuntimeContext, }; use crate::openhuman::channels::providers::telegram::TELEGRAM_APPROVAL_CLIENT_ID; use crate::openhuman::channels::routes::{ @@ -265,15 +267,21 @@ pub(crate) async fn process_channel_runtime_message( None }; + #[cfg(feature = "memory")] let memory_context = build_memory_context(&ctx.memory, &msg.content, ctx.min_relevance_score).await; + // No driver, so nothing to prepend. The enrichment below treats an empty + // string as "no memory context", which is already its no-hits path. + #[cfg(not(feature = "memory"))] + let memory_context = String::new(); + #[cfg(feature = "memory")] if ctx.auto_save_memory { let autosave_key = conversation_memory_key(&msg); let _ = ctx .memory .store( - crate::openhuman::agent::learning::transcript_ingest::CONVERSATION_RAW_NAMESPACE, + crate::openhuman::memory::namespaces::CONVERSATION_RAW_NAMESPACE, &autosave_key, &msg.content, tinymemory_api::types::MemoryCategory::Conversation, diff --git a/src/openhuman/channels/runtime/startup.rs b/src/openhuman/channels/runtime/startup.rs index 63b9676d67c..a8dfc009ef6 100644 --- a/src/openhuman/channels/runtime/startup.rs +++ b/src/openhuman/channels/runtime/startup.rs @@ -173,11 +173,14 @@ pub async fn start_channels(mut config: Config) -> Result<()> { let bus = crate::core::bus::BUS.get().expect("bus initialised"); let _tracing_handle = bus.subscribe(Arc::new(crate::core::bus::TracingSubscriber)); crate::openhuman::platform::health::bus::register_health_subscriber(); - crate::openhuman::memory::conversations::register_conversation_persistence_subscriber( - config.workspace_dir.clone(), - ); - crate::openhuman::memory::sync_events_bridge::register_sync_stage_bridge(&config); - crate::openhuman::integrations::composio::register_composio_trigger_subscriber(); + #[cfg(feature = "memory")] + { + crate::openhuman::memory::conversations::register_conversation_persistence_subscriber( + config.workspace_dir.clone(), + ); + crate::openhuman::memory::sync_events_bridge::register_sync_stage_bridge(&config); + crate::openhuman::integrations::composio::register_composio_trigger_subscriber(); + } // Surface parked ApprovalGate requests as chat messages so the user can // answer yes/no in the thread (chat-native approval, issue #1339). crate::openhuman::web_chat::register_approval_surface_subscriber(); @@ -196,6 +199,7 @@ pub async fn start_channels(mut config: Config) -> Result<()> { // `bootstrap_core_runtime` may also start it — `start_periodic_sync` // is intentionally cheap and the loop body no-ops when there are // no connections. + #[cfg(feature = "memory")] crate::openhuman::integrations::composio::start_periodic_sync(); // Task-sources: subscribe to Composio connection-created events for // one-shot fetches, and spawn the periodic poll that pulls work from @@ -793,6 +797,7 @@ pub async fn start_channels(mut config: Config) -> Result<()> { None }; // Register the tree summarizer event subscriber for observability logging. + #[cfg(feature = "memory")] let _tree_summarizer_handle = bus.subscribe(Arc::new( crate::openhuman::memory::tree::tree_runtime::bus::TreeSummarizerEventSubscriber::new(), )); @@ -809,6 +814,7 @@ pub async fn start_channels(mut config: Config) -> Result<()> { channels_by_name, turn_model_source: None, default_provider: Arc::new(provider_name), + #[cfg(feature = "memory")] memory: crate::openhuman::memory::ops::guard::active_memory_guard() .await .map_err(|e| anyhow::anyhow!("channels startup: memory unavailable: {e}"))?, diff --git a/src/openhuman/channels/runtime/test_support.rs b/src/openhuman/channels/runtime/test_support.rs index 63b323ea93c..168899fcb47 100644 --- a/src/openhuman/channels/runtime/test_support.rs +++ b/src/openhuman/channels/runtime/test_support.rs @@ -492,6 +492,7 @@ pub async fn run_dispatch_harness(options: DispatchHarnessOptions) -> DispatchHa crate::openhuman::agent::tinyagents::TurnModelSource::from_model(model), ), default_provider: Arc::new("harness-provider".to_string()), + #[cfg(feature = "memory")] memory: crate::openhuman::memory::guard::in_memory::guard_over(Arc::new(HarnessMemory { entries: options .memory_entries diff --git a/src/openhuman/config/schema/types.rs b/src/openhuman/config/schema/types.rs index ba2b12859a9..109b40f9d7a 100644 --- a/src/openhuman/config/schema/types.rs +++ b/src/openhuman/config/schema/types.rs @@ -351,7 +351,7 @@ pub struct Config { /// describes a data connector (Composio OAuth, local folder, GitHub /// repo, RSS feed, Twitter query, web page) that feeds memory. #[serde(default)] - pub memory_sources: Vec, + pub memory_sources: Vec, /// User-facing agent registry — shipped default agents plus user-authored /// custom agents and persisted enable/disable/tool-policy overrides. diff --git a/src/openhuman/desktop/app_state/ops.rs b/src/openhuman/desktop/app_state/ops.rs index 9353f65e300..b1f750ffffd 100644 --- a/src/openhuman/desktop/app_state/ops.rs +++ b/src/openhuman/desktop/app_state/ops.rs @@ -689,6 +689,7 @@ async fn finish_revalidated_user_activation( // No people-store rebind: people is served by the bound memory driver, and // the core-context rebind above already moved that binding to the activated // user's workspace. + #[cfg(feature = "memory")] crate::openhuman::memory::conversations::register_conversation_persistence_subscriber( target_config.workspace_dir.clone(), ); diff --git a/src/openhuman/desktop/provider_surfaces/ops.rs b/src/openhuman/desktop/provider_surfaces/ops.rs index f0b2c3ec8c1..a3e86c2450d 100644 --- a/src/openhuman/desktop/provider_surfaces/ops.rs +++ b/src/openhuman/desktop/provider_surfaces/ops.rs @@ -3,10 +3,10 @@ //! This initial cut keeps state in-memory so the RPC contract and UI wiring //! can land before the SQLite-backed store arrives. -use crate::openhuman::memory::{ApiEnvelope, ApiMeta, EmptyRequest}; use crate::rpc::RpcOutcome; use serde::Serialize; use std::collections::BTreeMap; +use tinymemory_core::rpc_models::{ApiEnvelope, ApiMeta, EmptyRequest}; use super::store; use super::types::{ProviderEvent, RespondQueueItem, RespondQueueListResponse}; diff --git a/src/openhuman/desktop/provider_surfaces/schemas.rs b/src/openhuman/desktop/provider_surfaces/schemas.rs index 9024d9edbac..42fc002716a 100644 --- a/src/openhuman/desktop/provider_surfaces/schemas.rs +++ b/src/openhuman/desktop/provider_surfaces/schemas.rs @@ -8,7 +8,7 @@ use serde_json::{Map, Value}; use crate::core::all::RegisteredController; use crate::core::{ControllerSchema, FieldSchema, TypeSchema}; -use crate::openhuman::memory::EmptyRequest; +use tinymemory_core::rpc_models::EmptyRequest; use super::ops; use super::types::ProviderEvent; diff --git a/src/openhuman/flows/bus.rs b/src/openhuman/flows/bus.rs index b6e12c9e0fa..0164cc81043 100644 --- a/src/openhuman/flows/bus.rs +++ b/src/openhuman/flows/bus.rs @@ -316,6 +316,7 @@ const DIGEST_RETENTION_CAP: usize = 50; /// and swallowed, never propagated — by the time this subscriber observes /// `FlowRunFinished`, the run has already settled its own `flow_runs` row, so /// a memory-layer hiccup must never retroactively affect run status. +#[cfg(feature = "memory")] pub struct FlowRunDigestSubscriber { config: Arc, /// Test-only memory override. In production this is `None` and the digest @@ -330,6 +331,7 @@ pub struct FlowRunDigestSubscriber { memory_override: Option>, } +#[cfg(feature = "memory")] impl FlowRunDigestSubscriber { pub fn new(config: Arc) -> Self { Self { @@ -463,6 +465,7 @@ impl FlowRunDigestSubscriber { } } +#[cfg(feature = "memory")] #[async_trait] impl EventHandler for FlowRunDigestSubscriber { fn name(&self) -> &str { diff --git a/src/openhuman/flows/memory_tools.rs b/src/openhuman/flows/memory_tools.rs index 3fc027f2532..46b2a449a32 100644 --- a/src/openhuman/flows/memory_tools.rs +++ b/src/openhuman/flows/memory_tools.rs @@ -75,54 +75,13 @@ fn trusted_flow_id() -> Option { } } -/// Prefix for a flow's private, sandboxed memory namespace (see -/// [`flow_namespace`]). -/// -/// **Deviates from the originally specced `"flow:"` (colon) separator — -/// deliberately.** The `Memory` trait's `UnifiedMemory` backend -/// (`src/openhuman/memory/store/`) is internally inconsistent about -/// namespace sanitization: `store_with_taint`/`recall`/`list`/ -/// `MemoryClient::clear_namespace` all route through -/// `UnifiedMemory::sanitize_namespace` -/// (`memory_store/namespace_store/init.rs`), which collapses any character -/// outside `[A-Za-z0-9_/-]` — including `:` — to `_` before touching SQLite. -/// But `Memory::forget` (`memory_store/memory_trait.rs`) queries -/// `WHERE namespace = ?1` against the **raw, unsanitized** argument. With a -/// `"flow:"` prefix, `forget("flow:", key)` would therefore silently -/// never match the row `store_with_taint` actually persisted as -/// `"flow_"` — the post-run digest subscriber's retention sweep -/// (`bus::FlowRunDigestSubscriber`) would then never evict old entries, and -/// `namespace_summaries()`-based cross-flow listing (`scope: "flows"` in -/// [`FlowMemoryRecallTool`]) would have to match the sanitized form anyway -/// since `namespace_summaries` reads the persisted (sanitized) column back -/// verbatim. Using `"flow_"` (already a fixed point of `sanitize_namespace`, -/// since flow ids are hyphenated UUIDs — no character in either the prefix -/// or a flow id ever needs sanitizing) makes every `Memory` method agree -/// with every other one on the exact namespace string, with no silent -/// mismatch anywhere. The namespace is still shared-root and -/// profile-independent exactly as specified — only the separator character -/// changed. -/// -/// Re-exported from `flows::mod` as `flows::FLOW_MEMORY_NAMESPACE_PREFIX` — -/// see that module for why this lives here rather than in `mod.rs` itself. -pub const FLOW_MEMORY_NAMESPACE_PREFIX: &str = "flow_"; - -/// Builds a flow's private, profile-independent memory namespace from a -/// `flow_id`. -/// -/// **Security invariant:** this is the *only* place in the codebase that may -/// construct this namespace string. Every caller — the `flow_memory_recall` -/// / `flow_memory_remember` agent tools below, the post-run digest -/// subscriber (`bus::FlowRunDigestSubscriber`), and the `flows_delete` -/// cleanup hook (`ops::flows_delete`) — goes through this function with a -/// `flow_id`, never with a caller-supplied raw namespace. A flow can -/// therefore never write to, or be told the name of, any memory namespace -/// but its own. -/// -/// Re-exported from `flows::mod` as `flows::flow_namespace`. -pub fn flow_namespace(flow_id: &str) -> String { - format!("{FLOW_MEMORY_NAMESPACE_PREFIX}{flow_id}") -} +// The naming rule and its prefix moved to the ungated `flows::namespace`: +// they are a `format!` with no driver behind them, and `flows` still needs +// them in a build with the `memory` family compiled out. Re-exported here so +// every historical `memory_tools::…` path keeps resolving — and so the +// security invariant on `flow_namespace` still has exactly ONE definition +// behind it, which is the whole point of that invariant. +pub use crate::openhuman::flows::namespace::{flow_namespace, FLOW_MEMORY_NAMESPACE_PREFIX}; /// The persisted-namespace prefix matching every flow's memory namespace, as /// [`Memory::namespace_summaries`] returns it. diff --git a/src/openhuman/flows/mod.rs b/src/openhuman/flows/mod.rs index 636c50c8180..c07dc28f26f 100644 --- a/src/openhuman/flows/mod.rs +++ b/src/openhuman/flows/mod.rs @@ -35,8 +35,13 @@ pub mod bus; pub mod discovery_tools; mod draft_store; pub mod medulla_bridge; +#[cfg(feature = "memory")] pub mod memory_tools; mod n8n_import; +/// A flow's private memory namespace naming rule. Ungated: it is a `format!` +/// with no driver behind it, and `flows` needs it whether or not the memory +/// family is compiled in. See the module docs. +pub mod namespace; pub mod node_contracts; pub mod ops; mod run_registry; @@ -80,4 +85,6 @@ pub use types::{ // see byte-identical cross-flow results to `flow_memory_recall`'s own // `scope: "flows"` arm, so both call the one implementation here rather than // each walking `namespace_summaries` independently. -pub use memory_tools::{cross_flow_recall, flow_namespace, FLOW_MEMORY_NAMESPACE_PREFIX}; +#[cfg(feature = "memory")] +pub use memory_tools::cross_flow_recall; +pub use namespace::{flow_namespace, FLOW_MEMORY_NAMESPACE_PREFIX}; diff --git a/src/openhuman/flows/namespace.rs b/src/openhuman/flows/namespace.rs new file mode 100644 index 00000000000..81e93ddf989 --- /dev/null +++ b/src/openhuman/flows/namespace.rs @@ -0,0 +1,41 @@ +//! A flow's private memory namespace — the naming rule, on its own. +//! +//! # Why this is not in `memory_tools` +//! +//! It was, and it is the same code. What moved is only *where*, and for one +//! reason: the naming rule is a pure `format!` over a `flow_id`, while +//! everything else in `memory_tools` needs a driver. When the `memory` family +//! is compiled out the tools go with it, but `flows` still needs the prefix — +//! `flows_delete`'s cleanup and the run-digest subscriber both name it — so +//! keeping the rule in the gated module would have forced either a stub twin +//! of a `format!` call or a `#[cfg]` on every caller. +//! +//! This is `skills::types`' rule again: a domain's inert parts live in a +//! dependency-free submodule and stay ungated; only the behaviour is gated. +//! +//! # The security invariant is unchanged +//! +//! [`flow_namespace`] is still the **only** place in the codebase that may +//! construct this namespace string, and moving it kept that true — there is +//! one definition, not two. Every caller (the `flow_memory_recall` / +//! `flow_memory_remember` agent tools, the post-run digest subscriber in +//! `flows::bus`, and the `flows_delete` cleanup hook) goes through this +//! function with a `flow_id`, never with a caller-supplied raw namespace. A +//! flow can therefore never write to, or be told the name of, any memory +//! namespace but its own. + +/// Prefix on every flow's private memory namespace. +/// +/// Underscore rather than a colon so the string survives round-tripping +/// through every `Memory` method that treats a colon as a separator: one +/// prefix, agreed on by recall, list and clear, with no silent mismatch. +pub const FLOW_MEMORY_NAMESPACE_PREFIX: &str = "flow_"; + +/// Builds a flow's private, profile-independent memory namespace from a +/// `flow_id`. +/// +/// See the module docs for the security invariant this function carries. +#[must_use] +pub fn flow_namespace(flow_id: &str) -> String { + format!("{FLOW_MEMORY_NAMESPACE_PREFIX}{flow_id}") +} diff --git a/src/openhuman/flows/tinyflows/caps/ops.rs b/src/openhuman/flows/tinyflows/caps/ops.rs index e2210a78de5..9a90d711389 100644 --- a/src/openhuman/flows/tinyflows/caps/ops.rs +++ b/src/openhuman/flows/tinyflows/caps/ops.rs @@ -155,7 +155,7 @@ async fn flow_tool_allowed( slug: &str, connected_toolkits: Option<&[String]>, ) -> bool { - use crate::openhuman::memory::sync::composio::providers::{ + use tinymemory_core::sync::composio::providers::{ catalog_for_toolkit, classify_unknown, find_curated, get_provider, load_user_scope_or_default, toolkit_from_slug, }; @@ -228,7 +228,7 @@ async fn flow_tool_allowed( /// offline (a registry lookup) so the common cataloged-toolkit path never pays /// for a connected-set fetch. fn slug_needs_connected_set(slug: &str) -> bool { - use crate::openhuman::memory::sync::composio::providers::{ + use tinymemory_core::sync::composio::providers::{ catalog_for_toolkit, get_provider, toolkit_from_slug, }; match toolkit_from_slug(slug) { @@ -283,7 +283,7 @@ async fn connected_toolkit_slugs(config: &Config) -> Option> { /// [`CommandClass`] the autonomy-tier gate ([`enforce_node_tier_gate`]) /// evaluates it under. /// -/// Reuses [`curated_scope_for`](crate::openhuman::memory::sync::composio::providers::curated_scope_for), +/// Reuses [`curated_scope_for`](tinymemory_core::sync::composio::providers::curated_scope_for), /// the same catalog walk `composio::ops`'s `gated_tools` hints use — a /// registered native provider's `curated_tools()` first, then the static /// `catalog_for_toolkit` fallback. **Fail-safe by construction:** only a @@ -296,7 +296,7 @@ async fn connected_toolkit_slugs(config: &Config) -> Option> { /// (prompts under Supervised/Full, blocks under ReadOnly). /// /// Deliberately does **not** fall back to -/// [`classify_unknown`](crate::openhuman::memory::sync::composio::providers::classify_unknown) +/// [`classify_unknown`](tinymemory_core::sync::composio::providers::classify_unknown) /// for uncurated slugs: that heuristic is tuned for the *curation* /// allowlist (`flow_tool_allowed`'s Path B — "is this slug even visible to /// the agent"), not for deciding whether a real side-effecting call skips @@ -307,7 +307,7 @@ async fn connected_toolkit_slugs(config: &Config) -> Option> { /// from what actually gates (a parallel re-implementation would list /// permissions that never prompt, or miss ones that do). pub(crate) async fn classify_composio_action_for_tier(slug: &str) -> CommandClass { - use crate::openhuman::memory::sync::composio::providers::{curated_scope_for, ToolScope}; + use tinymemory_core::sync::composio::providers::{curated_scope_for, ToolScope}; match curated_scope_for(slug) { Some(ToolScope::Read) => CommandClass::Read, @@ -648,12 +648,18 @@ pub fn build_capabilities(config: Arc, state_namespace: impl Into Option> { - use crate::openhuman::memory::sync::composio::providers::{ + use tinymemory_core::sync::composio::providers::{ catalog_for_toolkit, find_curated, get_provider, }; @@ -599,8 +599,8 @@ pub(crate) fn apply_probe_override(mut contract: ToolContract) -> ToolContract { /// "assume Read". fn resolve_composio_action_scope( slug: &str, -) -> Option { - use crate::openhuman::memory::sync::composio::providers::{ +) -> Option { + use tinymemory_core::sync::composio::providers::{ catalog_for_toolkit, classify_unknown, find_curated, get_provider, toolkit_from_slug, }; @@ -647,7 +647,7 @@ pub(crate) async fn probe_tool_output_sample( } match resolve_composio_action_scope(slug) { - Some(crate::openhuman::memory::sync::composio::providers::ToolScope::Read) => {} + Some(tinymemory_core::sync::composio::providers::ToolScope::Read) => {} Some(other) => { tracing::warn!( target: "flows", @@ -682,9 +682,7 @@ pub(crate) async fn probe_tool_output_sample( } } - let Some(toolkit) = - crate::openhuman::memory::sync::composio::providers::toolkit_from_slug(slug) - else { + let Some(toolkit) = tinymemory_core::sync::composio::providers::toolkit_from_slug(slug) else { return Err(format!( "get_tool_output_sample: could not extract a toolkit from slug '{slug}' — it must \ look like '_'." @@ -775,7 +773,7 @@ pub(crate) async fn probe_tool_output_sample( /// in the toolkit's live catalog — so callers can skip the preflight rather /// than block execution on a catalog hiccup. pub(crate) async fn composio_required_args(config: &Config, slug: &str) -> Option> { - let toolkit = crate::openhuman::memory::sync::composio::providers::toolkit_from_slug(slug)?; + let toolkit = tinymemory_core::sync::composio::providers::toolkit_from_slug(slug)?; let contracts = fetch_live_toolkit_catalog(config, &toolkit).await?; contracts .iter() @@ -796,7 +794,7 @@ pub(crate) async fn composio_required_args(config: &Config, slug: &str) -> Optio /// unknown" rather than blocking or guessing. `Some(vec![])` means the /// schema was found but names no top-level properties. pub(crate) async fn composio_response_fields(config: &Config, slug: &str) -> Option> { - let toolkit = crate::openhuman::memory::sync::composio::providers::toolkit_from_slug(slug)?; + let toolkit = tinymemory_core::sync::composio::providers::toolkit_from_slug(slug)?; let contracts = fetch_live_toolkit_catalog(config, &toolkit).await?; let contract = contracts .iter() diff --git a/src/openhuman/integrations/composio/catalog_tests.rs b/src/openhuman/integrations/composio/catalog_tests.rs index 869d855fce7..c2654b0422b 100644 --- a/src/openhuman/integrations/composio/catalog_tests.rs +++ b/src/openhuman/integrations/composio/catalog_tests.rs @@ -255,7 +255,7 @@ fn cache_probe_result_redacts_the_raw_sample_before_caching() { #[test] fn resolve_composio_action_scope_uses_the_curated_catalog_when_available() { - use crate::openhuman::memory::sync::composio::providers::ToolScope; + use tinymemory_core::sync::composio::providers::ToolScope; // GITHUB_LIST_REPOSITORY_ISSUES is curated as Read (github/tools.rs). assert_eq!( resolve_composio_action_scope("GITHUB_LIST_REPOSITORY_ISSUES"), @@ -285,7 +285,7 @@ fn resolve_composio_action_scope_rejects_an_uncurated_slug_on_a_cataloged_toolki #[test] fn resolve_composio_action_scope_falls_back_to_the_verb_heuristic_only_without_a_static_catalog() { - use crate::openhuman::memory::sync::composio::providers::ToolScope; + use tinymemory_core::sync::composio::providers::ToolScope; assert_eq!( resolve_composio_action_scope("MADEUPTOOLKIT_LIST_THINGS"), Some(ToolScope::Read) diff --git a/src/openhuman/integrations/composio/identity.rs b/src/openhuman/integrations/composio/identity.rs index bd60c653be0..dd6d2868184 100644 --- a/src/openhuman/integrations/composio/identity.rs +++ b/src/openhuman/integrations/composio/identity.rs @@ -7,7 +7,7 @@ //! `git config user.name`). //! //! The lookup goes through the per-toolkit -//! [`ComposioProvider::fetch_user_profile`](crate::openhuman::memory::sync::composio::providers::ComposioProvider::fetch_user_profile) +//! [`ComposioProvider::fetch_user_profile`](tinymemory_core::sync::composio::providers::ComposioProvider::fetch_user_profile) //! call, which already knows the right Composio action slug for each //! toolkit (`GITHUB_GET_THE_AUTHENTICATED_USER`, //! `GMAIL_GET_PROFILE`, …) and the JSON field that holds the username. @@ -115,11 +115,11 @@ pub async fn connection_identity(config: &Config, toolkit: &str) -> Option OpResult> { tracing::debug!(connection_id = %connection_id, "[composio] rpc delete_connection"); let client = resolve_client(config)?; + #[cfg(feature = "memory")] let toolkit = match resolve_toolkit_for_connection(&client, connection_id).await { Ok(toolkit) => Some(toolkit), Err(error) if clear_memory => { @@ -162,6 +163,16 @@ pub async fn composio_delete_connection( } Err(_) => None, }; + #[cfg(not(feature = "memory"))] + let toolkit = resolve_toolkit_for_connection(&client, connection_id) + .await + .ok(); + #[cfg(not(feature = "memory"))] + let memory_targets: Vec = { + let _ = clear_memory; + Vec::new() + }; + #[cfg(feature = "memory")] let memory_targets = if clear_memory { // Target discovery takes the config and resolves the bound driver // itself — the notion arm reads sync state through the driver's `Graph` @@ -223,6 +234,7 @@ pub async fn composio_delete_connection( ); } } + #[cfg(feature = "memory")] match crate::openhuman::memory::sources::registry::remove_composio_source_by_connection_id( connection_id, ) diff --git a/src/openhuman/integrations/composio/ops/mod.rs b/src/openhuman/integrations/composio/ops/mod.rs index b4795a8f4b9..2a4fe6158a9 100644 --- a/src/openhuman/integrations/composio/ops/mod.rs +++ b/src/openhuman/integrations/composio/ops/mod.rs @@ -90,10 +90,6 @@ pub(crate) use super::connected_integrations::sync_cache_with_connections; #[cfg(test)] pub(crate) use crate::openhuman::config::Config; #[cfg(test)] -pub(crate) use crate::openhuman::memory::sync::composio::providers::sync_state::SyncState; -#[cfg(test)] -pub(crate) use crate::openhuman::memory::sync::composio::providers::SyncReason; -#[cfg(test)] pub(crate) use connections::enrich_connections_with_identity; #[cfg(test)] pub(crate) use error_utils::{ @@ -106,6 +102,10 @@ pub(crate) use memory_cleanup::{composio_memory_targets_for_connection, MemoryCl pub(crate) use providers_ops::parse_sync_reason; #[cfg(test)] pub(crate) use tinymemory_core::store::MemoryClient; +#[cfg(test)] +pub(crate) use tinymemory_core::sync::composio::providers::sync_state::SyncState; +#[cfg(test)] +pub(crate) use tinymemory_core::sync::composio::providers::SyncReason; #[cfg(test)] #[path = "../ops_tests.rs"] diff --git a/src/openhuman/integrations/composio/ops_tests.rs b/src/openhuman/integrations/composio/ops_tests.rs index 27fbdb7c638..279555e1d14 100644 --- a/src/openhuman/integrations/composio/ops_tests.rs +++ b/src/openhuman/integrations/composio/ops_tests.rs @@ -2588,7 +2588,7 @@ async fn enrich_populates_email_from_cached_profile() { // `Once`-guarded, so a test that omits it passes only while some // earlier test in the same binary happened to run first. crate::openhuman::memory::host_impls::install_for_tests(); - use crate::openhuman::memory::sync::composio::providers::{ + use tinymemory_core::sync::composio::providers::{ profile::persist_provider_profile, ProviderUserProfile, }; let tmp = tempfile::tempdir().unwrap(); @@ -2628,7 +2628,7 @@ async fn enrich_populates_handle_for_github() { // `Once`-guarded, so a test that omits it passes only while some // earlier test in the same binary happened to run first. crate::openhuman::memory::host_impls::install_for_tests(); - use crate::openhuman::memory::sync::composio::providers::{ + use tinymemory_core::sync::composio::providers::{ profile::persist_provider_profile, ProviderUserProfile, }; let tmp = tempfile::tempdir().unwrap(); @@ -2676,7 +2676,7 @@ async fn enrich_handles_multiple_connections_same_toolkit() { // earlier test in the same binary happened to run first. crate::openhuman::memory::host_impls::install_for_tests(); // Two Gmail accounts — each gets its own identity label, not "Account N". - use crate::openhuman::memory::sync::composio::providers::{ + use tinymemory_core::sync::composio::providers::{ profile::persist_provider_profile, ProviderUserProfile, }; let tmp = tempfile::tempdir().unwrap(); @@ -2722,7 +2722,7 @@ async fn enrich_leaves_unmatched_connection_unchanged() { crate::openhuman::memory::host_impls::install_for_tests(); // Connection whose id has no cached profile row is returned with all // identity fields as None — the UI falls back to "toolkit · connection_id". - use crate::openhuman::memory::sync::composio::providers::{ + use tinymemory_core::sync::composio::providers::{ profile::persist_provider_profile, ProviderUserProfile, }; let tmp = tempfile::tempdir().unwrap(); diff --git a/src/openhuman/integrations/composio/periodic.rs b/src/openhuman/integrations/composio/periodic.rs index cee76d1b089..114d8e2bb01 100644 --- a/src/openhuman/integrations/composio/periodic.rs +++ b/src/openhuman/integrations/composio/periodic.rs @@ -1,4 +1,5 @@ //! Compatibility shim: composio periodic sync now lives in //! `memory_sync::composio::periodic`. +#[cfg(feature = "memory")] pub use crate::openhuman::memory::sync::composio::periodic::*; diff --git a/src/openhuman/integrations/composio/providers/mod.rs b/src/openhuman/integrations/composio/providers/mod.rs index 1fd19130f11..1beeeb6aa69 100644 --- a/src/openhuman/integrations/composio/providers/mod.rs +++ b/src/openhuman/integrations/composio/providers/mod.rs @@ -1,4 +1,4 @@ //! Compatibility shim: composio native provider sync code now lives in //! `memory_sync::composio::providers`. -pub use crate::openhuman::memory::sync::composio::providers::*; +pub use tinymemory_core::sync::composio::providers::*; diff --git a/src/openhuman/integrations/task_sources/filter.rs b/src/openhuman/integrations/task_sources/filter.rs index 5851e9b6091..960d8806622 100644 --- a/src/openhuman/integrations/task_sources/filter.rs +++ b/src/openhuman/integrations/task_sources/filter.rs @@ -6,7 +6,7 @@ //! `FilterSpec` variant into the shared filter envelope and stamps the //! per-fetch cap. -use crate::openhuman::memory::sync::composio::providers::TaskFetchFilter; +use tinymemory_core::sync::composio::providers::TaskFetchFilter; use super::types::FilterSpec; diff --git a/src/openhuman/integrations/task_sources/mod.rs b/src/openhuman/integrations/task_sources/mod.rs index 9fb5e1c2b72..7a43505149d 100644 --- a/src/openhuman/integrations/task_sources/mod.rs +++ b/src/openhuman/integrations/task_sources/mod.rs @@ -13,7 +13,7 @@ //! logic lives in the sibling modules, persistence is SQLite in //! `store.rs`, and the RPC surface is wired through `schemas.rs`. //! -//! [`fetch_tasks`]: crate::openhuman::memory::sync::composio::providers::ComposioProvider::fetch_tasks +//! [`fetch_tasks`]: tinymemory_core::sync::composio::providers::ComposioProvider::fetch_tasks pub mod bus; pub mod enrich; @@ -27,9 +27,6 @@ pub mod store; pub mod tools; pub mod types; -pub use crate::openhuman::memory::sync::composio::providers::{ - NormalizedTask, TaskContainer, TaskFetchFilter, TaskKind, -}; pub use periodic::start_periodic_poll; pub use pipeline::run_source_once; pub use route::TASK_SOURCES_THREAD_ID; @@ -38,6 +35,9 @@ pub use schemas::{ all_registered_controllers as all_task_sources_registered_controllers, schemas as task_sources_schemas, }; +pub use tinymemory_core::sync::composio::providers::{ + NormalizedTask, TaskContainer, TaskFetchFilter, TaskKind, +}; pub use types::{ EnrichedTask, FetchOutcome, FetchReason, FilterSpec, ProviderSlug, SourceTarget, TaskSource, TaskSourcePatch, diff --git a/src/openhuman/integrations/task_sources/ops.rs b/src/openhuman/integrations/task_sources/ops.rs index d3943440967..0a7393dca57 100644 --- a/src/openhuman/integrations/task_sources/ops.rs +++ b/src/openhuman/integrations/task_sources/ops.rs @@ -10,10 +10,10 @@ use std::sync::Arc; use serde_json::{json, Value}; use crate::openhuman::config::Config; -use crate::openhuman::memory::sync::composio::providers::{ +use crate::rpc::RpcOutcome; +use tinymemory_core::sync::composio::providers::{ get_provider, NormalizedTask, ProviderContext, TaskContainer, }; -use crate::rpc::RpcOutcome; use super::types::{ FetchReason, FilterSpec, ProviderSlug, SourceTarget, TaskSource, TaskSourcePatch, diff --git a/src/openhuman/integrations/task_sources/pipeline.rs b/src/openhuman/integrations/task_sources/pipeline.rs index e722e1fd234..c4e27cc4ee2 100644 --- a/src/openhuman/integrations/task_sources/pipeline.rs +++ b/src/openhuman/integrations/task_sources/pipeline.rs @@ -14,7 +14,7 @@ use std::collections::HashSet; use crate::core::bus::BUS; use crate::core::events::DomainEvent; use crate::openhuman::config::Config; -use crate::openhuman::memory::sync::composio::providers::{get_provider, ProviderContext}; +use tinymemory_core::sync::composio::providers::{get_provider, ProviderContext}; use super::types::{FetchOutcome, FetchReason, TaskSource}; use super::{enrich, filter, route, store}; diff --git a/src/openhuman/integrations/task_sources/pipeline_tests.rs b/src/openhuman/integrations/task_sources/pipeline_tests.rs index 624e3c0516a..f3a23faffe4 100644 --- a/src/openhuman/integrations/task_sources/pipeline_tests.rs +++ b/src/openhuman/integrations/task_sources/pipeline_tests.rs @@ -2,14 +2,14 @@ use super::*; use crate::openhuman::config::Config; use crate::openhuman::integrations::task_sources::store; use crate::openhuman::integrations::task_sources::types::{FilterSpec, ProviderSlug, SourceTarget}; -use crate::openhuman::memory::sync::composio::providers::{ - register_provider, ComposioProvider, NormalizedTask, ProviderContext, ProviderUserProfile, - TaskFetchFilter, -}; use async_trait::async_trait; use serde_json::json; use std::sync::{Arc, Mutex, MutexGuard, OnceLock}; use tempfile::TempDir; +use tinymemory_core::sync::composio::providers::{ + register_provider, ComposioProvider, NormalizedTask, ProviderContext, ProviderUserProfile, + TaskFetchFilter, +}; /// Serialize pipeline tests: they register a stub provider under the /// shared "github" registry slug, so they must not run concurrently. diff --git a/src/openhuman/integrations/task_sources/store.rs b/src/openhuman/integrations/task_sources/store.rs index 62a1e6f7200..cd28e412aa0 100644 --- a/src/openhuman/integrations/task_sources/store.rs +++ b/src/openhuman/integrations/task_sources/store.rs @@ -19,7 +19,7 @@ use sha2::{Digest, Sha256}; use uuid::Uuid; use crate::openhuman::config::Config; -use crate::openhuman::memory::sync::composio::providers::NormalizedTask; +use tinymemory_core::sync::composio::providers::NormalizedTask; use super::types::{ FetchReason, FilterSpec, ProviderSlug, SourceTarget, TaskSource, TaskSourcePatch, diff --git a/src/openhuman/integrations/task_sources/types.rs b/src/openhuman/integrations/task_sources/types.rs index 663475bcec9..826baaaf395 100644 --- a/src/openhuman/integrations/task_sources/types.rs +++ b/src/openhuman/integrations/task_sources/types.rs @@ -64,7 +64,7 @@ pub enum FilterSpec { /// How to fetch: Composio connection, local `gh`/REST, or `auto` /// (Composio-first with local fallback). Defaults to `auto`. #[serde(default)] - fetch_mode: crate::openhuman::memory::sync::composio::providers::GithubFetchMode, + fetch_mode: tinymemory_core::sync::composio::providers::GithubFetchMode, #[serde(default)] extra: Value, }, diff --git a/src/openhuman/memory/agent/memory_loader.rs b/src/openhuman/memory/agent/memory_loader.rs index 52445105576..4e68dd9e8dc 100644 --- a/src/openhuman/memory/agent/memory_loader.rs +++ b/src/openhuman/memory/agent/memory_loader.rs @@ -1,5 +1,4 @@ use crate::openhuman::memory::Memory; -use serde::{Deserialize, Serialize}; /// Maximum number of `[Prior conversations]` lines surfaced into the prompt /// at the start of a fresh chat. Tight cap on purpose: this block is meant @@ -27,24 +26,11 @@ const PRIOR_CONVERSATION_KEY_PREFIX: &str = "high."; /// prompt section that names the header verbatim. Tests assert on this /// constant too — see `memory_loader::tests` and /// `harness::memory_context::tests`. -pub const CROSS_CHAT_HEADER: &str = - "[Cross-chat context — historical; capabilities may have changed since]\n"; - -/// Lightweight citation object derived from recalled memory entries. -/// -/// These citations are attached to agent responses so the UI can show -/// provenance for memory-informed answers without exposing full raw memory. -#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] -pub struct MemoryCitation { - pub id: String, - pub key: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub namespace: Option, - #[serde(skip_serializing_if = "Option::is_none")] - pub score: Option, - pub timestamp: String, - pub snippet: String, -} +// Both live in the ungated `memory::citation` and are re-exported here so +// every historical `memory::agent::memory_loader::…` path keeps resolving. +// They are inert types in always-compiled signatures; the *producer* below is +// the part that needs a driver. See that module's docs. +pub use crate::openhuman::memory::citation::{MemoryCitation, CROSS_CHAT_HEADER}; /// Collect citation metadata from semantic memory recall for a user turn. /// diff --git a/src/openhuman/memory/citation.rs b/src/openhuman/memory/citation.rs new file mode 100644 index 00000000000..17fde76e44a --- /dev/null +++ b/src/openhuman/memory/citation.rs @@ -0,0 +1,46 @@ +//! Memory-recall provenance, as an inert type. +//! +//! # Why this is its own ungated module +//! +//! [`MemoryCitation`] is a plain serde struct and [`CROSS_CHAT_HEADER`] a +//! `&'static str`. Neither names a driver, a store, or the engine crate, and +//! neither can fail — but both appear in **always-compiled** signatures: the +//! agent's `last_turn_citations` field, its `pending_citations` join handle, +//! and the web-chat delivery path that renders source chips. +//! +//! So they follow `skills::types`' rule rather than `voice`'s: put a domain's +//! inert types in a dependency-free submodule and leave it **ungated**; stub +//! only the behaviour. Both builds then share ONE definition, and there is no +//! twin free to drift. The producer — `collect_recall_citations`, which is a +//! full recall — stays behind the gate in +//! [`memory_loader`](super::agent::memory_loader), which re-exports these two +//! so every historical path keeps resolving. +//! +//! With the family compiled out the vector is simply always empty: no recall +//! ran, so nothing cited anything. + +use serde::{Deserialize, Serialize}; + +/// Prefix stamped on recalled context drawn from a *different* conversation. +/// +/// The disclaimer is load-bearing rather than cosmetic: cross-chat context is +/// historical, and a capability the assistant had in the earlier conversation +/// may since have been removed. +pub const CROSS_CHAT_HEADER: &str = + "[Cross-chat context — historical; capabilities may have changed since]\n"; + +/// Lightweight citation object derived from recalled memory entries. +/// +/// Attached to agent responses so the UI can show provenance for +/// memory-informed answers without exposing full raw memory. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] +pub struct MemoryCitation { + pub id: String, + pub key: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub namespace: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub score: Option, + pub timestamp: String, + pub snippet: String, +} diff --git a/src/openhuman/memory/guard/in_memory.rs b/src/openhuman/memory/guard/in_memory.rs index 2db072db554..a48c294a360 100644 --- a/src/openhuman/memory/guard/in_memory.rs +++ b/src/openhuman/memory/guard/in_memory.rs @@ -38,11 +38,15 @@ use std::sync::Arc; use async_trait::async_trait; use parking_lot::Mutex; -use crate::openhuman::memory::api::capabilities::Capabilities; +use crate::openhuman::memory::api::capabilities::{Capabilities, Capability}; +use crate::openhuman::memory::api::chunks::{Chunk, Metadata, SourceKind}; use crate::openhuman::memory::api::error::MemoryError; use crate::openhuman::memory::api::health::MemoryHealth; +use crate::openhuman::memory::api::provider::chunks::{ChunkDetail, ChunkEmbedding, ChunkQuery}; use crate::openhuman::memory::api::provider::types::SourceScope; use crate::openhuman::memory::api::provider::types::{ExportPage, ExportRecord, ImportOutcome}; +use crate::openhuman::memory::api::provider::types::{IngestItem, IngestOutcome}; +use crate::openhuman::memory::api::provider::{MemoryChunks, MemoryIngest}; use crate::openhuman::memory::api::provider::{ MemoryCore, MemoryPortability, MemoryProvider, MemoryRecall, }; @@ -372,3 +376,391 @@ impl MemoryProvider for FixedRecallProvider { MemoryHealth::Ready } } + +/// An in-memory provider that stores **chunks**, for round trips that span +/// ingest and chunk listing. +/// +/// # Why this is not [`InMemoryProvider`] +/// +/// That one stores `MemoryEntry` behind the mandatory three and says why it +/// stops there: "advertising more would fail `audit_provider`, since no +/// optional accessor is overridden". The chunk handlers need two optional +/// families instead — [`MemoryIngest`] to write and [`MemoryChunks`] to read — +/// and a round trip is only meaningful when the same object serves both, so +/// they live together here rather than as two providers a test would have to +/// keep in sync. +/// +/// # What it does not pretend to be +/// +/// Ingest stores the item as **one** chunk. The real pipeline splits on token +/// budget, scores entities, and derives a deterministic id from +/// `(source_kind, source_id, seq_in_source, content)`. This is enough for "the +/// write reached the driver and the read brought it back through the guard", +/// which is what a handler test asserts; it is **not** enough to test chunking, +/// and an assertion about chunk counts or ids belongs against +/// `ingest_pipeline` where that logic actually lives. +/// +/// Dedupe is by `source_id`, matching the pipeline's `already_ingested` +/// behaviour, because the idempotency handler test turns on exactly that. +#[derive(Default)] +pub struct InMemoryChunks { + chunks: Mutex>, +} + +impl InMemoryChunks { + #[must_use] + pub fn new() -> Self { + Self::default() + } + + /// How many chunks are stored, for assertions that bypass the guard. + #[must_use] + pub fn len(&self) -> usize { + self.chunks.lock().len() + } + + /// Whether the store holds nothing. + #[must_use] + pub fn is_empty(&self) -> bool { + self.chunks.lock().is_empty() + } +} + +#[async_trait] +impl MemoryCore for InMemoryChunks { + async fn store( + &self, + _namespace: &str, + _key: &str, + _content: &str, + _category: MemoryCategory, + _session_id: Option<&str>, + _taint: MemoryTaint, + ) -> Result<(), MemoryError> { + Err(MemoryError::Other(anyhow::anyhow!( + "InMemoryChunks stores chunks, not entries" + ))) + } + + async fn get(&self, _namespace: &str, _key: &str) -> Result, MemoryError> { + Ok(None) + } + + async fn forget(&self, _namespace: &str, _key: &str) -> Result { + Ok(false) + } + + async fn list( + &self, + _namespace: Option<&str>, + _category: Option<&MemoryCategory>, + _session_id: Option<&str>, + ) -> Result, MemoryError> { + Ok(Vec::new()) + } + + async fn namespaces(&self) -> Result, MemoryError> { + Ok(Vec::new()) + } +} + +#[async_trait] +impl MemoryIngest for InMemoryChunks { + async fn ingest_document(&self, item: IngestItem) -> Result { + let mut chunks = self.chunks.lock(); + if chunks + .iter() + .any(|chunk| chunk.metadata.source_id == item.source_id) + { + // Matches the pipeline's `already_ingested` arm, which the adapter + // in `TinycortexProvider::ingest_document` reports as `skipped: 1`. + return Ok(IngestOutcome { + written: 0, + skipped: 1, + ids: Vec::new(), + }); + } + let timestamp = item.timestamp.unwrap_or_else(chrono::Utc::now); + let id = format!("{}::0", item.source_id); + let chunk = Chunk { + id: id.clone(), + content: item.content, + metadata: Metadata { + source_kind: SourceKind::Document, + source_id: item.source_id, + owner: item.owner, + timestamp, + time_range: (timestamp, timestamp), + tags: item.tags, + source_ref: item.source_ref, + path_scope: item.path_scope, + }, + token_count: 0, + seq_in_source: 0, + created_at: timestamp, + partial_message: false, + }; + chunks.push(chunk); + Ok(IngestOutcome { + written: 1, + skipped: 0, + ids: vec![id], + }) + } + + async fn ingest_chat(&self, messages: Vec) -> Result { + let mut outcome = IngestOutcome::default(); + for message in messages { + let one = self.ingest_document(message).await?; + outcome.written += one.written; + outcome.skipped += one.skipped; + outcome.ids.extend(one.ids); + } + Ok(outcome) + } +} + +#[async_trait] +impl MemoryChunks for InMemoryChunks { + async fn list_chunks( + &self, + query: &ChunkQuery, + scope: Option<&SourceScope>, + ) -> Result, MemoryError> { + let chunks = self.chunks.lock(); + let mut rows: Vec = chunks + .iter() + .filter(|chunk| { + let m = &chunk.metadata; + query.source_kind.is_none_or(|kind| kind == m.source_kind) + && query.source_id.as_ref().is_none_or(|id| *id == m.source_id) + && query.owner.as_ref().is_none_or(|owner| *owner == m.owner) + && query + .since_ms + .is_none_or(|since| m.timestamp.timestamp_millis() >= since) + && query + .until_ms + .is_none_or(|until| m.timestamp.timestamp_millis() <= until) + }) + // Scope is applied before the limit, as the trait requires: a + // disallowed source must not starve permitted ones out of the page. + .filter(|chunk| scope.is_none_or(|s| s.allows_source_id(&chunk.metadata.source_id))) + .cloned() + .collect(); + rows.sort_by_key(|chunk| std::cmp::Reverse(chunk.created_at)); + if let Some(offset) = query.offset { + rows = rows.into_iter().skip(offset).collect(); + } + if let Some(limit) = query.limit { + rows.truncate(limit); + } + Ok(rows) + } + + async fn get_chunk(&self, chunk_id: &str) -> Result, MemoryError> { + Ok(self + .chunks + .lock() + .iter() + .find(|chunk| chunk.id == chunk_id) + .cloned()) + } + + async fn chunk_detail(&self, _chunk_id: &str) -> Result, MemoryError> { + Err(MemoryError::Other(anyhow::anyhow!( + "InMemoryChunks does not store chunk detail" + ))) + } + + async fn storage_kinds(&self) -> Result, MemoryError> { + Ok(vec!["document".to_string()]) + } + + async fn chunk_embeddings( + &self, + _chunk_ids: &[String], + _model_signature: &str, + ) -> Result, MemoryError> { + Ok(Vec::new()) + } +} + +#[async_trait] +impl MemoryRecall for InMemoryChunks { + async fn recall( + &self, + _query: &str, + _limit: usize, + _opts: &OwnedRecallOpts, + _scope: Option<&SourceScope>, + ) -> Result, MemoryError> { + Ok(Vec::new()) + } +} + +#[async_trait] +impl MemoryPortability for InMemoryChunks { + async fn export_page( + &self, + _cursor: Option<&str>, + _limit: usize, + ) -> Result { + Err(MemoryError::Other(anyhow::anyhow!( + "InMemoryChunks does not implement export" + ))) + } + + async fn import_records( + &self, + _records: Vec, + ) -> Result { + Err(MemoryError::Other(anyhow::anyhow!( + "InMemoryChunks does not implement import" + ))) + } +} + +#[async_trait] +impl MemoryProvider for InMemoryChunks { + fn driver_id(&self) -> &str { + "in-memory-chunks" + } + + fn capabilities(&self) -> Capabilities { + Capabilities::mandatory() + .with(Capability::Chunks) + .with(Capability::Ingest) + } + + async fn health(&self) -> MemoryHealth { + MemoryHealth::Ready + } + + fn as_chunks(&self) -> Option<&dyn MemoryChunks> { + Some(self) + } + + fn as_ingest(&self) -> Option<&dyn MemoryIngest> { + Some(self) + } +} + +/// A real [`MemoryGuard`](super::MemoryGuard) over a fresh chunk store, plus +/// the store itself for assertions that bypass the guard. +#[must_use] +pub fn guarded_in_memory_chunks() -> (Arc, Arc) { + let provider = Arc::new(InMemoryChunks::new()); + let guard = guard_over(Arc::clone(&provider) as Arc); + (provider, guard) +} + +#[cfg(test)] +mod in_memory_chunks_tests { + use super::*; + use crate::openhuman::memory::api::chunks::DataSource; + + fn document(source_id: &str, owner: &str, content: &str) -> IngestItem { + IngestItem { + namespace: None, + source: DataSource::Upload, + source_id: source_id.to_string(), + owner: owner.to_string(), + source_ref: None, + content: content.to_string(), + mime: None, + timestamp: None, + tags: Vec::new(), + taint: MemoryTaint::default(), + path_scope: None, + } + } + + /// The point of this provider: a write and a read that both cross a real + /// [`MemoryGuard`] reach the same store. `InMemoryProvider` cannot serve + /// this — `as_chunks()` is `None` there, so the guard answers `Unsupported`. + #[tokio::test] + async fn a_document_written_through_the_guard_is_read_back_through_it() { + let (store, guard) = guarded_in_memory_chunks(); + + let outcome = guard + .as_ingest() + .expect("guard exposes ingest") + .ingest_document(document("doc-launch", "alice", "Phoenix launch canary")) + .await + .expect("ingest"); + assert_eq!(outcome.written, 1); + assert_eq!(store.len(), 1); + + let listed = guard + .as_chunks() + .expect("guard exposes chunks") + .list_chunks( + &ChunkQuery { + source_id: Some("doc-launch".to_string()), + ..ChunkQuery::default() + }, + None, + ) + .await + .expect("list_chunks"); + assert_eq!(listed.len(), 1); + assert!(listed[0].content.contains("Phoenix launch canary")); + + let fetched = guard + .as_chunks() + .expect("guard exposes chunks") + .get_chunk(&outcome.ids[0]) + .await + .expect("get_chunk") + .expect("chunk exists"); + assert_eq!(fetched.metadata.owner, "alice"); + } + + /// Dedupe is by `source_id`, mirroring the pipeline's `already_ingested` + /// arm that `TinycortexProvider::ingest_document` maps to `skipped: 1`. + #[tokio::test] + async fn a_repeated_source_id_is_skipped_rather_than_duplicated() { + let (store, guard) = guarded_in_memory_chunks(); + let ingest = guard.as_ingest().expect("guard exposes ingest"); + + ingest + .ingest_document(document("doc-launch", "alice", "first")) + .await + .expect("first ingest"); + let second = ingest + .ingest_document(document("doc-launch", "alice", "second")) + .await + .expect("second ingest"); + + assert_eq!(second.written, 0); + assert_eq!(second.skipped, 1); + assert_eq!(store.len(), 1, "the duplicate must not add a chunk"); + } + + /// The trait requires scope to be applied *before* the limit, so a + /// forbidden source cannot starve permitted ones out of the page. + #[tokio::test] + async fn a_scope_excludes_sources_outside_it() { + let (_store, guard) = guarded_in_memory_chunks(); + let ingest = guard.as_ingest().expect("guard exposes ingest"); + ingest + .ingest_document(document("allowed", "alice", "in scope")) + .await + .expect("ingest allowed"); + ingest + .ingest_document(document("denied", "alice", "out of scope")) + .await + .expect("ingest denied"); + + let scope = SourceScope::new(["allowed"]); + let listed = guard + .as_chunks() + .expect("guard exposes chunks") + .list_chunks(&ChunkQuery::default(), Some(&scope)) + .await + .expect("list_chunks"); + + assert_eq!(listed.len(), 1); + assert_eq!(listed[0].metadata.source_id, "allowed"); + } +} diff --git a/src/openhuman/memory/mod.rs b/src/openhuman/memory/mod.rs index c5bc8648da7..3f92be22a3c 100644 --- a/src/openhuman/memory/mod.rs +++ b/src/openhuman/memory/mod.rs @@ -35,6 +35,7 @@ pub mod agent; pub mod api; pub mod binding; +pub mod citation; pub mod driver; pub mod guard; pub mod host; @@ -47,6 +48,7 @@ pub mod host; /// legitimate, because `tinymemory-core` is a dev-dependency there and a /// dev-dependency is not linked into the shipped binary. pub mod host_impls; +pub mod namespaces; /// Host desktop policy: is the memory content root a vault Obsidian already /// knows about? See the module docs for why this is OpenHuman's and not the /// engine's. @@ -58,6 +60,7 @@ pub mod sync_events_bridge; // back from `tinymemory-core` with the rest of the agent tools — it is a `Tool` // impl end to end, and the engine crate cannot name that trait. pub mod query; +pub mod ranking; pub mod read_rpc; /// The host-side secret / PII scrubbers applied to anything this host persists /// or hands on. See the module docs for why this is OpenHuman's and not the diff --git a/src/openhuman/memory/namespaces.rs b/src/openhuman/memory/namespaces.rs new file mode 100644 index 00000000000..e3bd30d0ab8 --- /dev/null +++ b/src/openhuman/memory/namespaces.rs @@ -0,0 +1,40 @@ +//! Well-known memory namespace names, as inert strings. +//! +//! # Why these are here and not with the code that fills them +//! +//! Each of these constants names a bucket that a **gated** producer writes and +//! an **always-compiled** caller reads back. `CONVERSATION_RAW_NAMESPACE` is +//! written by `agent::learning::transcript_ingest` but named by the turn +//! autosave in `session::turn::core`; `REFLECTIONS_NAMESPACE` is written by +//! `agent::learning::reflection` but named by the prompt assembly in +//! `session::turn::context`. Both readers go through `Arc`, which +//! is contract vocabulary and compiles in every build — so with the `memory` +//! family off the *only* thing they were missing was the string. +//! +//! A `&'static str` cannot fail without a driver, so this follows +//! `skills::types`' rule: inert parts of a gated domain live in a +//! dependency-free submodule and stay ungated. The owning modules re-export +//! them, so every historical path keeps resolving and there is still one +//! definition of each. + +/// Raw, unprocessed conversation turns. +/// +/// These used to be stored with an empty namespace, which `sanitize_namespace` +/// maps to `global` — the same namespace the default recall reads. Every +/// message a user ever sent therefore accumulated in the one bucket retrieval +/// scanned in full, so recall got steadily slower for exactly the users who +/// used the product most; benchmarked at ~10k messages the scan dominated +/// per-turn cost. +/// +/// Distinct from the conversation-memory namespace, which holds *derived* +/// durable facts rather than raw turns. These are the unprocessed messages: +/// still queryable on demand by naming this namespace, but no longer swept +/// into every default-namespace read. +pub const CONVERSATION_RAW_NAMESPACE: &str = "conversation_raw"; + +/// Memory namespace + custom-category tag for explicit user reflections. +/// +/// Distinct from `learning_observations` (agent-extracted) and `user_profile` +/// (preference facts) — these are sentences the user authored about themselves +/// that should steer future agent behaviour. +pub const REFLECTIONS_NAMESPACE: &str = "learning_reflections"; diff --git a/src/openhuman/memory/ranking.rs b/src/openhuman/memory/ranking.rs new file mode 100644 index 00000000000..09a8f74f814 --- /dev/null +++ b/src/openhuman/memory/ranking.rs @@ -0,0 +1,249 @@ +//! Host-side re-ranking maths for the two agent search tools. +//! +//! # Why this is host code and not a capability family +//! +//! [`MemoryHybridSearchTool`] and [`MemoryVectorSearchTool`] already fetch +//! their candidates through [`MemoryGuard`], so by the time these functions run +//! the driver has answered and the bus round-trip is over. What is left is +//! arithmetic over values the caller is holding: a weighted sum of a +//! [`RetrievalScoreBreakdown`] the driver returned, a cosine between two +//! vectors the caller already has, and an MMR pass over that. None of it +//! touches a store, so routing it back over the bus would buy a serialization +//! hop and nothing else. +//! +//! It is also *policy the host chose*. Which of the four named profiles a mode +//! string selects, and how aggressively MMR trades relevance for diversity, are +//! decisions about what OpenHuman's agents should see — not facts about how a +//! particular engine stores things. A second driver should rank identically +//! here, which is exactly the argument for the maths living above the driver +//! rather than inside one. +//! +//! # Ported verbatim, and pinned to literals +//! +//! Every function here is a byte-for-byte port of the tinycortex original it +//! replaced (`memory::retrieval::scoring::hybrid_score`, +//! `memory::config::WeightProfile`, `memory::retrieval::mmr::mmr_select`, +//! `memory::store::vectors::cosine_similarity`). The tests assert **literal +//! expected values** rather than calling the engine, so they keep meaning once +//! the engine leaves the build — the same rule `util::redact`'s parity test +//! follows. +//! +//! [`MemoryHybridSearchTool`]: super::tools::search::hybrid_search::MemoryHybridSearchTool +//! [`MemoryVectorSearchTool`]: super::tools::search::vector_search::MemoryVectorSearchTool +//! [`MemoryGuard`]: super::guard::MemoryGuard + +use serde::{Deserialize, Serialize}; +use tinymemory_api::types::RetrievalScoreBreakdown; + +/// Named hybrid-retrieval weight profiles (graph / vector / keyword / freshness). +/// +/// Nothing here *enforces* that the four weights sum to `1.0` — the built-in +/// profiles are chosen that way by convention so scores land in a familiar +/// `[0.0, 1.0]`-ish range when every signal is itself in `[0.0, 1.0]`. A custom +/// profile with a different total simply rescales the final score. +#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize)] +pub struct WeightProfile { + /// Weight on graph/co-occurrence proximity signal. + pub graph: f64, + /// Weight on dense vector (cosine) similarity signal. + pub vector: f64, + /// Weight on lexical/keyword match signal. + pub keyword: f64, + /// Weight on recency; `0.0` disables freshness boosting. + pub freshness: f64, +} + +impl WeightProfile { + /// `balanced`: graph 0.35, vector 0.35, keyword 0.15, freshness 0.15. + pub const BALANCED: Self = Self { + graph: 0.35, + vector: 0.35, + keyword: 0.15, + freshness: 0.15, + }; + /// `semantic`: graph 0.15, vector 0.65, keyword 0.20. + pub const SEMANTIC: Self = Self { + graph: 0.15, + vector: 0.65, + keyword: 0.20, + freshness: 0.0, + }; + /// `lexical`: graph 0.25, vector 0.15, keyword 0.60. + pub const LEXICAL: Self = Self { + graph: 0.25, + vector: 0.15, + keyword: 0.60, + freshness: 0.0, + }; + /// `graph_first`: graph 0.55, vector 0.30, keyword 0.15. + pub const GRAPH_FIRST: Self = Self { + graph: 0.55, + vector: 0.30, + keyword: 0.15, + freshness: 0.0, + }; + + /// Resolve a profile by its wire name, returning `None` for unknown names. + #[must_use] + pub fn by_name(name: &str) -> Option { + match name { + "balanced" => Some(Self::BALANCED), + "semantic" => Some(Self::SEMANTIC), + "lexical" => Some(Self::LEXICAL), + "graph_first" => Some(Self::GRAPH_FIRST), + _ => None, + } + } +} + +/// Combine the four retrieval signals into a single ranking score under +/// `profile`, as the weighted sum `graph·g + vector·v + keyword·k + freshness·f`. +/// +/// `episodic_relevance` is left at `0.0` — episodic memory is not a tree- +/// retrieval signal — but is carried in the breakdown for wire compatibility +/// with [`RetrievalScoreBreakdown`]. +#[must_use] +pub fn hybrid_score( + profile: &WeightProfile, + graph_relevance: f64, + vector_similarity: f64, + keyword_relevance: f64, + freshness: f64, +) -> RetrievalScoreBreakdown { + let final_score = profile.graph * graph_relevance + + profile.vector * vector_similarity + + profile.keyword * keyword_relevance + + profile.freshness * freshness; + RetrievalScoreBreakdown { + keyword_relevance, + vector_similarity, + graph_relevance, + episodic_relevance: 0.0, + freshness, + final_score, + } +} + +/// Cosine similarity between two vectors. Returns `0.0` for mismatched +/// lengths, empty vectors, or zero-magnitude vectors. +/// +/// The raw cosine value is clamped only for floating-point drift, preserving +/// its mathematical `[-1.0, 1.0]` range. +#[must_use] +pub fn cosine_similarity(a: &[f32], b: &[f32]) -> f64 { + if a.len() != b.len() || a.is_empty() { + return 0.0; + } + let mut dot = 0.0_f64; + let mut norm_a = 0.0_f64; + let mut norm_b = 0.0_f64; + for (x, y) in a.iter().zip(b.iter()) { + let x = f64::from(*x); + let y = f64::from(*y); + dot += x * y; + norm_a += x * x; + norm_b += y * y; + } + let denom = norm_a.sqrt() * norm_b.sqrt(); + if denom <= f64::EPSILON { + return 0.0; + } + (dot / denom).clamp(-1.0, 1.0) +} + +/// A candidate for MMR selection. +pub struct MmrCandidate<'a> { + /// Caller-side index, echoed back on the result so the candidate can be + /// resolved to its original record. + pub index: usize, + /// Candidate embedding; must share dimensionality with every other + /// candidate, since cosine similarity is computed pairwise. + pub embedding: &'a [f32], + /// Precomputed relevance of this candidate to the query (typically a + /// cosine score). Higher is more relevant; weighted by `lambda`. + pub relevance: f64, +} + +/// Result of MMR selection: the original index and its MMR score. +#[derive(Debug, Clone)] +pub struct MmrResult { + /// Caller-side index echoed from the chosen [`MmrCandidate::index`]. + pub index: usize, + /// The MMR score at the step this item was selected: + /// `lambda · relevance − (1 − lambda) · max_similarity(c, selected)`. + /// Not comparable across runs with different `lambda`. + pub score: f64, +} + +/// Select up to `limit` items from `candidates` using Maximal Marginal +/// Relevance. +/// +/// `lambda` controls the relevance-diversity tradeoff: `1.0` is pure +/// relevance, `0.0` pure diversity, `0.7` the recommended default. It is +/// clamped to `[0.0, 1.0]`; `limit` is clamped to `candidates.len()`. +/// +/// # Gotcha: `query_vec` is unused +/// +/// Relevance-to-query is taken entirely from each candidate's precomputed +/// [`MmrCandidate::relevance`], which the caller must already have derived +/// from the query. Passing a mismatched or empty `query_vec` has no effect on +/// the result. The parameter is kept for shape parity with the tinycortex +/// original this replaced, so a diff between the two is empty rather than +/// "and the signature changed too". +#[must_use] +pub fn mmr_select( + query_vec: &[f32], + candidates: &[MmrCandidate<'_>], + limit: usize, + lambda: f64, +) -> Vec { + if candidates.is_empty() || limit == 0 { + return Vec::new(); + } + + let lambda = lambda.clamp(0.0, 1.0); + let limit = limit.min(candidates.len()); + + let mut selected_embeddings: Vec<&[f32]> = Vec::with_capacity(limit); + let mut results: Vec = Vec::with_capacity(limit); + let mut available: Vec = vec![true; candidates.len()]; + + for _ in 0..limit { + let mut best_idx: Option = None; + let mut best_mmr = f64::NEG_INFINITY; + + for (i, candidate) in candidates.iter().enumerate() { + if !available[i] { + continue; + } + let max_sim_to_selected = if selected_embeddings.is_empty() { + 0.0 + } else { + selected_embeddings + .iter() + .map(|sel| cosine_similarity(candidate.embedding, sel)) + .fold(f64::NEG_INFINITY, f64::max) + }; + let mmr_score = lambda * candidate.relevance - (1.0 - lambda) * max_sim_to_selected; + if mmr_score > best_mmr { + best_mmr = mmr_score; + best_idx = Some(i); + } + } + + let Some(idx) = best_idx else { break }; + available[idx] = false; + selected_embeddings.push(candidates[idx].embedding); + results.push(MmrResult { + index: candidates[idx].index, + score: best_mmr, + }); + } + + let _ = query_vec; + results +} + +#[cfg(test)] +#[path = "ranking_tests.rs"] +mod tests; diff --git a/src/openhuman/memory/ranking_tests.rs b/src/openhuman/memory/ranking_tests.rs new file mode 100644 index 00000000000..0287828dbe5 --- /dev/null +++ b/src/openhuman/memory/ranking_tests.rs @@ -0,0 +1,168 @@ +//! Parity tests for the ported ranking maths. +//! +//! Every expectation is a **literal**, never a call back into the engine. A +//! test that compared against `tinycortex::…` would pass by construction and +//! then stop compiling the day the engine leaves the build — which is the day +//! it would actually be needed. Same rule as `util::redact`'s parity test. + +use super::{cosine_similarity, hybrid_score, mmr_select, MmrCandidate, WeightProfile}; + +/// Tolerance for the one expectation that is not exact in binary floating +/// point. The clean cases below are asserted exactly on purpose. +const EPSILON: f64 = 1e-12; + +#[test] +fn named_profiles_resolve_and_unknown_names_do_not() { + assert_eq!( + WeightProfile::by_name("balanced"), + Some(WeightProfile::BALANCED) + ); + assert_eq!( + WeightProfile::by_name("semantic"), + Some(WeightProfile::SEMANTIC) + ); + assert_eq!( + WeightProfile::by_name("lexical"), + Some(WeightProfile::LEXICAL) + ); + assert_eq!( + WeightProfile::by_name("graph_first"), + Some(WeightProfile::GRAPH_FIRST) + ); + assert_eq!(WeightProfile::by_name("nonexistent"), None); + // The tool passes the agent's `mode` string straight in, so an empty or + // differently-cased value must miss rather than fall back to balanced. + assert_eq!(WeightProfile::by_name(""), None); + assert_eq!(WeightProfile::by_name("Balanced"), None); +} + +#[test] +fn hybrid_score_is_the_weighted_sum_of_the_four_signals() { + let breakdown = hybrid_score(&WeightProfile::BALANCED, 0.8, 0.6, 0.4, 0.2); + + // 0.35·0.8 + 0.35·0.6 + 0.15·0.4 + 0.15·0.2 = 0.58 + assert!((breakdown.final_score - 0.58).abs() < EPSILON); + // The four inputs are carried through untouched… + assert!((breakdown.graph_relevance - 0.8).abs() < EPSILON); + assert!((breakdown.vector_similarity - 0.6).abs() < EPSILON); + assert!((breakdown.keyword_relevance - 0.4).abs() < EPSILON); + assert!((breakdown.freshness - 0.2).abs() < EPSILON); + // …and episodic is always zero: it is not a tree-retrieval signal, and is + // present only for wire compatibility with the breakdown shape. + assert_eq!(breakdown.episodic_relevance, 0.0); +} + +#[test] +fn a_zero_freshness_profile_ignores_the_freshness_signal() { + // `semantic` sets freshness to 0.0, so a wildly fresh item scores the same + // as a stale one. Pins that the weight is applied rather than assumed. + let fresh = hybrid_score(&WeightProfile::SEMANTIC, 0.0, 1.0, 0.0, 1.0); + let stale = hybrid_score(&WeightProfile::SEMANTIC, 0.0, 1.0, 0.0, 0.0); + assert!((fresh.final_score - stale.final_score).abs() < EPSILON); + assert!((fresh.final_score - 0.65).abs() < EPSILON); +} + +#[test] +fn cosine_similarity_covers_its_whole_mathematical_range() { + assert_eq!(cosine_similarity(&[1.0, 0.0], &[1.0, 0.0]), 1.0); + assert_eq!(cosine_similarity(&[1.0, 0.0], &[0.0, 1.0]), 0.0); + // Anti-correlated stays negative — the value is clamped to [-1, 1] for + // floating-point drift only, not squashed into [0, 1]. + assert_eq!(cosine_similarity(&[1.0, 0.0], &[-1.0, 0.0]), -1.0); + + // 32 / sqrt(14 · 77) + let expected = 32.0_f64 / (14.0_f64.sqrt() * 77.0_f64.sqrt()); + assert!((cosine_similarity(&[1.0, 2.0, 3.0], &[4.0, 5.0, 6.0]) - expected).abs() < EPSILON); +} + +#[test] +fn cosine_similarity_returns_zero_for_the_degenerate_inputs() { + assert_eq!(cosine_similarity(&[], &[]), 0.0); + assert_eq!(cosine_similarity(&[1.0, 2.0], &[1.0]), 0.0); + assert_eq!(cosine_similarity(&[0.0, 0.0], &[1.0, 1.0]), 0.0); +} + +#[test] +fn mmr_prefers_a_diverse_candidate_over_a_more_relevant_duplicate() { + let near = [1.0_f32, 0.0]; + let orthogonal = [0.0_f32, 1.0]; + let candidates = vec![ + MmrCandidate { + index: 0, + embedding: &near, + relevance: 0.9, + }, + MmrCandidate { + index: 1, + embedding: &near, + relevance: 0.8, + }, + MmrCandidate { + index: 2, + embedding: &orthogonal, + relevance: 0.5, + }, + ]; + + let selected = mmr_select(&[], &candidates, 3, 0.7); + let order: Vec = selected.iter().map(|r| r.index).collect(); + + // Candidate 1 is more relevant than 2 but identical to the already-picked + // 0, so diversity demotes it below the orthogonal candidate. This is the + // whole point of the pass; a plain relevance sort would give [0, 1, 2]. + assert_eq!(order, vec![0, 2, 1]); + + assert!((selected[0].score - 0.63).abs() < EPSILON); // 0.7·0.9 + assert!((selected[1].score - 0.35).abs() < EPSILON); // 0.7·0.5 − 0.3·0.0 + assert!((selected[2].score - 0.26).abs() < EPSILON); // 0.7·0.8 − 0.3·1.0 +} + +#[test] +fn mmr_rewards_an_anti_correlated_candidate() { + let positive = [1.0_f32, 0.0]; + let anti_correlated = [-1.0_f32, 0.0]; + let orthogonal = [0.0_f32, 1.0]; + let candidates = vec![ + MmrCandidate { + index: 0, + embedding: &positive, + relevance: 0.9, + }, + MmrCandidate { + index: 1, + embedding: &anti_correlated, + relevance: 0.5, + }, + MmrCandidate { + index: 2, + embedding: &orthogonal, + relevance: 0.55, + }, + ]; + + let selected = mmr_select(&[], &candidates, 2, 0.5); + + // The negative cosine is a diversity reward: -0.5 · -1.0 lifts the + // anti-correlated candidate above the merely orthogonal one. + assert_eq!(selected[1].index, 1); + assert!((selected[1].score - 0.75).abs() < EPSILON); +} + +#[test] +fn mmr_clamps_limit_and_lambda_and_short_circuits_on_empty_input() { + let emb = [1.0_f32, 0.0]; + let candidates = vec![MmrCandidate { + index: 7, + embedding: &emb, + relevance: 0.5, + }]; + + // `limit` above the candidate count yields the candidates, not a panic. + assert_eq!(mmr_select(&[], &candidates, 99, 0.7).len(), 1); + // `lambda` out of range is clamped: 2.0 behaves as pure relevance. + let pure_relevance = mmr_select(&[], &candidates, 1, 2.0); + assert!((pure_relevance[0].score - 0.5).abs() < EPSILON); + + assert!(mmr_select(&[], &candidates, 0, 0.7).is_empty()); + assert!(mmr_select(&[], &[], 5, 0.7).is_empty()); +} diff --git a/src/openhuman/memory/stub.rs b/src/openhuman/memory/stub.rs new file mode 100644 index 00000000000..54c898c9d20 --- /dev/null +++ b/src/openhuman/memory/stub.rs @@ -0,0 +1,82 @@ +//! The `memory`-disabled surface. +//! +//! # What an off-build must still be able to say +//! +//! Roughly 230 production references across ~101 files outside this family +//! reach into it, and they do not agree on what "off" should mean. Two shapes +//! are needed, and picking one for everything would be wrong: +//! +//! - **Registration sites want absence.** The controller registry in +//! `core/all.rs` calls the `all_*_registered_controllers` aggregators +//! unconditionally. A stub that returned a controller answering +//! `Err("memory disabled")` would make `memory.*` a *known* method that +//! fails at run time — the opposite of the intended unknown-method. So the +//! aggregators return empty vectors, and `core/all.rs` needs no `#[cfg]` at +//! all. +//! - **Hot-path readers want a no-op.** The agent harness recalls memory on +//! every turn and the archivist records every turn. Those callers are always +//! compiled and do not branch on a feature, so their entry points here +//! return "nothing" rather than an error. That is not a new behaviour: it is +//! exactly what `NullMemoryProvider` already produces at run time when no +//! module is available, and what `DomainSet` models with `memory: false`. +//! +//! Anything that *writes* returns a build-fact error naming the gate, because +//! silently discarding a write is the one failure mode that looks like success. +//! +//! # The rule for editing this file +//! +//! Signatures must match the real ones exactly. The disabled build is the +//! **only** thing that catches drift here — CI's smoke lane runs `cargo check`, +//! not `cargo test --no-default-features` — so run +//! +//! ```text +//! cargo check --no-default-features --features "media,skills,flows,mcp,channels,medulla,http-server,scheduler-gate,file-logging,modules" +//! ``` +//! +//! before pushing any change to the memory surface. +//! +//! Do **not** add types here. Inert types belong in `memory::api` (the contract +//! re-export) or `memory::ranking`, both of which stay compiled in every build +//! precisely so there is one definition rather than two. + +use crate::core::all::RegisteredController; +use crate::core::ControllerSchema; + +/// The message every disabled write path returns. +/// +/// A build fact, phrased as one: a caller reading this in a log should know to +/// rebuild rather than to check their configuration. +pub const MEMORY_DISABLED: &str = + "memory feature disabled at compile time — rebuild with `--features memory`"; + +macro_rules! empty_controller_aggregators { + ($($schemas:ident / $controllers:ident),* $(,)?) => { + $( + /// No schemas: the family is compiled out, so its methods are + /// unknown rather than known-and-failing. + #[must_use] + pub fn $schemas() -> Vec { + Vec::new() + } + + /// No controllers, for the same reason as the schema half. + #[must_use] + pub fn $controllers() -> Vec { + Vec::new() + } + )* + }; +} + +empty_controller_aggregators!( + all_memory_controller_schemas / all_memory_registered_controllers, + all_memory_core_recall_controller_schemas / all_memory_core_recall_registered_controllers, + all_memory_documents_controller_schemas / all_memory_documents_registered_controllers, + all_memory_files_controller_schemas / all_memory_files_registered_controllers, + all_memory_ingest_controller_schemas / all_memory_ingest_registered_controllers, + all_memory_kv_graph_controller_schemas / all_memory_kv_graph_registered_controllers, + all_memory_learn_controller_schemas / all_memory_learn_registered_controllers, + all_memory_provider_controller_schemas / all_memory_provider_registered_controllers, + all_memory_sync_controller_schemas / all_memory_sync_registered_controllers, + all_memory_tool_memory_controller_schemas / all_memory_tool_memory_registered_controllers, +); diff --git a/src/openhuman/memory/tools/search/hybrid_search.rs b/src/openhuman/memory/tools/search/hybrid_search.rs index 41c76528745..bb7f953a0f9 100644 --- a/src/openhuman/memory/tools/search/hybrid_search.rs +++ b/src/openhuman/memory/tools/search/hybrid_search.rs @@ -12,8 +12,8 @@ use std::fmt::Write; use crate::openhuman::memory::api::provider::MemoryProvider; use crate::openhuman::memory::api::types::MemoryItemKind; use crate::openhuman::memory::ops::guard::active_memory_guard; +use crate::openhuman::memory::ranking::{hybrid_score, WeightProfile}; use crate::openhuman::tools::traits::{Tool, ToolResult}; -use tinycortex::memory::WeightProfile; pub struct MemoryHybridSearchTool; @@ -173,7 +173,7 @@ impl Tool for MemoryHybridSearchTool { .enumerate() .map(|(i, hit)| { let bd = &hit.score_breakdown; - let score = tinycortex::memory::retrieval::scoring::hybrid_score( + let score = hybrid_score( &profile, bd.graph_relevance, bd.vector_similarity, diff --git a/src/openhuman/memory/tools/search/vector_search.rs b/src/openhuman/memory/tools/search/vector_search.rs index 23770074b56..ec6d64ed369 100644 --- a/src/openhuman/memory/tools/search/vector_search.rs +++ b/src/openhuman/memory/tools/search/vector_search.rs @@ -15,9 +15,8 @@ use crate::openhuman::memory::api::chunks::SourceKind; use crate::openhuman::memory::api::provider::ChunkQuery; use crate::openhuman::memory::api::provider::MemoryProvider; use crate::openhuman::memory::ops::guard::active_memory_guard; +use crate::openhuman::memory::ranking::{cosine_similarity, mmr_select, MmrCandidate}; use crate::openhuman::tools::traits::{Tool, ToolResult}; -use tinycortex::memory::retrieval::mmr::{mmr_select, MmrCandidate}; -use tinycortex::memory::store::vectors::cosine_similarity; pub struct MemoryVectorSearchTool; diff --git a/src/openhuman/modules/boot.rs b/src/openhuman/modules/boot.rs index 352fc9907a7..27c5d42bc33 100644 --- a/src/openhuman/modules/boot.rs +++ b/src/openhuman/modules/boot.rs @@ -88,13 +88,25 @@ pub async fn load_declared_modules(config: &Config) { /// side-effect-free check `memory::binding::build` itself uses to decide what /// actually gets bound, so this can never disagree with the real binding. fn should_eager_load(record: &super::types::ModuleRecord, config: &Config) -> bool { - if record.id != super::memory::MODULE_ID { - return true; + // With the `memory` family compiled out there is no memory module to load, + // and no binding to ask. The registry still *lists* the artifact — it is a + // compiled-in `const` table of what this build trusts, not what it loads — + // but nothing reaches for it, so nothing is downloaded or `dlopen`ed. + #[cfg(not(feature = "memory"))] + { + let _ = config; + return record.id != super::registry::TINYMEMORY_MODULE_ID; + } + #[cfg(feature = "memory")] + { + if record.id != super::memory::MODULE_ID { + return true; + } + matches!( + crate::openhuman::memory::binding::admit(&config.subsystems.memory), + Ok((_, crate::core::subsystem::DriverClass::Module)) + ) } - matches!( - crate::openhuman::memory::binding::admit(&config.subsystems.memory), - Ok((_, crate::core::subsystem::DriverClass::Module)) - ) } #[cfg(test)] diff --git a/src/openhuman/modules/memory_host.rs b/src/openhuman/modules/memory_host.rs index 8b460ae43c4..793190db41b 100644 --- a/src/openhuman/modules/memory_host.rs +++ b/src/openhuman/modules/memory_host.rs @@ -562,7 +562,13 @@ fn into_domain_event(event: MemoryEvent) -> Option { + // The user-facing error surface lives in the memory tree's health + // module. Unreachable with the family off — nothing binds a module + // driver that could raise this — but the arm must still compile. + #[cfg(feature = "memory")] crate::openhuman::memory::tree::health::user_error::publish_local_model_unavailable_user_error(&origin); + #[cfg(not(feature = "memory"))] + let _ = origin; return None; } // Web-channel-only (openhuman#5820): the module's engine already diff --git a/src/openhuman/modules/registry.rs b/src/openhuman/modules/registry.rs index 7af72e5da4b..294314cd079 100644 --- a/src/openhuman/modules/registry.rs +++ b/src/openhuman/modules/registry.rs @@ -26,6 +26,9 @@ use super::types::{LoadPolicy, ModuleRecord, PlatformAsset}; +/// Stable registry identifier for the optional TinyMemory module. +pub const TINYMEMORY_MODULE_ID: &str = "tinymemory"; + /// The `tinydocs` module: `.docx` / `.pptx` synthesis and `.pdf` extraction. /// /// Lazy, because a user who never asks for a document should not pay a download, diff --git a/src/openhuman/security/credentials/ops.rs b/src/openhuman/security/credentials/ops.rs index ee35ba5fb2c..0d64d59d97c 100644 --- a/src/openhuman/security/credentials/ops.rs +++ b/src/openhuman/security/credentials/ops.rs @@ -630,10 +630,13 @@ async fn store_session_inner( // binding to the per-user workspace. Seeding a host-side global as well // opened the engine's database a second time in this process (#4378 fixed // the workspace it pointed at; the module port removes the second reader). - crate::openhuman::memory::conversations::register_conversation_persistence_subscriber( - effective_config.workspace_dir.clone(), - ); - logs.push("conversation persistence bound to active workspace".to_string()); + #[cfg(feature = "memory")] + { + crate::openhuman::memory::conversations::register_conversation_persistence_subscriber( + effective_config.workspace_dir.clone(), + ); + logs.push("conversation persistence bound to active workspace".to_string()); + } // Start all login-gated services (voice, orchestration, and local AI). // Uses the effective config so services see the user-scoped workspace @@ -852,6 +855,7 @@ pub async fn clear_session(config: &Config) -> Result String { uuid::Uuid::new_v4().to_string() diff --git a/src/openhuman/threads/schemas.rs b/src/openhuman/threads/schemas.rs index be15b0c79a1..083886c3571 100644 --- a/src/openhuman/threads/schemas.rs +++ b/src/openhuman/threads/schemas.rs @@ -6,15 +6,15 @@ use serde_json::{Map, Value}; use crate::core::all::{ControllerFuture, RegisteredController}; use crate::core::{ControllerSchema, FieldSchema, TypeSchema}; use crate::openhuman::agent::task_board::{TaskBoard, TaskBoardCard, TaskBoardStore}; -use crate::openhuman::memory::{ +use crate::openhuman::threads::turn_state::{ + ClearTurnStateRequest, GetTurnStateForRequestRequest, GetTurnStateRequest, +}; +use tinymemory_core::rpc_models::{ AppendConversationMessageRequest, ConversationMessagesRequest, CreateConversationThreadRequest, DeleteConversationThreadRequest, EmptyRequest, GenerateConversationThreadTitleRequest, UpdateConversationMessageRequest, UpdateConversationThreadLabelsRequest, UpdateConversationThreadTitleRequest, UpsertConversationThreadRequest, }; -use crate::openhuman::threads::turn_state::{ - ClearTurnStateRequest, GetTurnStateForRequestRequest, GetTurnStateRequest, -}; use super::ops; diff --git a/src/openhuman/tools/impl/system/mod.rs b/src/openhuman/tools/impl/system/mod.rs index 89fc77b9972..93c873469f2 100644 --- a/src/openhuman/tools/impl/system/mod.rs +++ b/src/openhuman/tools/impl/system/mod.rs @@ -13,6 +13,7 @@ mod resolve_time; mod retrieve_tool_output; mod schedule; mod shell; +#[cfg(feature = "memory")] mod tool_stats; mod update_apply; mod update_check; @@ -37,6 +38,7 @@ pub use resolve_time::ResolveTimeTool; pub use retrieve_tool_output::RetrieveToolOutputTool; pub use schedule::ScheduleTool; pub use shell::ShellTool; +#[cfg(feature = "memory")] pub use tool_stats::ToolStatsTool; pub use update_apply::UpdateApplyTool; pub use update_check::UpdateCheckTool; diff --git a/src/openhuman/tools/mod.rs b/src/openhuman/tools/mod.rs index 6e31da5668b..5f7a3841a56 100644 --- a/src/openhuman/tools/mod.rs +++ b/src/openhuman/tools/mod.rs @@ -16,6 +16,7 @@ pub(crate) mod user_filter; pub(crate) mod implementations; pub use crate::openhuman::agent::artifacts::tools::*; +#[cfg(feature = "memory")] pub use crate::openhuman::agent::learning::tools::*; pub use crate::openhuman::agent::orchestration::tools::*; pub use crate::openhuman::agent::tools::*; @@ -30,6 +31,7 @@ pub use crate::openhuman::flows::builder_tools::*; #[cfg(feature = "flows")] pub use crate::openhuman::flows::discovery_tools::*; #[cfg(feature = "flows")] +#[cfg(feature = "memory")] pub use crate::openhuman::flows::memory_tools::*; #[cfg(feature = "flows")] #[cfg(feature = "flows")] @@ -40,10 +42,13 @@ pub use crate::openhuman::integrations::task_sources::tools::*; pub use crate::openhuman::integrations::tools::*; #[cfg(feature = "mcp")] pub use crate::openhuman::mcp::registry::tools::*; +#[cfg(feature = "memory")] pub use crate::openhuman::memory::agent::tools::*; -#[cfg(feature = "memory-git")] +#[cfg(all(feature = "memory-git", feature = "memory"))] pub use crate::openhuman::memory::tools::diff::*; +#[cfg(feature = "memory")] pub use crate::openhuman::memory::tools::goals::*; +#[cfg(feature = "memory")] pub use crate::openhuman::memory::tools::*; pub use crate::openhuman::platform::cost::tools::*; pub use crate::openhuman::platform::doctor::tools::*; diff --git a/src/openhuman/tools/ops.rs b/src/openhuman/tools/ops.rs index e126b9a9fe6..445cf20ba2a 100644 --- a/src/openhuman/tools/ops.rs +++ b/src/openhuman/tools/ops.rs @@ -437,8 +437,11 @@ pub fn all_tools_with_runtime( Box::new(WalletTxReceiptTool::new()), #[cfg(feature = "web3")] Box::new(WalletLookupTxTool::new()), + #[cfg(feature = "memory")] Box::new(MemoryStoreTool::new(security.clone())), + #[cfg(feature = "memory")] Box::new(MemoryRecallTool::new()), + #[cfg(feature = "memory")] Box::new(MemoryForgetTool::new(security.clone())), // #4458: the memory read→dedupe→write→update-index protocol // (`agent::harness::memory_protocol`) can only close its write cycle via a @@ -455,30 +458,41 @@ pub fn all_tools_with_runtime( Box::new(UpdateMemoryMdTool::new(root_config.workspace_dir.clone())), // #002: read-only self-diagnosis of the memory pipeline so the agent // can explain an empty/stalled wiki + the fix. + #[cfg(feature = "memory")] Box::new(MemoryDoctorTool::new(config.clone())), // #5172: read-only access to the compiled persona flavour profiles // (communication/coding_style/stack/workflow/environment/directives/ // anti_preferences) that persona ingestion builds but nothing // previously surfaced to the agent loop. + #[cfg(feature = "memory")] Box::new(MemoryFlavourTool::new(config.clone())), + #[cfg(feature = "memory")] Box::new(MemoryQueryTool), // memory_search tools — vector search, chunk context, hybrid search, // and previously unregistered raw store tools. + #[cfg(feature = "memory")] Box::new(MemoryVectorSearchTool), + #[cfg(feature = "memory")] Box::new(MemoryChunkContextTool), + #[cfg(feature = "memory")] Box::new(MemoryHybridSearchTool), + #[cfg(feature = "memory")] Box::new(MemoryStoreRawSearchTool), + #[cfg(feature = "memory")] Box::new(MemoryStoreRawChunksTool), + #[cfg(feature = "memory")] Box::new(MemoryStoreKindsTool), // Explicit user-preference pinning — always registered so the model // can save user-stated preferences regardless of whether the full // inference-based learning subsystem is enabled. The preference // injection into the system prompt is controlled independently by // `config.learning.explicit_preferences_enabled`. + #[cfg(feature = "memory")] Box::new(RememberPreferenceTool::new(security.clone())), // Two-lane explicit preferences (general → system prompt, situational → // per-query recall). Written verbatim to user_pref_{general,situational}; // bypasses the inference/stability pipeline. Always registered. + #[cfg(feature = "memory")] Box::new(SavePreferenceTool::new(security.clone())), // WhatsApp data store — read-only agent surface (issue #1341). The // store lives in the Tauri shell; these tools reach it over the @@ -581,16 +595,27 @@ pub fn all_tools_with_runtime( // every mutator ships default-OFF via `tools::user_filter` // (learning_manage toggle) — they persistently rewrite the assistant's // model of the user. enrich_profile also flags external_effect. + #[cfg(feature = "memory")] Box::new(LearningListFacetsTool), + #[cfg(feature = "memory")] Box::new(LearningGetFacetTool), + #[cfg(feature = "memory")] Box::new(LearningCacheStatsTool), + #[cfg(feature = "memory")] Box::new(LearningUpdateFacetTool), + #[cfg(feature = "memory")] Box::new(LearningPinFacetTool), + #[cfg(feature = "memory")] Box::new(LearningUnpinFacetTool), + #[cfg(feature = "memory")] Box::new(LearningForgetFacetTool), + #[cfg(feature = "memory")] Box::new(LearningRebuildCacheTool), + #[cfg(feature = "memory")] Box::new(LearningResetCacheTool), + #[cfg(feature = "memory")] Box::new(LearningSaveProfileTool), + #[cfg(feature = "memory")] Box::new(LearningEnrichProfileTool), // Task & productivity tools (issue: agent-tool expansion). // Read/observe + bounded-write tools are registered here; the @@ -1027,6 +1052,7 @@ pub fn all_tools_with_runtime( tool_tracking_enabled = root_config.learning.tool_tracking_enabled, "evaluating ToolStatsTool registration" ); + #[cfg(feature = "memory")] if root_config.learning.enabled && root_config.learning.tool_tracking_enabled { tools.push(Box::new(ToolStatsTool::new())); tracing::debug!("ToolStatsTool registered"); diff --git a/src/openhuman/tools/schemas.rs b/src/openhuman/tools/schemas.rs index 1ea7873e8a8..8f5a26c627e 100644 --- a/src/openhuman/tools/schemas.rs +++ b/src/openhuman/tools/schemas.rs @@ -752,38 +752,57 @@ fn handle_searxng_search(params: Map) -> ControllerFuture { fn handle_apify_linkedin_scrape(params: Map) -> ControllerFuture { Box::pin(async move { - let profile_url = params - .get("profile_url") - .and_then(Value::as_str) - .map(str::trim) - .filter(|s| !s.is_empty()) - .map(str::to_string) - .ok_or_else(|| "missing or empty `profile_url`".to_string())?; - - let config = config_rpc::load_config_with_timeout().await?; - let client = crate::openhuman::integrations::build_client(&config).ok_or_else(|| { - "Apify scrape unavailable — no backend session token. Sign in first.".to_string() - })?; - - let data = crate::openhuman::agent::learning::linkedin_enrichment::scrape_linkedin_profile( - &client, - &profile_url, - ) - .await - .map_err(|e| format!("Apify LinkedIn scrape failed: {e:#}"))?; - - let markdown = - crate::openhuman::agent::learning::linkedin_enrichment::render_profile_markdown( - &profile_url, - &data, + #[cfg(not(feature = "memory"))] + { + let _ = params; + return Err( + "apify_linkedin_scrape unavailable: the `memory` feature is disabled in this build" + .to_string(), ); + } - let payload = json!({ "data": data, "markdown": markdown }); - let log = vec![format!( - "tools.apify_linkedin_scrape: url={profile_url} markdown_chars={}", - markdown.chars().count() - )]; - RpcOutcome::new(payload, log).into_cli_compatible_json() + #[cfg(feature = "memory")] + { + let profile_url = params + .get("profile_url") + .and_then(Value::as_str) + .map(str::trim) + .filter(|s| !s.is_empty()) + .map(str::to_string) + .ok_or_else(|| "missing or empty `profile_url`".to_string())?; + + let config = config_rpc::load_config_with_timeout().await?; + let client = + crate::openhuman::integrations::build_client(&config).ok_or_else(|| { + "Apify scrape unavailable — no backend session token. Sign in first." + .to_string() + })?; + + // The scrape and the markdown renderer both live in + // `agent::learning::linkedin_enrichment`, which enriches profile + // facets — memory-family behaviour end to end. With the family off the + // whole handler is a build-fact error rather than a partial answer. + let data = + crate::openhuman::agent::learning::linkedin_enrichment::scrape_linkedin_profile( + &client, + &profile_url, + ) + .await + .map_err(|e| format!("Apify LinkedIn scrape failed: {e:#}"))?; + + let markdown = + crate::openhuman::agent::learning::linkedin_enrichment::render_profile_markdown( + &profile_url, + &data, + ); + + let payload = json!({ "data": data, "markdown": markdown }); + let log = vec![format!( + "tools.apify_linkedin_scrape: url={profile_url} markdown_chars={}", + markdown.chars().count() + )]; + RpcOutcome::new(payload, log).into_cli_compatible_json() + } }) } diff --git a/src/openhuman/web_chat/presentation.rs b/src/openhuman/web_chat/presentation.rs index 473c010a896..e3e4f6fc921 100644 --- a/src/openhuman/web_chat/presentation.rs +++ b/src/openhuman/web_chat/presentation.rs @@ -56,7 +56,7 @@ pub(crate) async fn deliver_response( request_id: &str, full_response: &str, user_message: &str, - citations: &[crate::openhuman::memory::agent::memory_loader::MemoryCitation], + citations: &[crate::openhuman::memory::citation::MemoryCitation], usage: Option<&LastTurnUsage>, ) { let usage_payload = usage_payload(usage); @@ -481,7 +481,7 @@ async fn try_reaction(user_message: &str) -> Option { #[cfg(any(test, debug_assertions))] pub mod test_support { - use crate::openhuman::memory::agent::memory_loader::MemoryCitation; + use crate::openhuman::memory::citation::MemoryCitation; pub fn segment_for_delivery_for_test(text: &str) -> Vec { super::segment_for_delivery(text) diff --git a/src/openhuman/web_chat/run_task.rs b/src/openhuman/web_chat/run_task.rs index 1d25e9718ea..5ba75d7cb55 100644 --- a/src/openhuman/web_chat/run_task.rs +++ b/src/openhuman/web_chat/run_task.rs @@ -185,7 +185,7 @@ pub(crate) async fn run_chat_task( // Blocking pool: the store takes a process-global mutex and reads // the thread's whole JSONL under it, so doing this inline parked an // async worker on the chat hot path (#5156). - match crate::openhuman::memory::conversations::blocking::get_messages( + match tinymemory_core::conversations::blocking::get_messages( config.workspace_dir.clone(), thread_id.to_string(), ) diff --git a/src/openhuman/web_chat/types.rs b/src/openhuman/web_chat/types.rs index 7301d34b68c..73ab4b1d975 100644 --- a/src/openhuman/web_chat/types.rs +++ b/src/openhuman/web_chat/types.rs @@ -69,7 +69,7 @@ pub(super) struct ParallelEntry { #[derive(Debug, Clone)] pub(super) struct WebChatTaskResult { pub(super) full_response: String, - pub(super) citations: Vec, + pub(super) citations: Vec, /// Holistic token/cost/context totals for the turn (parent + sub-agents), /// forwarded to the frontend on `chat_done`. `None` for synthetic results /// (e.g. budget-exhausted placeholders) that never ran a real turn. diff --git a/vendor/tinyagents b/vendor/tinyagents index 954b3dcf580..e0f3210622b 160000 --- a/vendor/tinyagents +++ b/vendor/tinyagents @@ -1 +1 @@ -Subproject commit 954b3dcf58038e8c11d674976fb779f1c72c6af4 +Subproject commit e0f3210622b4f654c822999a294ed654120c04a6