test(saas): forget stays within the root; delivery recovery after restart (WIP) by senamakel · Pull Request #7294 · tinyhumansai/openhuman

senamakel · 2026-10-10T14:13:23Z

Draft: work in progress, handed off. It builds (cargo check --all-targets) and is formatted, but the tests have not been run on this head. See "To finish" below.

What

These are test-only follow-ups to the SaaS profiles work.

Forget stays within the caller's root (tests/saas_mode/memory.rs, from test(saas): memory round-trip isolation across profiles #7271). tinymemory v1.27.1 and forget_within are now on main (feat(memory): confine forget-by-id to the caller's reach (tinymemory v1.27.x) #7284), so this removes the exception that tolerated out-of-root listings and reads during a forget call. The test now fails if the whole-tree sweep comes back. A local mutation check confirmed this: switching memory/ops.rs back to the unscoped forget made the test fail. That mutation was reverted and never committed.
Delivery recovery after a restart (tests/saas_mode/delivery_recovery.rs, new). A profile's detached sub-agent finishes while its result cannot be delivered. On the next open of the profile (ProfileHost::open → recover_on_open, from fix(saas): key sub-agent delivery slots by profile #7273), the result must reach that profile's thread, and never another profile's thread with the same id. To support this:
- mock_llm gains HANG_AFTER_TOOL, which holds a turn open after its tool call.
- openhuman-cli gains a tinyagents-tasks dev-dependency, with its lockfile line.

To finish

Run cargo test -p openhuman-cli --test saas_mode_e2e memory and ... delivery_recovery.
- The delivery test was mid-rewrite when the work stopped. It is meant to seed the pending-delivery record that a dying core leaves behind, instead of killing a core mid-turn.
- Check that it fails if recover_on_open is removed.
Not started: extend tests/saas_mode/hashed_ids.rs (from fix(profiles): serialize profile lifecycle; e2e for hashed ids #7281) to scan file contents under the temp root, not just paths, responses and logs. Neither raw user id may appear.
Decide whether the tinyagents-tasks dev-dependency is needed. Drop it if the seeding approach doesn't use it.
Some of main's CI lanes are already red, independent of this PR: rust-layout, agent-runtime-boundary, saas-ambient and module-pins.

The memory isolation test no longer carves out an exception for forgetting by id, since `memory::ops::forget` now routes ids through the engine's scoped `forget_within` rather than sweeping from the tree's root. The assertion is tightened to require every named scope to sit inside the profile's root, and failure messages note when a request happened during the forget. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

…ol call The mock LLM now recognises a HANG_AFTER_TOOL marker in the user message and, once the turn's tool call has run, holds the follow-up request open like HANG instead of answering with the tool result. This lets tests keep a turn in flight after its tool call executed. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

Add tests exercising delivery recovery under saas mode so the retry and recovery paths are verified against the saas configuration. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

Wire the new delivery_recovery and mock_llm modules into the saas mode end-to-end test harness so their cases are compiled and run. The mock_llm module is marked allow(dead_code) since it is only used by some tests. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

Add the tinyagents-tasks crate as a dev-dependency so the SaaS delivery-recovery e2e can read the completion log a detached sub-agent leaves behind. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

The mock LLM no longer holds a follow-up request open after a tool result, so the HANG_AFTER_TOOL marker and its handling are gone. The user-message lookup now happens after the tool-result branch, which keeps the remaining hang and probe behaviour unchanged. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

Add tests covering delivery recovery in saas mode, verifying that in-flight deliveries are retried and completed correctly after a recovery event. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

Record the new tinyagents-tasks dependency in Cargo.lock so the lockfile stays in sync with the workspace manifest. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

The messages helper now extracts the messages field from the RPC response and panics with the full body when it is missing, instead of stringifying the whole response. This makes failures point at the actual problem rather than producing a confusing assertion downstream. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

# Conflicts: # tests/saas_mode_e2e.rs

Co-authored-by: Medulla <medulla@tinyhumans.ai>

coderabbitai · 2026-10-10T14:13:32Z

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI
Review profile: CHILL
Plan: Advanced
Run ID: 6e571cea-5aa9-40ea-b1b3-7eaf9ab0dfd7

📥 Commits

Reviewing files that changed from the base of the PR and between a107572 and 58a4384.

⛔ Files ignored due to path filters (1)

Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (4)

crates/openhuman-cli/Cargo.toml
tests/saas_mode/delivery_recovery.rs
tests/saas_mode/memory.rs
tests/saas_mode_e2e.rs

 ____________________________________________________________
< If you don't finish then you're just busy, not productive. >
 ------------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

tinysweeper · 2026-10-11T04:28:01Z

Tiny Sweeper review

Tiny Sweeper reviewed this change across 5 lane(s) and found 0 active actionable finding(s). Detailed lane evidence and any incomplete work are listed below.

State: Incomplete
Priority: none
Reviewed head: 58a43846b722
Updated: 2026-10-11T04:29:22Z

Review snapshot

Change surface	Files	Review signal	Count
Production	0	Active findings	0
Tests	3	Noted findings	0
Documentation	0	Resolved findings	0
Configuration	1	Pending checks/questions	9

Completeness: Incomplete
Test assessment: No supported feature-to-test mapping was available; this does not mean tests are absent or passed.

Features

None identified with supported citations.

Tests

No supported feature-to-test mapping was produced. Test execution is not inferred.

Findings

No active actionable findings.

Could not review: crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs, tinysweeper/description

Before merge

Complete the critique review for crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs.
Complete the security review for crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs.
Complete the description review for tinysweeper/description.

Agent review details

critique

Conclusion: Neutral
Scope reviewed: incomplete; unanswered: crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs
Lane summary: Reviewed 0 files; 0 findings. 4 files could not be reviewed: crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs.

security

Conclusion: Neutral
Scope reviewed: incomplete; unanswered: crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs
Lane summary: Reviewed 0 files; 0 findings. 4 files could not be reviewed: crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs.

tests

Conclusion: Neutral
Scope reviewed: all assigned evidence
Lane summary: No behavioural change: nothing outside documentation, configuration and tests.

commits

Conclusion: Neutral
Scope reviewed: all assigned evidence
Lane summary: Nothing sensitive found in what this pull request commits.

description

Conclusion: Neutral
Scope reviewed: incomplete; unanswered: tinysweeper/description
Lane summary: No reviewer produced a usable response.

Evidence and run details

Models: deep, openai/gpt-5.6-luna
Spend: $0.017666
Tokens: 64741 input · 2026 output · 2509 cached · 0 embedding

Head	State	Pass summary
`58a43846b722`	incomplete	0 active finding(s), 0 resolved finding(s) (at 2026-10-11T04:29:22Z)

chatgpt-codex-connector · 2026-10-11T04:28:05Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-11T04:35:32.829027Z	`58a4384`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

tinysweeper

tinysweeper found nothing blocking, but could not review everything, so this is not an approval: crates/openhuman-cli/Cargo.toml, tests/saas_mode/delivery_recovery.rs, tests/saas_mode/memory.rs, tests/saas_mode_e2e.rs, tinysweeper/description.

          $0.0177 · 64,741 in / 2,026 out · 2,509 cached (4%) · deep, openai/gpt-5.6-luna
critique: $0.0138 · 52,896 in / 1,265 out · 2,509 cached (5%) · openai/gpt-5.6-luna

senamakel and others added 11 commits October 10, 2026 16:17

test(saas_mode): cover delivery recovery in saas mode

b34b5c8

Add tests exercising delivery recovery under saas mode so the retry and recovery paths are verified against the saas configuration. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

build(openhuman-cli): add tinyagents-tasks dev-dependency

c91bb55

Add the tinyagents-tasks crate as a dev-dependency so the SaaS delivery-recovery e2e can read the completion log a detached sub-agent leaves behind. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

test(saas_mode): add delivery recovery tests

53ed17a

Add tests covering delivery recovery in saas mode, verifying that in-flight deliveries are retried and completed correctly after a recovery event. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

chore(deps): add tinyagents-tasks to lockfile

0da251f

Record the new tinyagents-tasks dependency in Cargo.lock so the lockfile stays in sync with the workspace manifest. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>

Merge remote-tracking branch 'upstream/main' into saas-leftover-tests

bcd9b32

# Conflicts: # tests/saas_mode_e2e.rs

style: cargo fmt

58a4384

Co-authored-by: Medulla <medulla@tinyhumans.ai>

senamakel marked this pull request as ready for review October 11, 2026 04:27

senamakel merged commit 3cc9770 into tinyhumansai:main Oct 11, 2026
13 of 17 checks passed

tinysweeper Bot reviewed Oct 11, 2026

View reviewed changes

tinysweeper Bot added the priority: p3 Whenever. Cosmetic, a nicety, or a cleanup with no user visible effect. label Oct 11, 2026

Navigation Menu

Repository navigation

test(saas): forget stays within the root; delivery recovery after restart (WIP) - #7294

test(saas): forget stays within the root; delivery recovery after restart (WIP)#7294
senamakel merged 11 commits into
tinyhumansai:mainfrom
senamakel:saas-leftover-tests

senamakel commented Oct 10, 2026

Uh oh!

coderabbitai Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

tinysweeper Bot commented Oct 11, 2026 •

edited

Loading

critique

security

tests

commits

description

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 11, 2026 •

edited

Loading

Uh oh!

tinysweeper Bot left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

senamakel commented Oct 10, 2026

What

To finish

Uh oh!

coderabbitai Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tinysweeper Bot commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tiny Sweeper review

Review snapshot

Features

Tests

Findings

Before merge

critique

security

tests

commits

description

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

tinysweeper Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Oct 10, 2026 •

edited

Loading

tinysweeper Bot commented Oct 11, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 11, 2026 •

edited

Loading