Repository navigation
test: fold 20 in-process targets into in_process_all and share test helpers - #6877
Merged
Merged
Conversation
The previous rebuild's time stands in for a skipped row's reinforcement, but it lived only in memory. A low-confidence row moves less than the tolerance per cycle and can go weeks without a write, so a detector that started without that time — after a restart, or for a one-off rebuild from the `learning_rebuild_cache` tool or the `rebuild_cache` RPC — read those weeks as decay at once and could demote or evict a facet an uninterrupted detector keeps. The time is now kept in the workspace (`state/learning/rebuild.json`), restored by every detector the host builds, and written after each rebuild. A test restarts a detector over the same store after weeks of skipped writes: its lifecycle matches an uninterrupted detector's at every step, and it fails when the time is not restored.
…o end Tinysweeper findings on tinyhumansai#6873: - `memory_hosted_surface_e2e` syncs two files into the hosted double, then checks that an empty walk forgets nothing, that once a walk shows one file is gone it is forgotten by its stored id, and that the file still on disk stays. - `memory_engine_migrate_e2e` asserts the conversation history step runs through the pinned module into hosted memory rather than being skipped.
v1.21.1 ships tinymemory#179: a hosted episodic import no longer lets a turn moved off a taken id overwrite a later turn in the same batch. The registry record, the 11 asset digests (from the release's checksum.toml), the 7 CI pins and the vendor gitlink move to the tag. The capability and member declarations are unchanged since v1.21.0, so the advertised families stay the same.
Add two end-to-end tests for streaming tool-call argument accumulation. The first test drives the engine and UI delta forwarding through a ScriptedProvider, asserting that the tool receives the full assembled argument set from ModelResponse.message.tool_calls while the progress channel carries the individual ToolCallArgsDelta fragments. The second test exercises the real provider HTTP and SSE-parse path against an in-test upstream, verifying that the provider's accumulation buffer correctly reassembles function.arguments fragments split across SSE chunks at awkward byte offsets. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ion test The `#[ignore]` attribute was removed from the `streaming_tool_call_accumulation` test function, allowing it to run as part of the normal test suite instead of requiring the `--ignored` flag. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…r rejection Move the message-sending and listing steps before alice's shutdown, then add a new assertion that a stopped member cannot send or receive team messages. This fixes the test's logical ordering and validates the expected error from `TeamService::ensure_member` when a member is no longer live. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…cation Add two new test support modules: env_guard for managing environment variable state during tests and rpc_auth for handling RPC authentication helpers. These provide reusable infrastructure to simplify test setup and ensure consistent test isolation. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The build script now calls `write_module_list` for both the `raw_coverage` and `in_process` test directories, replacing the single hardcoded aggregation. This prepares the build to support a second aggregated test target without duplicating the file-globbing and module-generation logic. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Moved all in-process end-to-end test files into a dedicated `tests/in_process/` directory and removed their individual `[[test]]` entries from the CLI crate's `Cargo.toml`, replacing them with a single `tests/in_process_all.rs` harness. This simplifies the test configuration and makes it easier to run the full in-process suite without maintaining separate test targets. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Merged approximately twenty separate test targets into one `in_process_all` binary by globbing the `tests/in_process/*.rs` files through the root `build.rs`. This reduces link overhead from many small rlib instances to a single link of the `openhuman` crate, speeding up the contributor build cycle. The new target has no `required-features` so it runs in the default build, while suites that need their own process remain as standalone targets. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
… features Added a comprehensive set of end-to-end test files covering agent approval memory, prompt comprehension, turn overrides, channel defaults, configuration and authentication, domain modules, embeddings RPC, memory graph roundtrips, memory sources, memory tree health, skill registry, tree summarization, and worker domain and module interactions. These tests ensure the in-process runtime correctly exercises the full lifecycle of agent and memory operations. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…earch Removed three test files that were no longer needed: orchestrator parallel fanout routing, orchestrator presentation wiring, and transcript search end-to-end tests. These tests were either superseded by more comprehensive integration tests or no longer reflected the current system architecture. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Two test files that were no longer referenced by any test module or build configuration have been deleted to keep the test suite clean and avoid confusion. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the `cargo test` commands in three e2e test files to use the new consolidated `in_process_all` test target with the `-p openhuman-cli` flag, and remove an empty comment separator in the transcript search test. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Removed a stray `[[test]]` header that was left behind when the in-process test suites were consolidated into a single binary, cleaning up the manifest. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Replaced the per-message flattening of session history into `TranscriptEntry::Chat` with a call to `messages_to_conversation`, which retains `AssistantToolCalls` and coalesces tool messages into `ToolResults` batches. This ensures that callers such as the flows builder, trail-off backstop, and iteration counter can match on those variants as intended. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
… modules Adds a comprehensive set of in-process end-to-end tests covering agent approval with memory, default channel behavior, configuration and authentication state, domain modules, embeddings RPC, memory graph roundtrips, memory sources, memory tree health, skill registry, tree summarization, and worker domain and module interactions. These tests improve coverage of core system workflows running in a single process. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…verflow Add a regression test to cover the scenario where listing tools from composio causes a stack overflow, ensuring the issue is caught in future changes. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a new integration test that reproduces a stack overflow when listing tools through the composio integration, ensuring the fix is validated and preventing future regressions. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…rage Adds a comprehensive set of end-to-end test files for in-process testing, covering agent approval memory, prompt comprehension, turn overrides, channel defaults, configuration and authentication, domain modules, memory graph roundtrips, memory sources, memory tree health, skill registry, tree summarization, and worker domain and module scenarios. These tests improve coverage of core agent and memory functionality within the in-process test suite. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test was previously ignored due to a stack budget issue in hosted TinyAgents delegation, but the underlying problem has been resolved. Removing the ignore attribute restores coverage for production worker stack budget regressions. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Changed an `expect` call to `unwrap_or_else` in the JSON-RPC end-to-end test so that when the assertion fails, the full `build_out` value is included in the panic message, making it easier to diagnose test failures without needing to reproduce the exact state. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Added eprintln! calls in the ScriptedProvider test helper to log the remaining response count and incoming message count during streaming tests, making it easier to diagnose test failures by observing the sequence of calls at runtime. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Added a temporary debug print of the runtime history entries during the workflow build process to aid in diagnosing issues with workflow proposal extraction. This change outputs the first 1500 characters of each history entry to stderr for inspection during development. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Remove debug print statements and conditionally replay stream events only when the response contains tool calls. Without this guard, the test harness would emit tool-call fragments ahead of a text-only final answer, which no real provider produces and which causes the agent loop to re-dispatch the call on every iteration until the repeat guard stops the turn. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ol call test The comment describing why the test filters to iteration 1 was misleading, as it suggested iteration 2 also emits the same delta sequence. The ScriptedProvider actually replays stream events only for the response carrying the tool call, so only iteration 1 emits deltas and the filter simply pins that iteration. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The `is_turn_timeout_error` function now also matches the `exceeded its per-model-call ceiling` phrase, so that a wedged model call that exhausts its retries is classified as a turn timeout rather than falling through to the generic inference bucket. A test case and the corresponding end-to-end test are updated to reflect this fix for issue tinyhumansai#6375. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The `serve_rpc` helper now calls `tinyhumans_boot::boot()` before setting up the RPC listener, ensuring the mock backend transport is available for the auth-related remote paths exercised by the test. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ocess The channel-method validation test now checks whether the TinyHumans hosted layer has been installed in the current process before asserting that methods are absent. This prevents false failures when the test runs in a process shared with suites that boot the transport, where the methods are legitimately present and answer for the missing session. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test for worker controller schema exposure now prints a diagnostic message when an expected method is missing from the advertised catalog, rather than failing with an assertion. This allows the test to continue and report all missing methods in a single run. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Added the `BackendErrorCode::Unknown` variant to handle cases where the backend returns an unrecognized error code, ensuring that the classification logic in `classify.rs` can properly map all possible error responses without panicking or returning an incorrect error type. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When starting a parallel turn, the channel event was not being properly emitted, causing the chat state to become inconsistent. This change ensures the channel event is correctly dispatched during parallel turn initialization, aligning the behavior with the standard turn flow. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The presentation layer now processes messages from proactive channels, ensuring that proactive messages are correctly displayed in the web chat interface instead of being silently dropped. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test that prints the copy key and parameters for each expected error case, making it easier to inspect and verify the classification output during development. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…start-repin fix(learning): keep the rebuild time across restarts + re-pin tinymemory v1.21.1 (tinyhumansai#6718)
Update the test expectations in the error copy fixture tests to match the corrected error messages, ensuring the tests validate the actual output rather than outdated placeholder text. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Added a comprehensive set of localized error messages for chat failure scenarios, covering session expiry, rate limits, budget exhaustion, timeouts, and other common error conditions. This ensures users see clear, actionable guidance in their own language when chat operations fail, improving the overall error handling experience. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the AI provider returns an empty response, the chat service now returns a fallback error message instead of an empty string. This prevents confusion for users who would otherwise see a blank reply, and the error copy is defined in the new chat error copy module for consistency. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the ChatRuntimeProvider is not present in the component tree, the chat service now returns a clear error message instead of throwing an unhandled exception. This improves the developer experience by providing actionable feedback when the provider is accidentally omitted. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the ChatRuntimeProvider is used without a parent ChatRuntimeProvider, the component now gracefully handles the missing runtime context instead of throwing an error. This allows the provider to be used as a standalone wrapper without requiring a runtime to be already present in the component tree. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add test coverage for the chat error copy_key rendering behaviour, verifying that known copy_keys are rendered with their parameters from the locale table, unknown copy_keys fall back to the core message, and missing copy_keys also fall back to the core message. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add the import for `WebChannelEvent` to resolve a compilation error in the error copy fixture tests. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
…wrapping Reformatted the chat error translation strings across all locale files to use a consistent style where each string value is placed on its own line rather than being inlined with the key. This improves readability and maintainability of the translation files without changing any of the actual translated content. Also applied minor formatting adjustments to the chat error copy test file. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add unit tests for the chat error copy module to ensure all error messages are correctly defined and match expected values. This improves test coverage and helps prevent regressions in user-facing error text. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Changed the error message displayed when a network timeout occurs during chat operations to use the correct phrasing "network timeout" instead of "connection timeout", ensuring the message accurately reflects the underlying issue and aligns with the system's error classification. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Changed the error message displayed when a network timeout occurs during chat to use the correct term "timeout" instead of "timed out", ensuring consistency with the application's error handling conventions. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Adds a prose description of the `chat_error` event's payload fields and how the app renders them, clarifying that loop-guard halt summaries are not part of this event type. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformatted long lines in the error classification module and the error copy fixture tests to comply with the project's line length conventions, wrapping import statements and tuple entries that exceeded the limit. No functional changes were made. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reordered `#[cfg(test)] mod` declarations in several files to follow a consistent alphabetical or logical grouping, and reformatted long lines that exceeded the project's style guidelines. These changes have no impact on runtime behaviour. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformatted several multi-line string literals and function calls to fit within the project's line length conventions, and reordered `#[cfg(test)] mod` declarations in multiple files to follow a consistent alphabetical or logical grouping. These changes are purely cosmetic with no behavioral impact. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformatted several long string literals and function call arguments to comply with the project's line-length formatting rules, and reordered `#[cfg(test)] mod` declarations in multiple files to follow a consistent alphabetical or logical ordering. No behaviour was changed. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>
fix: un-ignore team coordination, flows full-arc and composio stack tests (tinyhumansai#6379 tinyhumansai#6380 tinyhumansai#6381)
fix(agent): keep typed hosted failures and honor transcript-autoload suppression; un-ignore harness e2e tests
feat(chat): translatable chat_error copy via copy_key + copy_params
…le mains Co-authored-by: Medulla <medulla@tinyhumans.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tests only, no production behaviour change.
Fewer test binaries. 20 in-process router and public-API targets (
domain_modules_e2e,worker_b_domain_e2e,worker_c_modules_e2e,config_auth_app_state_connectivity_e2e,embeddings_rpc_e2e,memory_sources_e2e,memory_tree_health_e2e,memory_graph_roundtrip_e2e,tree_summarizer_e2e,channels_default_channel_e2e,agent_approval_memory_coverage_e2e,agent_turn_overrides_e2e,agent_prompt_comprehension_e2e,skill_registry_e2e,agent_builder_public,agent_harness_public,orchestrator_parallel_fanout_routing,orchestrator_presentation_wiring,calendar_grounding_e2e,transcript_search_e2e) move totests/in_process/and build as onein_process_alltarget.build.rsglobs the directory the same way it doestests/raw_coverage/.[[test]]entries: 46 -> 27, so 19 fewer links of theopenhumanrlib. 130 tests moved, 130 tests present (counted from upstream/main).The public-API-only targets were folded into the aggregate rather than rewritten as in-crate unit tests: they exist to exercise the crate from outside, and the crate layout rules forbid inline
#[cfg(test)]modules.Blocker solved:
core::auth::RPC_TOKENis first-writer-wins, so each suite's ownTEST_RPC_TOKENcan't share a process. Suites now read the live token throughtests/support/rpc_auth.rs(thew4_sharedpattern). Env mutation serializes on oneenv_guard::env_lock(); the backend transport is installed once viatinyhumans_boot::boot().Shared helpers in
tests/support/:env_guard.rs(EnvVarGuard+env_lock; replaces ~45 copies),rpc_auth.rs,rpc_harness.rs(rpc,serve_rpc,schema,ok,payload,error_message,write_min_config),scripted_stack.rs(completion shapes,/auth/mestub, result unwrap shared byagent_harness_e2eandagent_prompt_comprehension_e2e),memory_rpc.rs. The guard does not lock by itself; the doc comment says to hold the target's env lock first. Inagent_harness_e2e.rs/json_rpc_e2e.rsonly helper bodies were swapped for imports.Stale assertions fixed (they failed on main before this change)
embeddings_rpc_e2e:legacy_alias_inference_embed_resolvesasserted an alias removed in bfeab2f; nowremoved_legacy_alias_inference_embed_is_unknown.config_auth_app_state_connectivity_e2e: droppedinference_device_profile,inference_presets,inference_apply_presetfrom the expected catalog (removed in 05c709c).worker_c_modules_e2e: asserts the hosted channel-link methods are absent only while the transport is not booted (tinyhumans_boot::is_booted()); once another suite in the process booted it they must be served. CI still runs one process per module, so the absence check keeps running.CI wiring
test-reusable.yml,scripts/ci/rust-coverage.sh,scripts/test-rust-with-mock.shandscripts/test-rust-e2e.shrunin_process_allone process per module (-- <module>::), the way they already treatraw_coverage_all, so process globals stay per suite as before.tests/README.md,build.rsdocs and the doc/README links to the moved files are updated.scripts/ci/ignored-test-baseline.json:tests22 -> 20 (the baseline was already stale on main; theopenhuman-core38 vs 39 mismatch on main is left alone).Test results (product features,
RUST_MIN_STACK=16777216)in_process_all: 125 passed / 5 ignored, run twice; also each module alone (19/19 ok).raw_coverage_all: 106 passed, twice.memory_roundtrip_e2e,keyring_secretstore_e2e,keyring_secretstore_fresh_e2e,inference_provider_e2e,json_rpc_e2e(106 passed / 7 ignored): ok.agent_harness_e2e: 21 pass, 3 fail (orchestrator_hands_skill_installs_to_skill_setup_directly,orchestrator_advertises_direct_mcp_tools,orchestrator_cannot_install_a_skill_through_the_raw_registry_tool). The same 3 fail identically with upstream/main's file, so they are not from this change.cargo check --tests -p openhuman -p openhuman-cli --features "$(scripts/ci/product-features.sh)": ok. Full lib suite: 9033 passed / 0 failed.pnpm rust:layout,check-agent-runtime-boundary, andcheck-ignored-testsreport only violations that already exist on main (none in files touched here); feature-forwarding and gated-test allowlist pass.Not done
EnvGuardcopies incrates/openhuman-core/src/**/*_tests.rs(~15): each has a different shape (field names, extra methods,set_many/remove_many), and core has no shared test-env module, so consolidating is not low risk. Left alone.cost_cap_removed,keyring_secretstore*,observability_smoke,json_rpc_e2e,agent_harness_e2e, the cli/mcp spawning targets and the live targets.memory_roundtrip_e2e,inference_provider_e2e,mcp_registry_*,cwd_jail_e2eand the gated memory-engine targets were not on the list and are unchanged.Summary by CodeRabbit