Skip to content

Harness: fatal tool-arg schema validation aborts the whole turn (legacy loop self-corrected) #4451

Closed

Description

@senamakel

Part of the v0.58.7 → HEAD TinyAgents-migration audit (see parent issue).

Problem

The tinyagents 1.3 harness runs fatal JSON-schema validation on every tool call before execution (tinyagents-1.3.0/src/harness/agent_loop/mod.rs:618 → validate_call, enforcing required, types, and enum). Any violation returns TinyAgentsError::Validation, which propagates out of run_loop and fails the entire turn with a generic "tinyagents harness run failed: Validation(...)".

The legacy engine (v0.58.7 engine/core.rs / engine/tools.rs) had no pre-execution schema gate: args went straight to tool.execute(), and argument errors came back as recoverable tool results the model could read and self-correct on the next iteration.

ArgRecoveryMiddleware (src/openhuman/tinyagents/middleware.rs:1449) makes it worse: it coerces non-object arguments to {} — which then guarantees a "<field> is required" validation abort for any tool with required params (most: e.g. src/openhuman/threads/tools.rs:121, src/openhuman/agent_memory/tools.rs:58, billing tools). Its own comment shows the failure mode was understood, but the fix only covers the non-object case.

Failure scenario

Model emits query_memory with {"q": "…"} instead of {"query": "…"} (or truncated/streaming-mangled JSON args) → whole chat turn errors → user sees chat_error. On the subagent path the whole subagent run dies the same way. This is a routine model behavior that the old loop absorbed; it will be hit in normal operation.

Fix plan

  1. Convert schema-validation failures into model-visible tool errors instead of run aborts. Options, in preference order:
    a. If the crate exposes a policy for validation failures (analogous to UnknownToolPolicy::ReturnToolError), enable it in assemble_turn_harness (src/openhuman/tinyagents/mod.rs).
    b. Otherwise, pre-validate in our adapter: in ArgRecoveryMiddleware::before_tool (or a new SchemaGuardMiddleware), run the same validate_call check ourselves; on failure, short-circuit the call and inject a synthetic failed ToolResult (error: Some("invalid arguments for <tool>: <detail>. Expected schema: …")) so the loop continues and the model self-corrects — mirroring the legacy behavior.
    c. If neither is possible, wrap SharedToolAdapter::execute tools with permissive schemas (required: []) at registration and let each tool's own argument handling produce the recoverable error (tools already do this — they predate the schema gate).
  2. Fix ArgRecoveryMiddleware to stop coercing to {} when the schema has required fields — recover what it can (e.g. parse JSON-encoded-string args, strip markdown fences), otherwise fall through to the tool-error path from (1).
  3. Upstream (tinyagents repo): file for a ValidationPolicy::ReturnToolError run-policy option.

Acceptance criteria

  • A turn where the model sends a missing-required-field / wrong-type / bad-enum tool call continues, the model receives a descriptive tool error, and typically self-corrects in the next iteration.
  • No path exists where a single malformed tool call converts to a turn-level chat_error.
  • Regression tests: chat-path and subagent-path e2e where a scripted provider emits one malformed call then a corrected one (extend tests/agent_harness_e2e.rs).

Activity

  1. added
    subtaskSubtask of a larger tracked effort.
    rust-coreCore Rust runtime in src/: CLI, core_server, shared infrastructure.
    agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.
    on Jul 3, 2026
  2. self-assigned this
    on Jul 7, 2026
  3. M3gA-Mind commented on Jul 7, 2026

    @M3gA-Mind
    Collaborator

    Already resolved by merged PR #4495 ("TinyAgents migration parity & bug fixes (#4451–#4469)") plus follow-up #4504 — the fix is present in main. Closing as completed.

  4. added theissue type on Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.bugpriority: criticalBlocks boot or core functionalityrust-coreCore Rust runtime in src/: CLI, core_server, shared infrastructure.subtaskSubtask of a larger tracked effort.

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions