Repository navigation
fix(dash): run exactly the tool call the approval card shows - #980
Conversation
In `rocm dash` and interactive chat, the approval card for a mutating
assistant tool call (launch_server, install_sdk, install_engine,
stop_server, watcher_enable, watcher_disable) was rendered from the
call's argument object, but the approved replay went through
run_internal_mcp_call, which swapped in a nested `arguments` object
when one was present. A model-supplied call carrying such a field
showed one command on the card and ran a different one once the user
approved it.
The replay now validates and runs the object it is given, unchanged.
validate_chat_tool_call refuses any `arguments` field outright, so the
dash, chat and `rocm mcp-call` paths all refuse such a call before it
runs, and the dash and chat refuse it before a card is shown: no ROCm
tool declares a parameter of that name, so a call carrying one holds
two payloads and is refused rather than resolved to either.
The hidden `rocm mcp-call` command still accepts a pasted MCP
tools/call params envelope and unwraps it at its own boundary, but only
a pure envelope: `name`, `arguments` and the protocol's optional
`_meta`, which is dropped. It refuses an envelope naming a different
tool, and passes an object mixing tool fields with `arguments` through
whole, so validation refuses it. Arguments that are not a JSON object
are now refused rather than run as `{}`.
Tested by unit tests that the validator refuses a nested `arguments`
field for every ROCm tool name, that the dash card and the approved
replay agree, that the replay refuses a nested payload, that the
mcp-call boundary unwraps only a pure envelope, and that no tool in
either catalog declares an `arguments` parameter; and end to end by
chat-17, where a mock model answers with a two-payload launch_server
call and the model is told the call was refused and no approval prompt
is shown, chat-18, where a whole tools/call request handed to
`rocm mcp-call` runs the tool it names, and chat-19, where a request
with tool fields beside a nested `arguments` object is refused and
prints no tool result. The e2e mock server gains a scripted tool-call
reply keyed on the user's words, with its own harness test.
Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
a6a4b2f to
cde2775
Compare
Security scan found no critical or high findings on lines changed by this pull requestView the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
juhovainio
left a comment
There was a problem hiding this comment.
Looks good to me. The fix sits at the right layer: run_internal_mcp_call no longer unwraps a nested arguments object, and validate_chat_tool_call refuses it on every path, so the dash approval card and the command that runs can no longer diverge. The regression tests plus chat-17/18/19 cover the behavior, and the docs are updated. CI is green apart from the MI300X lane, which has no runners.
Two small non-blocking nits are inline. Also minor: the dash-side tool list and the validator's tool-name list are hand-maintained (the PR body already says so; safety doesn't depend on them since the refusal runs first), and one comment in chat.rs wraps wider than its neighbours.
This is based on reading the diff and CI results, not on running it locally.
| // A call that carries one holds a second payload next to the first, and | ||
| // the approval card and the command it approves must never be able to | ||
| // read different ones. | ||
| if call.arguments.get("arguments").is_some() { |
There was a problem hiding this comment.
Nit: .get("arguments").is_some() also refuses "arguments": null. Worth a Null case in validate_chat_tool_call_refuses_a_nested_arguments_field_for_every_tool so that behavior is pinned rather than incidental.
There was a problem hiding this comment.
Right: .get("arguments").is_some() is true for "arguments": null too, so that call is refused with the same nested-arguments error. That is the intended outcome (no tool takes an arguments parameter, so a null one is no more meaningful than an object), but you're correct that no test pins it: the every-tool test only sends {"arguments": {}}. I'm leaving it for a follow-up that adds a Null case to validate_chat_tool_call_refuses_a_nested_arguments_field_for_every_tool, rather than moving the approved head for a test-only change.
| .is_some_and(serde_json::Value::is_object) | ||
| && object | ||
| .keys() | ||
| .all(|key| matches!(key.as_str(), "name" | "arguments" | "_meta")); |
There was a problem hiding this comment.
Nit: an MCP params object carrying any other protocol key (e.g. task) counts as "mixed" and gets the nested-arguments refusal. Fine for a hidden command, but the refusal could say which keys make a pure envelope (name, arguments, _meta).
There was a problem hiding this comment.
Agreed on both counts. Any key outside name, arguments and _meta makes is_envelope false, so the object passes through whole and validate_chat_tool_call refuses it with "must not contain a nested arguments field", which doesn't tell the caller the envelope itself was the problem. It fails closed, so I've kept it out of this PR; a follow-up can have mcp_call_cli_arguments name the keys a pure envelope may hold when it sees an arguments object next to other keys.
juhovainio
left a comment
There was a problem hiding this comment.
Independent review
Traced the fix end to end against the code (not just the PR's own framing):
- Every path that can build an approval card or execute a tool now calls
validate_chat_tool_callfirst, and that validator unconditionally refuses a nestedargumentskey: the dash seam (BinToolExecutor::execute/execute_approvedindash_seam.rs), the interactive/one-shot chat turn loop (two call sites inchat/turn.rs, both before the read-only dispatch and beforechat_tool_approval_requestbuilds the card), andrun_internal_mcp_call/internal_mcp_tool_callinmcp_bridge.rs. The approval card (tool_policy.rs::chat_tool_approval_request/rocm_chat_tool_requested_args) and the replayed command both read fields from the same, already-validatedcall.argumentsobject, so they cannot disagree. mcp_call_cli_arguments's envelope unwrap is correctly scoped to therocm mcp-callCLI boundary only, runs before validation, and only unwraps a "pure" envelope (keys limited toname/arguments/_meta, with a name mismatch refused); anything else (mixed objects) is passed through whole and still caught by the downstream nested-argumentsrefusal. Checked the branch-by-branch logic (absent/non-string/mismatchedname) against the test cases and it holds.- The
install_sdk_dry_runarm's switch away from the old unwrap helper is safe:validate_chat_tool_callis still called upstream of it in the turn loop, so non-object/nested-argument input never reaches it. - Test coverage is non-vacuous: the dash-side
no_rocm_tool_declares_an_arguments_parametertest'sdefsarray matchesROCM_READ_TOOL_NAMES/ROCM_MUTATING_TOOL_NAMES1:1 (14+6=20, verified by reading both lists), so the length-equality assertion isn't trivially satisfied by omitting a tool from both sides. The newan_approval_dialog_shows_its_review_title_and_approve_buttontest exercises the realdraw_approval/draw_approval_inner/buttons_linerendering path, the same one the e2e absence-check depends on. The e2e scenarios (chat-17/18/19) target the actual refusal/approval boundary and their trigger string doesn't collide with other fixtures (MANAGED_MODEL_PROMPT). - Docs (
architecture.md,llm-tool-use.md,testing.md) accurately describe the new behavior and boundary, and the listedcargo test/cargo xtask e2efilters match real test/scenario names. - No AGENTS.md rule violations found, no related prior-fix history for this bug (confirmed via
git log -Son the removed unwrap helper — this was a long-standing latent bug, not a reintroduction or incomplete prior fix), and the PR description's claims match the diff exactly (968/+, 47/-, 15 files). - CI is green; the only non-success check is
E2E tests (MI300X, current)queued with no available runner, not a failure.
I independently arrived at the same two non-blocking observations already on this PR (the "arguments": null edge case being refused too, and the mixed-envelope refusal message not naming the allowed keys) — agree both are nitpicks, not blockers.
No blocking findings.
🤖 Automated by an AI agent
|
Thanks for both reviews. On the points in the review bodies that have no inline thread:
|
One conflict, in `apps/rocm/src/chat/mcp_bridge.rs`, and it is the code-was-removed shape rather than two additions colliding. Main's #980 deletes `internal_mcp_arguments` outright and replaces it with `internal_mcp_tool_call` plus `mcp_call_cli_arguments`, so that a pasted MCP envelope is unwrapped in one named step instead of silently inside the dispatcher -- the point of that change being that the approval card and the executed call cannot disagree. This branch still carried the old function. Resolved by taking main's design whole and keeping only this branch's own addition from the other side, the `engines` MCP tool-result helper. Keeping this branch's `internal_mcp_arguments` would have reinstated the unwrapping main had just removed, which is the opposite of what #980 is for; grep confirms no caller of it survives. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
One conflict, in `apps/rocm/src/chat/mcp_bridge.rs`, and it is the code-was-removed shape rather than two additions colliding. Main's #980 deletes `internal_mcp_arguments` outright and replaces it with `internal_mcp_tool_call` plus `mcp_call_cli_arguments`, so that a pasted MCP envelope is unwrapped in one named step instead of silently inside the dispatcher -- the point of that change being that the approval card and the executed call cannot disagree. This branch still carried the old function. Resolved by taking main's design whole and keeping only this branch's own addition from the other side, the `engines` MCP tool-result helper. Keeping this branch's `internal_mcp_arguments` would have reinstated the unwrapping main had just removed, which is the opposite of what #980 is for; grep confirms no caller of it survives. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Problem
In
rocm dashand interactiverocm chat, a mutating assistant tool call (launch_server,install_sdk,install_engine,stop_server,watcher_enableorwatcher_disable) pauses on an approval card built from the call's argument object. Once approved, the call was replayed throughrun_internal_mcp_call, which swapped in a nestedargumentsobject whenever one was present. A tool call carrying such a field therefore showed one command on the card and ran a different one after the user approved it: the card readrocm serve shown/model --managedwhile approval ranrocm serve other/model --managed --port 8123.Fix
run_internal_mcp_call(inchat/mcp_bridge.rs) validates and runs the argument object it receives, unchanged, throughinternal_mcp_tool_call, and never unwraps it.argumentsfield are refused.validate_chat_tool_callintool_policy.rsrefuses them. No ROCm tool declares a parameter of that name (pinned byno_rocm_tool_declares_an_arguments_parameteragainst the CLI's tool catalog, with a dash-side copy over the dash's tool definitions), so such a call holds two payloads, and it is refused rather than resolved to either one. The check runs on every path that validates a tool call: the dash and interactive chat refuse before a card is shown, and one-shot chat androcm mcp-callrefuse before anything runs, read-only tools included.rocm mcp-callkeeps a narrow envelope unwrap at its own boundary. The hidden command still accepts a pasted MCPtools/callparams object and unwraps it inmcp_call_cli_arguments, but only a pure envelope:name,argumentsand the protocol's optional_meta, which is dropped. It refuses an envelope whosenameis not the tool on the command line. An object mixing tool fields with anargumentskey is passed through whole and refused by validation.docs/llm-tool-use.mdstates the rule;docs/architecture.md(thechat/mcp_bridge.rsandtool_policy.rsrows) anddocs/testing.md(a new "Assistant tool-call arguments" block beside the direct MCP helper smoke) match it.User-visible changes:
argumentsfield is refused. In dash and interactive chat, the model is told why and no approval prompt appears.rocm chat "<prompt>", such a call ends the turn with that error, as any invalid tool call already does there.rocm mcp-callno longer unwraps anargumentsobject mixed with other fields; it refuses the call.rocm mcp-callrefuses--arguments-jsonvalues that are not a JSON object (null,[], a string), which it used to run as{}. Omitting the flag still means{}.Tests
Scenarios (
tests/e2e-cucumber/features/chat.feature, mock lane, no GPU):@id:chat-ambiguous-tool-call-refused-before-approval,@requires-os:linux): a mock model answers "please start a model server" with alaunch_servercall carrying two sets of arguments. It asserts that thetoolmessage the CLI sends back says the call was refused for its nestedargumentsfield and was not surfaced for approval, and that no approval prompt is on screen. The mock server gains a scripted tool-call reply keyed on the user's words, with its own harness test.@id:chat-mcp-call-runs-a-whole-tool-call-request): a wholetools/callparams object (with_meta) handed torocm mcp-call path_existsruns the tool it names and reports the requested directory.@id:chat-mcp-call-refuses-two-sets-of-arguments):rocm mcp-call path_existswith an existing outer path and a missing nested one is refused for its nested field and prints no tool result.chat-18 and chat-19 carry no OS tag, so they run on every OS the mock lane runs on; chat-17 drives the TUI through a pseudo-terminal and is Linux only.
Unit tests (
apps/rocmunless noted):validate_chat_tool_call_refuses_a_nested_arguments_field_for_every_tool,seam_refuses_nested_arguments_before_approval_and_on_replay,seam_refuses_nested_arguments_on_read_only_tools,seam_approval_card_matches_the_command_approval_runs,seam_approval_card_matches_the_command_for_plain_calls(guards against refusing valid calls),mcp_call_cli_unwraps_only_a_pure_params_envelope,mcp_call_refuses_arguments_that_are_not_an_object,mcp_call_refuses_tool_fields_mixed_with_a_nested_arguments_object,no_rocm_tool_declares_an_arguments_parameter(here and inrocm-dash-tui), andan_approval_dialog_shows_its_review_title_and_approve_buttoninrocm-dash-tui, which pins the two dialog strings chat-17 checks are absent.Each new test was seen to fail with its target broken:
validate_chat_tool_callseam_refuses_*tests, the mixed-objectmcp_calltest, and chat-17internal_mcp_tool_callunwrapping the nested objectseam_approval_card_matches_the_command_approval_runsrun_internal_mcp_call, refusal keptseam_refuses_nested_arguments_before_approval_and_on_replay,mcp_call_refuses_tool_fields_mixed_with_a_nested_arguments_objectmcp_call_cli_argumentscall removed from therocm mcp-calldispatch_metadropped from the envelope keysmcp_call_cli_unwraps_only_a_pure_params_envelopea_scripted_tool_call_answers_only_the_triggering_user_turnVerified locally (Linux)
cargo fmt --all --checkandcargo clippy --workspace --all-targets -- -D warningsare clean;cargo xtask check-architecture-docandcargo xtask check-module-sizepass.cargo xtask e2e -- -n "^(chat|dash)-": 69 of 73 scenarios pass, chat-17, chat-18 and chat-19 among them. The other four need a real served model with network downloads (chat-06, chat-07, dash-52) or a WSL host (dash-24).cargo test --workspace --all-targets --no-fail-fastfails only in tests this change does not touch: twobuild_scripttests that rungit commitin a scratch repo, andservice_control::tests::stop_of_a_container_engine_is_unconfirmed_when_the_engine_stop_fails, which fails the same way onmainon this host.Not verified
cargo xtask smokestops atrocmd sandbox examine snapshot, because bubblewrap cannot configure loopback in the local sandbox; every step before it passes.rocm mcp-callpath by chat-18 and chat-19.no_rocm_tool_declares_an_arguments_parameterlists the dash's tool definitions by hand, tied only by count to its tool-name lists, so a dash tool added later is not covered automatically. The CLI validator refuses such a call either way.match, so it applies to any tool name.