Skip to content

[Compatibility] CodeModeOnly external models still miss real exec tool events for nested apply_patch #2111

Closed as not planned

Description

@louis-tepe

Client or integration

Codex Desktop on macOS, signed native app

Provider or upstream service

  • xAI OAuth: xai/grok-4.6
  • Vercel AI Gateway: meta/muse-spark-1.2-contributor

OpenCodex version

2.26.0 (v2.26.0, commit b4336b75af3cf0428c575cad5d47b0f0b0afbfa3)

Endpoint or capability

Routed Responses -> openai-chat; CodeModeOnly exec -> nested tools.apply_patch(...)

Current behaviour

The 2.25/2.26 semantic Code Mode catalog nudge is present and is an improvement. A minimal redacted live canary still shows model/provider-dependent failures at the tool-call boundary.

The request advertises one freeform custom tool named exec. Its description identifies the V8 JavaScript body, declares tools.apply_patch(input: string), and gives the exact Begin/End delimiters and Add File + rule. The requested edit is adding one UTF-8 line to a new file. No generated code was executed.

Three fresh calls per available route:

Route Valid custom_tool_call(name=exec) containing tools.apply_patch Other result
xai/grok-4.6, tool_choice=required 1/3 2/3 returned a textual JSON-looking tool call instead of a Responses tool event; one then exhausted output
Vercel Muse, tool_choice=auto 0/3 3/3 ended incomplete without a tool event

A configured local Qwen route returned immediate provider-unreachable 502s and is recorded as unavailable, not as a compatibility failure.

One representative sanitized Grok text result began with:

{"name":"exec","arguments":{"input":"await tools.apply_patch(`*** Begin Patch\\n*** Add File: hello.txt\\n+café\\n*** End Patch`)"}}

The patch and JavaScript are semantically correct, but the result is message text rather than a tool event, so Codex cannot execute it.

Expected behaviour

For routed CodeModeOnly turns, the provider should emit a real call to the advertised top-level exec tool. The JavaScript body should invoke the nested helper:

await tools.apply_patch(`*** Begin Patch
*** Add File: hello.txt
+café
*** End Patch
`)

Transport control

Offline hostile-payload tests pass: the openai-chat {input:string} wrapper round-trips multiline patches, quotes, backslashes, template syntax, and Unicode exactly, while malformed envelopes fail closed. I therefore do not have evidence of deterministic bridge corruption or double escaping.

Suggested narrowly scoped improvement

Could the non-OpenAI Code Mode nudge add a compact canonical nested apply_patch example when, and only when, the semantic freeform Code Mode exec description declares that helper?

Constraints that seem important:

  1. Keep apply_patch nested; never advertise it as a top-level tool.
  2. Inject nothing for native OpenAI routes, structured exec, a visible bare shell bridge, or Code Mode descriptions without the helper.
  3. Preserve raw-string semantics; do not blindly copy the direct custom-tool Lark grammar into a JSON function schema.
  4. Cover wire-renamed exec, tool_choice, xAI required, Vercel auto-only, and hostile string round-trips.

This may still be a model/provider limitation rather than an adapter defect. Explicit per-route degraded capability reporting would be preferable to silently claiming parity if the nudge cannot make the output reliable.

Checks

Activity

  1. added
    providerProvider adapters, OpenAI-compat presets, upstream API quirks
    toolstool_calls, MCP, web-search / sidecar tools
    on Aug 19, 2026
  2. github-actions commented on Aug 19, 2026

    @github-actions
    Contributor

    Issue closed: insufficient detail

    Thanks for taking the time to submit this issue. It was closed automatically because the structured report is missing information needed to evaluate or reproduce it.

    • Both the request/reproduction and the actual response/error are absent.
    • Upstream documentation is empty without stating that no public specification exists.

    Please edit the issue to add:

    • Include at least a minimal redacted request or the actual error output.
    • Add a URL to the provider specification, or state that no public spec exists.

    See the Contributing guide for details.

    Once the report passes the automated checks, it will be reopened automatically unless a maintainer has changed its state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    providerProvider adapters, OpenAI-compat presets, upstream API quirksprovider-compatibilityProvider compatibility reportstoolstool_calls, MCP, web-search / sidecar tools

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions