Repository navigation
opencode tool_calls problem with OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov #4599
Description
Describe the bug
When using opencode against ovms with OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov at some point after a few steps, I get json-output in the opencode console, I guess something with the API response of ovms is slightly wrong:
This does not happen with non-coder Qwen3 models I tested.
It does also happen with the coder model when omitting the --tool_parser qwen3coder --reasoning_parser qwen3 --enable_tool_guided_generation true options.
To Reproduce
Steps to reproduce the behavior:
- Launch the ovms command with
--pullto get the model - OVMS launch command:
ovms --source_model OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov --model_repository_path models --task text_generation --rest_bind_address 127.0.0.1 --rest_port 8000 --target_device GPU --tool_parser qwen3coder --reasoning_parser qwen3 --enable_tool_guided_generation true --cache_size 2 --log_level TRACE - Client command:
opencode - See error
Expected behavior
opencode should just work?
Logs
see https://gist.github.com/mmoll/723fd621dd2f2bfb1fb5871ce4b549ab
Configuration
- OpenVINO and ovms version: 2026.4.0
Additional context
Add any other context about the problem here.
Activity
mzegla commentedon Sep 24, 2026
Looks like tool calls from previous conversation rounds are not properly wired back to the promt on next chat turns resulting in context slowly degradating until at some point it starts generating tool calls in wrong format. I can see you are using OVMS without python support. Could you try the same with OVMS with enabled python?
Among release assets look for package with suffix
_python_on. Version with python uses different chat template rendering engine that might help here.mmoll commentedon Sep 25, 2026
As I'm using this on Debian/testing and there are no ubuntu26 ovms builds available I can't get a matching Python version... So I guess I have to switch to a dockerized version, or are there Ubuntu 26 builds coming soon?
mzegla commentedon Sep 25, 2026
Ubuntu 26 is planned to be added soon, possibly in 2026.5. In such case, I'd suggest trying dockerized version for now.
mmoll commentedon Sep 25, 2026
OK, will do that.
I'd like to leave this issue open for now and I'll test again with a Ubuntu 26 build. However, if the ovms build with python_on fixes it, it would still be nice to fix the issue for python_off also.
Brijesh-Thakkar commentedon Oct 31, 2026
I dug into this, and the cause seems slightly different from tool calls not being wired back. They are passed in, but they’re rendered as raw JSON.
The chat_template.jinja in OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov (and the int8 repo) differs from Qwen’s original by one line: tool_call.arguments | from_json. That filter is only registered by the Python Jinja engine. Under Minja the template fails during capability detection, so Minja falls back to its polyfill. That polyfill serialises earlier tool_calls as JSON into the assistant turn and turns role: "tool" into a user message containing {"tool_response": …}. The model then sees its own past calls as JSON and starts imitating that. The probe does detect this (probe.cpp:111-119), but servable_initializer.cpp:177 only logs a warning.
To verify, I rendered user → assistant(tool_calls) → tool → user with the openvino-genai 2026.4 tokenizer and the repo’s template files (no weights). chat_template.jinja produces the JSON blob. chat_template_original.jinja, which both repos already ship without from_json, produces proper <tool_call><function=…> output, since requiresObjectArguments handles the argument conversion.
I also scanned the chat templates of all OpenVINO org models on HF. Only these two repos use from_json. However, a few other templates also fail under Minja for different reasons (e.g. LFM2.5 and MiniCPM5 tool calls end up as JSON; Gemma 4 and Qwen3.8 fail to render at all in my probe), and all of these surface only as a warning.
I’d like to work on this. Proposed fix:
In OVMS: make a failed tool-call probe visible instead of silently corrupting the prompt (log an error and reject requests that include tools or tool history, or another behaviour you prefer), with tests in chat_template_end_to_end_minja_test.cpp covering a multi-turn tool-call round trip.
For the two Qwen3-Coder repos: switch chat_template.jinja to the existing from_json-free version, if that’s something your team can update on HF.
Does that direction work for you, and what should the failure behaviour in (1) be?
mzegla commentedon Nov 2, 2026
@przepeck - could you comment on this one? You've been working on this recently, so perhaps some of that is already addressed or in progress. Could @Brijesh-Thakkar help us in any way?
przepeck commentedon Nov 2, 2026
This issue was already addressed with my PR #4681
No additional work needed here.
Chat template of Qwen3-Coder in our repository wasn't fixed when chat template probing were introduced to preserve backward compatibility with previous versions of OVMS.
Brijesh-Thakkar commentedon Nov 4, 2026
Thanks @przepeck, I missed #4681.