Repository navigation
Local-model serving: optional chat-completions transport and constrained ID fields - #2
Open
const-apex wants to merge 1 commit into
Open
const-apex wants to merge 1 commit into
const-apex wants to merge 1 commit into
Conversation
… local servers (vLLM, llama.cpp)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds two opt-in configuration options for running SwarmWorld against open-weight servers (vLLM, llama.cpp) and one diagnostic improvement.
All defaults reproduce current behaviour; the paper profiles are unaffected, and the prompt and action-schema hashes recorded in trace headers are unchanged unless a flag is set.
Changes:
llm.transport: chat_completions(default:
responses).llama-server's
/v1/responsesconverts requests to chat completions internally and dropstext.format, so JSON comes back unconstrained (18 of 33 plans failed local validation in our test).With this option:
{base_url}/chat/completionswithresponse_format: {type: json_schema, json_schema: {name, schema, strict}}, the same schema object the Responses path sends.max_tokens,temperatureandreasoning_effortmap one-to-one;input_tokens/output_tokens/total_tokensso traces and the compute audit stay uniform;finish_reason: lengthraises the same incomplete error;<think>...</think>block is stripped when a server returns reasoning inline.llm.extra_bodymerges arbitrary request fields into either transport (used forchat_template_kwargs.enable_thinking: falsewith Qwen on vLLM).llm.strict_id_patterns(default
false).Small models copy the prompt's "use NONE when irrelevant" into
reply_toandtarget_artifact_id(29 and 22 of 42 actions with Gemma 4 E4B), and the engine rejects every such action.With the flag on:
patternin the schema (^(message_[0-9]+)?$,^(agent_[0-9]+)?$,^(artifact_[0-9]+)?$), so constrained decoding cannot emit placeholders;pattern(vLLM:guidance; xgrammar rejects it).Diagnostics
On
finish_reason: lengththe error string now records the length, head and tail of the truncated reply, so runaways can be classified from the trace.Tests
Tested:
ruff check src testsclean;prompt_sha256in trace headers equals the released traces' value.Related: #1 (late state-digest divergence when replaying the released traces on revision 9); this PR does not touch replay.