feat: stateful sandbox sessions via toolExecution.sandbox sub-config - #291
Merged
Merged
Conversation
Surfaces the Code API's best-effort stateful runtime sessions without a
new ToolExecutionEngine value (the remote sandbox tools are host-
constructed with closure-held auth/files, and the backend speaks the
same /exec protocol — so a sub-config, not a transport swap).
- ToolExecutionConfig.sandbox { statefulSessions, runtimeSessionHint };
statefulSessions factory param on the 4 remote tools (prompt text only).
- ToolNode injects _runtime_session_hint into config.toolCall (explicit
hint else configurable.thread_id), independent of the transient
exec-session block, on both the direct and event-driven paths.
- execute_code + bash_tool send runtime_session_hint on the request and
get hedged 'best-effort' descriptions (usually persists, may reset,
only /mnt/data is durable); bash wording is filesystem-tier. PTC/BashPTC
plumb the wire hint on the initial request only but keep their stateless
prompt in v1 (flipping the 'fresh interpreter' contract is the biggest
behavior change; gate it separately once server sessions are proven).
- Artifacts + ExecuteResult echo runtime_session_id / runtime_status.
Fully additive: stateless servers ignore the field; the flag is
prompt-only and never hits the wire.
Verified end-to-end with a real Anthropic model (claude-sonnet-4-5)
driving execute_code across two turns against a session-mode runner:
turn 1 wrote /mnt/data/answer.txt, turn 2 read it back (new->reused),
every request carried runtime_session_hint. Unit: 30/30 ToolNode
session, 6/6 CodeExecutor stateful+wire, 9/9 BashExecutor.
Collaborator
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dc95f6a767
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| args: entry.args, | ||
| stepId: entry.stepId, | ||
| codeSessionContext, | ||
| runtimeSessionHint, |
There was a problem hiding this comment.
When event-driven sandbox execution is enabled with toolExecution.sandbox.statefulSessions, this field is passed into buildToolExecutionRequestPlan, but that helper's ToolExecutionPlanCall/prepared request construction only copies id, name, args, stepId, and codeSessionContext. The value is therefore dropped before the ToolExecuteBatchRequest reaches the host, so event-driven execute_code/bash_tool calls never carry the runtime session hint and won't reuse the configured stateful runtime even though the direct ToolNode path does.
Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment.
Fixed in 5f95acf. Confirmed valid: ToolExecutionPlanCall didn't declare runtimeSessionHint and buildToolExecutionRequestPlan never copied it onto the ToolCallRequest, so the field was silently dropped (excess property via .map) before the batch reached the host — the ToolNode event path resolved and passed it, but it was lost in planning, and the stream.ts eager path never passed it at all.
The fix carries the field through the plan builder (call + request, mirroring codeSessionContext), extracts a shared resolveRuntimeSessionHint so ToolNode's direct path and both event-driven planners derive it identically, and resolves+passes the hint in the eager path (same code-session gate as participatesInCodeSession). Added builder + resolver regression tests in eagerEventExecution.session.test.ts.
buildToolExecutionRequestPlan dropped runtimeSessionHint: ToolExecutionPlanCall didn't declare it and the builder never copied it onto the ToolCallRequest, so event-driven and eager execute_code/bash_tool calls never carried the hint and couldn't reuse the configured stateful runtime (only the direct ToolNode path did). Add the field to the plan call + request, extract a shared resolveRuntimeSessionHint used by ToolNode and the stream.ts eager planner, and resolve+pass the hint in the eager path. Regression tests on the builder + resolver.
Collaborator
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5f95acf1f1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+682
to
+685
| runtimeSessionHint: | ||
| CODE_EXECUTION_TOOLS.has(toolCall.name) || | ||
| graph.codeSessionToolNames?.includes(toolCall.name) === true | ||
| ? resolveRuntimeSessionHint(graph.toolExecution, threadId) |
There was a problem hiding this comment.
When eagerEventToolExecution.enabled and toolExecution.sandbox.statefulSessions are both on, this sends the durable runtimeSessionHint with streamed eager execute_code/bash requests before the model turn is finalized. The eager path is explicitly speculative (see the revised/superseded-call guard in takeMatchingEagerEventExecution), so if a provider later revises the tool args, ToolNode discards the eager result but the already-started stateful sandbox has still been mutated under the conversation's warm runtime, corrupting subsequent code executions. Please exclude code-execution tools from eager prestart when stateful sessions are enabled, or otherwise avoid using the durable runtime for speculative executions.
Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment.
Fixed in 3b31c19. Valid, and a good catch on the tension with the prior fix. isEagerExecutionExcludedTool now excludes CODE_EXECUTION_TOOLS when toolExecution.sandbox.statefulSessions is on, so execute_code/bash are never speculatively prestarted against the durable warm runtime (stateless mode keeps the throwaway-VM eager optimization, where discard is harmless). The eager planner also no longer attaches runtimeSessionHint at all — the durable runtime is only ever touched by the committed ToolNode path.
Comment on lines
+255
to
+260
| /* Stateful sessions: forward the hint so the Code API can route this | ||
| * execution to a warm per-session runtime. Additive — stateless | ||
| * servers ignore the unknown field. */ | ||
| if ( | ||
| typeof _runtime_session_hint === 'string' && | ||
| _runtime_session_hint !== '' |
There was a problem hiding this comment.
When the Code API backend supports runtime_session_hint, a tool call can include that extra argument even though toolExecution.sandbox.statefulSessions is off; because the request body is built from ...rest before this injected-hint gate, the model-supplied runtime_session_hint is already in postData and this block only overwrites it when ToolNode injected a hint. That bypasses the new opt-in isolation and lets the model choose/reuse a stateful runtime for execute_code (the same pattern exists in BashExecutor). Please drop/allowlist this wire field from raw tool args and only set it from _runtime_session_hint.
Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment.
Fixed in 3b31c19. Valid. runtime_session_hint is now destructured out of rawInput (and dropped) in both CodeExecutor and BashExecutor before ...rest is spread into postData, so the wire field can only ever be set from ToolNode's injected _runtime_session_hint. A model can no longer smuggle a hint through its tool args to opt into / pick a stateful runtime when statefulSessions is off. Regression tests added: model-supplied hint stripped, and injected-hint-wins over model args.
| * hint plumbs through, but the "each call is a fresh interpreter" wording | ||
| * is intentionally unchanged until server-side session behavior is proven. | ||
| */ | ||
| statefulSessions?: boolean; |
There was a problem hiding this comment.
When callers construct run_tools_with_code or run_tools_with_bash directly and pass this new public statefulSessions option, it has no effect: both PTC factories only read their existing init fields and only send runtime_session_hint when ToolNode injected _runtime_session_hint. Because this option is documented as sending the hint but is never read, direct users will believe they enabled stateful PTC while the request remains stateless unless they also configure ToolNode separately. Please either wire this option to a real hint source or remove it from the factory params.
Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment.
Fixed in 3b31c19. Valid — PTC is stateless in v1, and the factories never read this option, so it was a misleading public knob. Removed statefulSessions from ProgrammaticToolCallingParams (and the bash alias). The initial /exec/programmatic request still forwards a ToolNode-injected _runtime_session_hint when present; the factory-level opt-in will be re-added with real behavior when PTC stateful prompting lands.
…ints Addresses Codex review on 5f95acf: - Never eagerly prestart execute_code/bash when statefulSessions is on: the eager path is speculative, and a revised/discarded turn would leave writes applied to the durable warm workspace. isEagerExecutionExcludedTool now excludes CODE_EXECUTION_TOOLS under stateful (stateless keeps the throwaway-VM optimization). The eager planner no longer attaches runtimeSessionHint at all. - Strip model-supplied runtime_session_hint from raw tool args in CodeExecutor and BashExecutor: the hint is host-controlled and must only come from ToolNode's injected _runtime_session_hint, else a tool call could opt itself into / pick a stateful runtime with statefulSessions off. - Remove the no-op statefulSessions option from the PTC factory params: PTC is stateless in v1 and never read it, so it was a misleading public knob. Regression tests: model-hint stripping + injected-hint-wins in CodeExecutor.stateful.
Collaborator
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3b31c19dbe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+367
to
+368
| description: buildCodeExecutionToolDescription(params ?? undefined), | ||
| schema: buildCodeExecutionToolSchema(params ?? undefined), |
There was a problem hiding this comment.
When callers enable the new run-scoped toolExecution: { sandbox: { statefulSessions: true } } but keep using the existing createCodeExecutionTool()/createBashExecutionTool() factories, ToolNode now injects a runtime_session_hint, yet the model-facing description and schema here are still selected only from factory params. In that common configuration the backend runs statefully while the model is still told the environment is stateless, so the new sub-config silently misses the advertised prompting behavior unless every caller also duplicates the flag into each tool factory.
Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment.
Partly pushing back, with a docs fix in 2fa55c8.
The literal suggestion (drive the description from the run-scoped toolExecution.sandbox config) isn't reachable here: tool descriptions are bound to the LLM at construction time (createCodeExecutionTool/createBashExecutionTool set description via buildCodeExecutionToolDescription, and the tools are bindTools'd before the graph runs). The toolExecution.sandbox config is applied later, inside ToolNode at execution time, so by then the model has already been shown the description — there's no point at which the run config could change what it saw. The factory param is the only lever for description text, by construction.
So the two gates are intentional and different-lifecycle: factory param = construction-time prompt, run config = run-time wire hint. The host is expected to set both from one flag (LibreChat drives both off the single stateful_code_sessions capability), and the drift you describe is non-corrupting: a stateful backend with a stateless-described tool just means the model doesn't exploit persistence (re-runs setup), never a wrong result.
What I did fix: the SandboxExecutionConfig.statefulSessions and factory-param JSDocs now spell out the coupling and the exact failure mode explicitly (bound-at-construction, set-both-from-one-flag), so a caller reading either option can't miss that the run-config gate alone won't adjust the prompt. Collapsing to a single gate (derive hint injection from a stateful-built tool instead of a separate run-config flag) is a plausible future simplification, but it's a design change beyond this PR.
Codex flagged that enabling run-scoped toolExecution.sandbox.statefulSessions without the tool-factory statefulSessions param leaves the model told the environment is stateless while the backend runs statefully. The dual gate is intentional: tool descriptions bind to the LLM at construction time (before the run config is applied inside the graph), so the run config cannot retroactively change what the model was shown — only the factory param can. Spell that out on both fields and the required 'set both from one flag' pairing (non-corrupting if they drift: the model just won't exploit persistence).
Collaborator
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
JumpLink
added a commit
to faktenforum/agents
that referenced
this pull request
Jul 28, 2026
…fact preservation (#4) * ⏳ feat: Default Prompt Cache to 1-Hour TTL for Anthropic & Bedrock (#249) * feat: default prompt cache to 1h TTL with promptCacheTtl opt-out Make the 1-hour extended prompt-cache TTL the default for Anthropic and Bedrock whenever prompt caching is enabled, and add a promptCacheTtl ('5m' | '1h') client option to opt back into the legacy 5-minute cache. - Add buildAnthropicCacheControl / buildBedrockCachePoint / resolvePromptCacheTtl helpers and thread ttl through every cache-marker site (tail, stable-prefix, legacy, system prompt, tool definitions, summarization). - Preserve cachePoint.ttl through Bedrock Converse message conversion. - Re-apply the original breakpoint TTL when stripping an unsupported Anthropic assistant prefill. - Leave OpenRouter on the legacy 5-minute marker (no ttl). - Add unit coverage for the 1h default and 5m legacy paths; update fixtures. - '5m' omits the ttl field, keeping payloads byte-identical to prior behavior. * fix: gate Bedrock 1h prompt-cache TTL by model support (Codex review) - Add resolveBedrockPromptCacheTtl: an explicit promptCacheTtl is honored, but the omitted-default resolves to 1h only on Bedrock models that support the extended cache (Claude Opus 4.5 / Sonnet 4.5 / Haiku 4.5). Every other Bedrock model — including Sonnet/Opus 4.6 and all 3.x/4.0/4.1 — falls back to the 5-minute default so existing promptCache configs are never rejected for an unsupported ttl. Applied at the system, message-tail, and tool-cache sites. - Re-stamp a pre-marked Anthropic tool to the resolved ttl so a stale 5-minute tool breakpoint never precedes a 1-hour system/message breakpoint (Anthropic requires longer-TTL breakpoints to appear first). - Add unit coverage for the gated resolver (supported/unsupported/explicit) and Anthropic tool normalization; point Bedrock AgentContext fixtures at a 1h-supported model. * docs: clarify Bedrock promptCacheTtl model-gated default * fix: normalize all stale tool cache markers to resolved TTL (Codex review) - Anthropic: strip stray cache_control from earlier static tools (not just the tail) so a leftover 5-minute marker can't precede the resolved 1-hour tool/system/message breakpoint. - Bedrock: normalize an existing tool cachePoint to the resolved TTL instead of returning it unchanged, so a stale 5-minute tool breakpoint never precedes the 1-hour system/message breakpoints. - Add regression tests for both paths. * fix: harden tool cache TTL ordering edge cases (Codex review) - Bedrock: resolve the tool-cache TTL against the configured Claude model (captured at construction as promptCacheModelId) instead of this.model, which is temporarily swapped to the application-inference-profile ARN during generation — preventing a 5m tool breakpoint before 1h system/message points. - Anthropic: strip stale cache_control from deferred tools too (and the all-deferred path), since every tool serializes before system/messages. - Anthropic: detect/strip/clear a *direct* cache_control on non-built-in native tools (hasCacheControl/getCacheControlTtl/stripCacheControl/markCacheControl), so a stale or competing direct marker can't precede the resolved breakpoint. - Add regression tests for all three paths. * fix: preserve tool reorder + raw-tool direct marker (Codex review) - partitionAndMarkAnthropicToolCache: restore the deferred-tools guard on the return-original fast path. A correctly pre-marked static tool with a deferred tool ahead of it in the input must still return the partitioned array so the cache breakpoint precedes discovered/deferred tools (regression from r3). - markCacheControl: keep provider-shaped tools (built-ins AND raw Anthropic tool objects with input_schema) on a direct cache_control. extras is not promoted onto the payload for raw tools, so moving the marker there dropped tool-prefix caching instead of upgrading it to a 1h direct marker. - Update/extend tests (raw-tool direct upgrade; deferred reorder when unmutated). * refactor: tie Bedrock 1h default to promptCache, drop model whitelist (Codex review) Live probe (main .env) shows Bedrock 5m-only models (Sonnet 4.6, Opus 4.6) ACCEPT cachePoint ttl '1h' without error — they downgrade server-side rather than rejecting. The round-1 'unsupported models reject 1h' premise is therefore false, and per maintainer direction the 1h default is now tied to promptCache (not a model whitelist), matching the original request and Anthropic's behavior. - Remove BEDROCK_EXTENDED_TTL_MODELS / bedrockModelSupportsExtendedTtl / resolveBedrockPromptCacheTtl; Bedrock now uses resolvePromptCacheTtl (default 1h, override via promptCacheTtl) at every site. Drop the promptCacheModelId field added for gating. - Fix (Codex r5): a LangChain custom tool pre-marked with a direct cache_control (not under extras) no longer counts as 'already correct' — the check now reads the effective payload location per tool shape and re-stamps under extras. - Fix (Codex r5): sanitizeBedrockSystemMessage normalizes an existing system cachePoint to the resolved TTL, so a stale 5m system checkpoint never precedes a 1h message tail (Bedrock longer-TTL-first ordering). - Tests: drop the model-gating suites; add coverage for direct-marker LangChain re-stamp and stale-system-cachePoint normalization. tsc + 275 unit tests + lint clean; live Anthropic + Bedrock green. * feat: extend 1h prompt-cache TTL default to OpenRouter OpenRouter already shared the single tail-cache strategy with Anthropic but was left on the legacy 5m marker. Since OpenRouter uses the identical Anthropic cache_control format and documents the same { type:'ephemeral', ttl:'1h' } extended TTL (forwarded to its Claude upstreams, which downgrade gracefully), extend the 1h default to it for parity with the direct Anthropic and Bedrock paths. - Add promptCacheTtl to ChatOpenRouterCallOptions (stripped before the OpenAI client like promptCache); default 1h, opt out with '5m'. - Thread the resolved ttl through partitionAndMarkOpenRouterToolCache and the OpenRouter system block; generalize AgentContext.getPromptCacheTtl + the Graph message tail to resolve for OpenRouter (not just Anthropic). - Add OpenRouter tool-cache ttl tests; update the OpenRouter system + live fixtures to the 1h shape. - Verified live (OPENROUTER_API_KEY): wire payload carries ttl:'1h' and caching works through OpenRouter. tsc + 279 unit tests + lint clean; Anthropic + Bedrock + OpenRouter live all green. * fix: OpenRouter summarization TTL + stale tool marker stripping (Codex review) - summarization: resolve promptCacheTtl for OpenRouter too (not just Anthropic), so OpenRouter self-compaction uses the 1h default instead of 5m. - openrouter/toolCache: strip stale cache_control off earlier static and deferred tools before stamping the resolved breakpoint. _convertToOpenAITool passes already-OpenAI-format tools through as-is, so a caller-supplied/reused 5m marker would otherwise sit before the 1h breakpoint and violate ordering. - Add tests for both stripping cases. Re: gating the OpenRouter 1h default by model — live probe (OPENROUTER_API_KEY) shows google/gemini-2.5-flash returns 200 for cache_control {ttl:'1h'}, same as the bare marker OpenRouter already sent it, so non-Claude models accept and ignore it (no rejection). Consistent with the Bedrock decision, the default stays tied to promptCache. tsc + 177 unit tests + lint clean; OpenRouter live green. * v3.2.39 * refactor: rename read/write/edit `file_path` param to `path` (#250) * feat(coding-tools)!: rename read/write/edit `file_path` param to `path` Models intermittently emit the very common `path` instead of `file_path` for read_file/write_file/edit_file, which fails Zod validation ("Received tool input did not match expected schema") and derails the agent run — the model then retries or returns empty. Observed reliably with Kimi K2 via Fireworks/ClickHouse Inference (the investigator dies right after the first tool call). Rather than coerce the alias at every execution path, align the schema with what models actually emit — and with the sibling search tools (grep_search / glob_search / list_directory), which already use `path`. Now every built-in coding tool takes its file/dir target in `path`. - ReadFileToolSchema + Local{Read,Write,Edit}FileToolSchema: `file_path` -> `path` - handlers read `input.path` - workspace-policy extractors collapse to a single `path` extractor for all tools - tests updated BREAKING CHANGE: the read_file/write_file/edit_file parameter is now `path` (was `file_path`). Consumers that construct or read these tool-call args must update. Verified end-to-end against ClickHouse Inference (kimi-k2p7-code): a real investigator loop goes 4/10 -> 10/10 with no coercion. Tool + hook suites green (the one failing suite, directToolHITLResumeScope, fails identically on clean main — pre-existing). * fix(cloudflare-ptc): rename generated read/write/edit helpers to `path` Codex (#250): the Cloudflare Sandbox programmatic runners generate in-sandbox read_file/write_file/edit_file helpers that still took `file_path` (Python defs + the JS `payload.file_path` reads), so `write_file(path=...)` failed only in the Cloudflare programmatic path after the schema switched to `path`. Rename the tool wrappers' param to `path` (the internal `_resolve`/`_is_within_workspace` utils and the code-execution temp-file locals keep `file_path` — unrelated). * v3.2.41 * fix(cloudflare): client-side timeout on sandbox exec() to bound stalled containers (#252) * fix(cloudflare): client-side timeout on sandbox exec() to bound stalled containers The native Cloudflare Sandbox DO exec() is effectively uncancellable from the host (ExecOptions has no signal, so supportsExecSignal is false for the native transport) and its `timeout` option is not enforced when the container/RPC itself stalls, while the in-sandbox `timeout(1)` wrapper only bounds a command that is actually running. So a stalled exec (an unresponsive / cold container) hangs until the host's run-level abort, burning the entire run budget on a single tool call. The direct exec paths (executeCloudflareBash / executeCloudflareCode) had no client-side timeout at all. Add withClientTimeout(): a Promise.race backstop around every sandbox.exec() so the host await settles within clientExecTimeoutMs() (outerTimeoutMs + 5s) regardless of transport. On timeout the underlying exec may keep running in the DO (a native-DO exec can't be truly cancelled), so its late settlement is swallowed to avoid an unhandled rejection. The spawn path already timed out via spawnLocalProcess's kill timer; this closes the gap for the direct exec paths. Fixes #251. * fix(cloudflare): address Codex review on the exec client-timeout - unref the backstop timer so a timed-out / early-killed spawn doesn't keep the process/event loop alive until the timer fires (also fixes the jest open-handle warning) [Codex P2] - fire-and-forget the executeCloudflareCode temp-dir cleanup so a fully-stalled sandbox doesn't add a second client timeout to the caller's latency [Codex P2] - apply the client-side timeout to the bash_programmatic_tool_calling exec in CloudflareProgrammaticToolCalling.executeGeneratedCloudflareBash, which called sandbox.exec() directly and could still hang; export withClientTimeout + clientExecTimeoutMs to share them [Codex P2] * fix(cloudflare): address Codex cycle-2 review - unref the client-timeout backstop ONLY on the spawn path; the awaited direct-exec paths keep it ref'd so the timeout is guaranteed to fire even if nothing else is pending [Codex P2] - await temp-dir cleanup after a successful code exec; only detach it (unref'd) on a stalled/failed exec so normal runs still clean up before returning [Codex P2] - abort signal-aware execs on client timeout via a new execWithClientTimeout helper (creates a controller, passes signal when supportsExecSignal, aborts on timeout); applied to the direct bash/code + bash-programmatic paths [Codex P2] - test: signal-aware exec is aborted when the client timeout fires * fix(cloudflare): address Codex cycle-3 review - in withClientTimeout, reject the client-timeout FIRST and only then run onTimeout (abort), so a signal-aware exec's resulting AbortError can't win the race and surface to the caller instead of the timeout message [Codex P2] - route temp-dir cleanup through execWithClientTimeout (add an unref pass-through), so a stalled cleanup is actually aborted on signal-aware runtimes instead of leaking the socket until the bridge/network stack times it out [Codex P2] * fix(cloudflare): address Codex cycle-4 — preserve caller abort signal execWithClientTimeout overwrote a caller-provided options.signal with its private timeout controller, so on signal-aware runtimes a run/user cancellation no longer reached the exec until the client timeout fired. Now compose them: forward the caller's signal to the timeout controller so EITHER source cancels the exec. Test added for the composition. * fix(cloudflare): address Codex cycle-5 review - strip a caller-provided options.signal for native runtimes (supportsExecSignal false) so the {...options} spread can't reintroduce a signal the native DO RPC cannot clone/consume [Codex P2] - remove the composed caller-abort listener in a finally after the exec settles, so reusing a long-lived/shared caller signal across execs doesn't leak listeners or trigger MaxListeners warnings [Codex P2] - test: native runtime strips the caller signal * fix(bedrock): gate explicit prompt-cache checkpoints to Claude models (#253) * fix(bedrock): gate only the tool cache point to Claude models Amazon Nova rejects a `cachePoint` under `toolConfig.tools` — `Malformed input request: #/toolConfig/tools/0: extraneous key [cachePoint] is not permitted` — yet LibreChat auto-defaults Nova to `promptCache: true`, so the SDK injected the unsupported tool checkpoint and Nova 400'd. Live probe against us.amazon.nova-lite-v1:0 (Converse): - tool cachePoint -> 400 ValidationException - message cachePoint -> 200, cacheWriteInputTokens=240 - system cachePoint -> 200, cacheWriteInputTokens=241 So only the *tool* checkpoint is Claude-only; Nova caches system/messages fine. Add `supportsBedrockToolCache(model)` and gate ONLY the tool cache point on it, leaving message/system caching enabled for Nova: - bedrock/index.ts invocationParams — skip the toolConfig cache point on non-Claude models, using a model id captured at construction (survives the application-inference-profile ARN swap during generation) - Graph.ts — skip Bedrock tool marking on non-Claude models; message/system tail cache stays gated on promptCache alone - AgentContext.ts summarization path stays gated on promptCache alone This is a capability gate (the key is rejected outright), distinct from the 1h/5m TTL value which Bedrock downgrades gracefully. Fixes danny-avila/LibreChat#13838 * fix(bedrock): treat the default Claude model as tool-cache-capable (Codex review) When `model` is omitted, LangChain initializes the model to a default Claude model, but the tool-cache gates saw '' / undefined and wrongly skipped caching for that valid Claude path. - index.ts: capture `fields?.model ?? this.model` (super() sets this.model to the default Claude model) instead of `?? ''` - Graph.ts: treat an omitted clientOptions.model as the default Claude model (only an explicit non-Claude model skips tool marking) - test: omitted model + promptCache + tools still gets the tool cache point * v3.2.42 * fix(bedrock): clamp the extended 1h TTL to 5m on non-Claude models (#256) The extended 1h prompt-cache TTL is Anthropic-only on Bedrock. Non-Claude models reject it — Nova returns `ValidationException: Extended TTL prompt caching is only supported for Anthropic models` (verified live) — so the new 1h default broke Nova's message/system caching, which worked at 5m before. Add `resolveBedrockPromptCacheTtl(ttl, model)`: Claude (and the omitted-model default) keep the 1h default / configured ttl; non-Claude models clamp to 5m even when 1h is explicitly set. Wire it into the three Bedrock cachePoint TTL sites (index.ts tool, Graph.ts message tail, AgentContext summary). So Nova now caches system/messages at 5m instead of erroring. * test: cover custom OpenAI-compatible vLLM reasoning + qwen3_coder tool calls (#254) * test: cover custom OpenAI-compatible vLLM reasoning + qwen3_coder tool calls Regression coverage for a generic custom endpoint (provider openai, default reasoningKey, non-standard model name) that streams reasoning in the modern vLLM `reasoning` field (reasoning_content null throughout) and streams tool_calls with fragmented arguments (qwen3_coder). Locks in that delta.reasoning surfaces as think content and that the fragmented streamed tool call accumulates into a structured call. Mirrors the wire captures in LibreChat discussion #13849. * test: address Codex review on vLLM regression spec - Use a non-OpenAI baseURL so the model exercises the custom-endpoint (final-signal) path instead of the official-OpenAI sequential-seal path that an unset baseURL selects (isOfficialOpenAIBaseURL returns true). - Drive the qwen3_coder tool call through the full Run.processStream graph path: assert the streamed fragments are dispatched into a TOOL_CALLS run step and the get_weather tool executes with the assembled args, rather than asserting only on local AIMessageChunk concatenation. - Replace Record<string, unknown> converter inputs with typed OpenAI completion chunk/delta shapes (AGENTS.md: limit unknown/Record). * test: assert streamed tool-call deltas assemble args (Codex round 2) The TOOL_CALLS ON_RUN_STEP can be produced from the final assembled AIMessage (handleToolCalls) even if the streamed-delta path regresses, so asserting only on the step name would miss a streaming-UI regression. Capture ON_RUN_STEP_DELTA (handleToolCallChunks) and assert the fragmented qwen3_coder argument chunks assemble to {"location": "Berlin"}. * fix(cloudflare): client-side timeout on native sandbox file-IO RPCs (#255) * fix(cloudflare): client-side timeout on native sandbox file-IO RPCs PR #252 bounded the three `sandbox.exec()` sites with `withClientTimeout`, but the native-DO FILE-IO RPCs were left unwrapped. `createCloudflareWorkspaceFS`'s `readFile`/`writeFile`/`stat`/`readdir`/`mkdir`/`unlink`/`open` call `sandbox.readFile()`/`writeFile()`/`listFiles()`/`mkdir()`/`deleteFile()` directly — the same uncancellable native Durable Object transport (no `signal`, no reliably-enforced timeout). So a stalled/cold container hangs the host await on a single file read until the run-level abort, burning the whole budget on one tool call — the exact failure exec() had before #252. Observed live: an issue-triage `read_file` stalled ~552s before the wall-clock budget killed the run (the in-batch `execute_bash` was bounded at ~130s by #252; the parallel `read_file` was not). The v24 "read_file over bash" guidance had ironically moved traffic from the now-bounded exec path onto the still-unbounded FS path. Wrap every native FS RPC in `createCloudflareWorkspaceFS` (and the `listFiles` inside `findChildInfo`) with the same `withClientTimeout` backstop, using a new `clientFsTimeoutMs(timeoutMs) = timeoutMs + 5000` (file IO has no in-sandbox `timeout(1)` layer to honor, so a single headroom margin suffices). A normal, byte-capped read completes well within it; a stalled container can't outlast it. Tests: stalled `readFile` and stalled `listFiles` reject with a client-side timeout; a fast read resolves and leaves no dangling backstop timer. Cloudflare suite 28/28, typecheck + lint clean. * fix(cloudflare): address Codex review on FS-RPC client timeout P1 — distinguish FS timeouts from "file not found". The backstop now throws a `WorkspaceClientTimeoutError` (code WORKSPACE_CLIENT_TIMEOUT) instead of a plain Error, and the two ENOENT-only fallbacks rethrow it: the write_file pre-read in LocalCodingTools (was treating a stalled read as "file absent" -> overwrite) and FileCheckpointer.stat (was snapshotting a stalled stat as "absent" -> deletes the file on rewind). A stalled read can no longer masquerade as a missing file. P2 — keep the backstop active through the streamed-read drain. The race now wraps readFile + normalizeReadFileContent together (readFile/open/stat-fallback), so a sandbox.readFile that resolves to { content: ReadableStream } can't stall mid-drain after the timer cleared. P2 — bound execute_code temp-dir setup. executeCloudflareCode's pre-exec ctx.sandbox.mkdir / writeFile were unbounded, so a native-DO stall there hung the host before reaching the (already-bounded) exec path. Not addressed here (documented limitation): a timed-out *mutating* FS RPC abandons the host await but cannot cancel the native DO RPC, so a late write may still land — the same uncancellable-transport property #252 accepted for exec(). Poisoning the run on a mutating timeout is a larger behavior change left out of this focused backstop. Tests: distinguishable-error assertion on the readFile stall; streamed-read drain stall; execute_code mkdir-setup stall; FileCheckpointer rethrows a stat timeout instead of recording absent. Suites green (CloudflareSandboxExecution 30/30, FileCheckpointer + local-tool suites 306 passed), tsc + eslint clean. * fix(cloudflare): surface FS-RPC timeouts from generic swallow sites Round-2 Codex follow-on to the distinguishable WorkspaceClientTimeoutError: now that a stalled FS RPC throws a typed error, every generic catch that absorbs FS failures as benign must rethrow the timeout — otherwise a stalled container yields WRONG results (corrupted search, doubled latency) instead of failing. - engine stat(): a directory-probe listFiles timeout now rethrows instead of falling through to the readFile branch (which waited through a SECOND backstop, ~2x the timeout, before surfacing). - grep_search/glob_search Node fallback walker (LocalCodingTools): readdir timeout rethrows instead of skipping the directory (which silently corrupted results to "no matches"). - grep_search Node fallback scanner: both the per-file stat AND readFile timeouts rethrow instead of skipping the file (which could report "no matches" even when the stalled-unreadable file contained the match). The outer fallback catch already rethrows non-FallbackGrepError, so these surface as a real failure. Test: engine stat probe timeout rethrows and does NOT fall through to readFile. CloudflareSandboxExecution 31/31; local-tool + checkpointer suites 275 passed; tsc + eslint clean. * fix: surface FS-RPC timeouts from remaining probe & rollback swallow sites Round-3 Codex follow-on — completes the sweep of generic catches that absorb a WorkspaceFS error as benign. Now that a stalled native-DO RPC throws the typed WorkspaceClientTimeoutError, the remaining swallow sites must rethrow it so a stalled container fails loudly instead of producing wrong results / false success: Probes (wrong-result): - CompileCheckTool.pathExists: a stat timeout no longer returns false (which picks a weaker/wrong toolchain after waiting through the backstop). - CompileCheckTool package.json / pyproject.toml reads: timeout no longer substitutes '' (which silently misses typescript/mypy detection). - syntaxCheck jsonCheck read: a stalled read no longer returns ok:true (passing the syntax gate without actually reading the file). - syntaxCheck runPostEditSyntaxCheck: a checker timeout no longer collapses to null (which would let the write through without a real syntax gate). Rollback (false-success / data integrity): - revertStrictWrite: a timed-out revert no longer lets the caller claim "reverted to pre-write state" while the rejected bytes remain on disk. - FileCheckpointerImpl.rewind: a timed-out restore write or delete no longer counts the path as restored — rewind surfaces the timeout instead of silently leaving the workspace half-restored. Non-timeout failures keep their existing benign behavior everywhere. Test: rewind rethrows a restore-write timeout instead of claiming success. All affected suites green (FileCheckpointer/Cloudflare/local-tool 181 + 127 passed); tsc + eslint clean. * fix: handle second-order effects of surfacing FS-RPC timeouts Round-4 Codex follow-on — two consequences of making the new client-timeout surface (rather than swallow sites): - execute_code temp-dir leak: the client-bounded mkdir/writeFile setup ran BEFORE the try/finally, so a setup timeout exited without cleanup, orphaning .lc-exec/<uuid> (the uncancellable write can still land late on a cold container). Moved setup inside the try so the existing detached cleanup runs. - strict-mode revert bypassed: runPostEditSyntaxCheck now rethrows a stalled validation read, but that escaped write_file/edit_file before their strict rollback block, leaving the just-written bytes on disk while the call failed. Wrapped maybeRunSyntaxCheck so a validation-read timeout routes through revertStrictWrite (in strict mode) before surfacing — best-effort, since the revert may itself time out on the same stalled container. Tests: execute_code still issues `rm -rf` cleanup when setup writeFile stalls; write_file reverts (unlinks the new file) when the strict validation read times out. Affected suites 183 passed; tsc + eslint clean. * v3.2.43 * fix(reducer): skip null/undefined entries before coerceMessageLikeToMessage (#258) * fix(reducer): skip null/undefined entries before coerceMessageLikeToMessage messagesStateReducer maps both input arrays through coerceMessageLikeToMessage without first checking for null/undefined entries. When a provider emits an empty or partial stream chunk, that entry arrives as undefined, and coerceMessageLikeToMessage throws "Cannot read properties of undefined (reading 'role')", which crashes the LangGraph run. Filtering out null/undefined before coercion is a minimal, defensive fix that preserves message order and leaves all existing behavior (id assignment, REMOVE_ALL handling, merge/dedupe) untouched. Refs LibreChat Discussion #12284. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(reducer): coerce and skip null/undefined in a single pass Replace the filter().map() two-pass with a coerceMessages() helper that skips null/undefined while coercing in one loop. Message arrays are iterated frequently, and AGENTS.md calls out minimizing extra passes over them; this also de-duplicates the left/right coercion logic. Behavior is unchanged and reducer.spec.ts still passes. --------- Co-authored-by: Rommy <255708385+cosmic-fire-eng@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Danny Avila <danny@librechat.ai> * v3.2.44 * fix: Scope Langfuse tracing per run (#259) * fix: include exposed reasoning in Langfuse traces * fix: scope Langfuse tracing per run * feat: attach librechat langfuse routing attributes * fix(langfuse): honor env tool output tracing config * refactor(utils): share boolean env parsing * fix(langfuse): remove reasoning trace exposure * test(langfuse): cover parallel per-run routing * fix(langfuse): preserve redaction and routing config cascades * fix(langfuse): preserve active callback runtime scope * v3.2.45 * feat: Explicit `preserveReasoningContent` option for `formatAgentMessages` (#264) Lets OpenAI-compatible callers opt into cross-turn reasoning_content reconstruction without spoofing provider: deepseek. Explicit option wins; falls back to the existing DeepSeek thinking-mode default when unset. * v3.2.46 * chore: bump `@langchain/core` and `@langchain/openai` (#265) * chore: update @langchain/core dependency to version ^1.2.1 * chore: bump undici to 6.27.0 * chore: bump @langchain/openai to 1.5.3 with upstream test parity - Bump @langchain/openai 1.4.5 -> 1.5.3 (dependencies + overrides); reconcile the fork by dropping the obsolete @ts-expect-error in openrouter invocationParams that 1.5.3 fixes upstream. - Add src/llm/openai/llm.spec.ts inheriting upstream 1.5.3 converters/completions, completions class, and stream-events tests against the fork (50 tests); register it in eslint.config.mjs ignores. * chore: bump `@langchain/anthropic` to 1.5.1 with upstream parity (#266) * chore: bump @langchain/anthropic to 1.5.1 with upstream parity - Bump @langchain/anthropic ^1.3.28 -> ^1.5.1 (install + build clean). - Reconcile invocationParams against 1.5.x: gate `thinking` on the new thinkingExplicitlySet so we omit it (not send {type:'disabled'}, an unsupported param on some models) when the user didn't set it; forward `strict` to tool formatting; forward the top-level `cache_control` call option. - Fix handleToolChoice (stale copy): map OpenAI-style 'required' -> {type:'any'} and 'none' -> {type:'none'}; previously both fell through to {type:'tool', name:...}, so 'none' forced a tool named "none" instead of disabling tools. - Inherit upstream 1.5.1 tests against the fork: strict tool calling (17), stream-events (29), standard-content/message_outputs/tools (14), + thinking/cache_control invocationParams tests in llm.spec.ts. Register the new specs in eslint ignores. Live llm.spec tests left to CI. * chore: drop @anthropic-ai/sdk override, align to ^0.103.0 @langchain/anthropic 1.5.1 declares @anthropic-ai/sdk ^0.103.0, but the `$@anthropic-ai/sdk` override was forcing the whole tree down to our old ^0.92.0. Bump our dep to ^0.103.0 (the version langchain-anthropic expects) and remove the override: npm now dedupes to a single 0.103.0 copy naturally (vertex-sdk wants >=0.50.3 <1, langchain-anthropic wants ^0.103.0 — both satisfied), preserving the single-SDK-copy invariant the override existed to enforce. Build clean; 107 anthropic tests pass. * chore: bump @langchain/deepseek + @langchain/xai + @langchain/mistralai (#267) * chore: bump @langchain/deepseek + @langchain/xai with test parity - Bump @langchain/deepseek ^1.0.25 -> ^1.1.3 and @langchain/xai ^1.3.17 -> ^1.4.3. Both pin @langchain/openai 1.5.3 (matching our forced version), so a single openai copy; install + build clean, no fork changes needed. - Inherit upstream deterministic tests against the forks (vitest -> jest): ChatDeepSeek reasoning/<think>/streamEvents (14), ChatXAI completions/serialization/server-tools/streamEvents (23). Confirmed our ChatDeepSeek <think> reasoning parser reproduces upstream's content/reasoning split exactly. New specs registered in eslint ignores; live tests to CI. * chore: bump @langchain/mistralai to 1.2.0 Version-only bump (^1.0.8 -> ^1.2.0). ChatMistralAI is used directly (no fork) in the provider/type maps; build clean, no behavior change. * chore: bump @langchain/google-* to 2.2.0 (#268) * chore: bump @langchain/google-* to 2.2.0 Bump the four @langchain/google-* packages (common/gauth/genai/vertexai) 2.1.31 -> 2.2.0. The existing `$@langchain/google-*` overrides dedupe all transitive copies to a single 2.2.0 each. Build clean; the forks (CustomChatGoogleGenerativeAI, ChatVertexAI) compile against the new base classes with no fork changes. 19 deterministic google/vertexai unit tests pass; the live llm.spec suites run in CI. * test: inherit google/vertexai streamEvents tests against the forks Inherit upstream 2.2.0 streamEvents + stream-converter tests (vitest -> jest) against CustomChatGoogleGenerativeAI (9) and ChatVertexAI (7), verifying the forks' inherited native ChatModelStreamEvent path. convertGoogleGeminiStream unit-tested directly (exported); convertGoogleGenAIStream routed through the fork's streamEvents (not exported). Registered in eslint ignores. One intentional divergence asserted: the fork's legacy .stream() path applies system instructions via client.systemInstruction (old @google/generative-ai convention) rather than request.systemInstruction. * chore: drop redundant @langchain/google-* overrides The `$@langchain/google-*` overrides (added in #155 to pin around a broken core/uuid CJS release) are redundant at 2.2.0: our four google deps pin exact 2.2.0, the family inter-deps pin each other exact 2.2.0 (gauth->common, vertexai->gauth), and nothing else in the tree pulls google-*. Verified by reinstalling without them: a single 2.2.0 copy of each package, build clean, 16 inherited tests pass. The exact direct-dep pins are the real version lock now. (The separate `uuid` override is kept — independent.) * chore: bump @langchain/aws to 1.4.2 with cache_control + test parity (#269) * chore: bump @langchain/aws to 1.4.2 with cache_control + test parity - Bump @langchain/aws ^1.3.5 -> ^1.4.2 (build clean; @aws-sdk/* floats up to the ^3.1059.0 1.4.2 wants). - Forward request-level cache_control: 1.4.2 added a `cache_control` call option applied via the *unexported* `applyCachePointsToConversePayload`. Vendored it (src/llm/bedrock/cachePoints.ts) and wired it into our reimplemented `_streamResponseChunks`. The non-streaming path delegates to super, which already applies it, so only the streaming reimpl needed the fix. - Inherit upstream deterministic tests against CustomChatBedrockConverse: cache_control request mapping (2) + streamEvents/invocationParams/message_outputs (17). Confirmed our fork keeps Bedrock cache tokens additive (NOT folded into input_tokens) per the #13795 double-count fix. Registered in eslint ignores; live llm.spec -> CI. * chore: raise @aws-sdk/client-bedrock-runtime floor to ^3.1075.0 @langchain/aws 1.4.2 requires @aws-sdk/client-bedrock-runtime ^3.1059.0; our declared floor was a stale ^3.1013.0. It already resolved up to 3.1075.0 (one copy — the 3.x caret is wide, unlike anthropic's 0.x), so this is hygiene: make the declared floor match what the bump requires / the lockfile pins. No resolution change. * chore: bump @langchain/langgraph to 1.4.5 (#270) Version-only bump (^1.2.9 -> ^1.4.5); langgraph is the graph runtime, no LLM fork. Single copy, build clean, 2461 deterministic tests pass (live specs run in CI). The 1.3/1.4 additions (native streamEvents runtime, ToolNode runtime.state, RunControl draining, DeltaChannel, HITL resume+update+goto) are scoped separately as enablers for upcoming work. * v3.2.51 * feat: forward runtime.state to tools (langgraph 1.4.1), drop getCurrentTaskInput (#273) Adopt langgraph 1.4.1's ToolRuntime.state. Our forked ToolNode (extends RunnableCallable, not langgraph's prebuilt) now threads the run input (graph state) through run() -> runTool -> tool.invoke, building langgraph's exact runtime shape ({...config, state, toolCallId, config, context, store, writer}) so tools read graph state off their 2nd argument. Replaces getCurrentTaskInput() (relies on node:async_hooks, browser-incompatible, deprecated) in MultiAgentGraph's two handoff tools and the handoff-test dev script with runtime.state. Zero getCurrentTaskInput() calls remain. Adds ToolNode.runtimeState.test.ts proving forwarding (message-state, array, Send-de-enveloped) + toolCallId. LEFT INTACT (verified independent of state access, removing them is wrong): the run.ts __pregel_scratchpad.currentTaskInput leak cleanup (langgraph populates currentTaskInput unconditionally in pregel/algo.js) and the ToolNode interrupt() AsyncLocalStorage shim (interrupt() requires the ALS frame, interrupt.js). * feat(run): forward update + goto on resume (langgraph 1.4.5) (#272) * feat(run): forward update + goto on resume (langgraph 1.4.5) Run.resume() gains an optional 4th commandOptions param ({ update?, goto? }, typed off the Command constructor), threaded into the resume Command. A human-in-the-loop approval can now commit a state edit AND reroute in the SAME superstep (one checkpoint, no flicker) per langgraph 1.4.5, instead of resume-only. Backward-compatible: existing callers are untouched (update omitted, goto stays []). Adds a Run.create-harness test asserting forwarding + backward-compat. * docs(run): clarify update/goto resume semantics (codex P2 caveats) * test(run): prove langgraph applies update on an input resume Command (executing e2e) * v3.2.52 * feat: default durability to 'exit' when a checkpointer is active (#275) * feat(run): default durability to 'exit' when a checkpointer is active Runs that attach a checkpointer (the HITL MemorySaver fallback or a host-supplied saver) only need to persist at the graph's exit/interrupt boundary, not after every superstep. processStream now defaults `durability: 'exit'` whenever a checkpointer is present, so normal runs skip per-superstep checkpoint writes. An explicit caller `durability` still wins, and runs without a checkpointer keep langgraph's default. Extract the repeated processStream/resume config shape into `RunStreamConfig` and add the optional `durability` field (local `Durability` union, since langgraph does not export it from the root). Coverage: unit tests for the default / explicit-override / no-checkpointer cases, plus a mongodb-memory-server integration test (real Run + fake streaming model + official MongoDBSaver) proving the default writes one checkpoint document versus more than one under an async override. * fix(run): address Codex review on durability default - Detect the checkpointer at graph-creation time via a `hasCheckpointer` flag instead of reading `Graph.compileOptions` in processStream. For a standard graph with HITL enabled and caller compileOptions that omit a checkpointer, the constructor restores the raw caller options onto Graph.compileOptions and drops the fallback MemorySaver from that metadata, so the old check missed it and that HITL run kept langgraph's per-superstep default. Captured before the overwrite, the flag is correct. - Expose `durability` on AgentSessionRunOptions.config so session consumers can pass an object-literal override without a cast. - Move the mongodb-memory-server spec to durability-checkpoint.integration.test.ts (excluded from the unit shards, which avoids a mongod binary download in the default suite) and run it via a change-gated step in the integration job so coverage is preserved. Adds a HITL-fallback regression test for the first fix. * fix(deps): bump langgraph to 1.4.6, use mongodb-memory-server-core - Bump @langchain/langgraph to 1.4.6, which emits valid UUIDs for exit-mode delta task_ids (1.4.5 produced a 6-segment string that Postgres / LangGraph API checkpointers reject for the `checkpoint_writes.task_id` uuid column). Since this PR makes `durability: 'exit'` the default whenever a checkpointer is present, pin the patched version so a host-supplied Postgres saver with delta channels is not broken by it. - Switch the integration-test devDep from mongodb-memory-server to mongodb-memory-server-core, which has no postinstall binary download, so `npm ci` no longer fetches a mongod binary on every cache miss. The binary downloads lazily only when the gated integration test runs. * test(durability): assert the full storage optimization, not just doc count Strengthen the Mongo integration test to lock in the optimization end to end: - Normal run (real Run + fake model): assert exit writes exactly one checkpoint and zero per-superstep write docs, while async writes strictly more of both collections. - HITL interrupt (langgraph graph + real saver): assert exit checkpoints only at the interrupt boundary (one at pause, two once resumed) while async checkpoints every superstep and writes strictly more. * test(durability): cover multiple sequential interrupts under exit Adds a regression test for back-to-back ask-user-question rounds: each interrupt writes its own checkpoint at the boundary, the chain grows (1 -> 2 -> 3) rather than overwriting, both answers are applied, and the run resumes correctly each time. Confirms exit durability never discards a checkpoint the thread still needs across multiple interrupts. * v3.2.53 * fix(anthropic): coerce missing thinking text on signed blocks (#279) Opus 4.7+ omits thinking text by default, returning a signed but text-less thinking block. Replaying it built `thinking: undefined`, which JSON.stringify drops, sending a thinking block with no `thinking` field and tripping a 400 `messages.N.content.M.thinking.thinking: Field required` on the next iteration of a multi-step tool turn. Coerce the missing text to '' so the required field is always present; unsigned text-less blocks are still dropped as foreign reasoning. * v3.2.54 * fix: fail fast on tool calls truncated by the output token limit (#280) * fix: fail fast on tool calls truncated by the output token limit When a model turn is cut off by the max output token limit while it is still streaming a tool call, the arguments are necessarily incomplete (the tool-use block is the last thing the model emits). Providers surface this differently: Bedrock Converse buries it in response_metadata.messageStop.stopReason, Anthropic uses stop_reason, OpenAI finish_reason='length', Google finishReason='MAX_TOKENS'. The partial args parse into a well-formed-but-empty tool call (e.g. create_file missing its content), so the agent loops on a malformed request until it hits the recursion limit. Detect the truncation stop reason across providers and, when the turn carries a tool call, throw an actionable OutputTruncationError from the single invoke funnel instead of executing/looping on the incomplete call. Adds unit coverage for the cross-provider detector and a gated live Bedrock reproduction (RUN_BEDROCK_TRUNCATION_LIVE=1). * fix: skip truncation guard for atomic-tool-call providers (Google/Vertex) Google/Vertex (GenAI) deliver function calls as complete objects sealed on arrival, not streamed argument deltas, and treat MAX_TOKENS as a final chunk. A Gemini response with a valid functionCall ending in finishReason MAX_TOKENS therefore has complete args, so the provider-blind guard wrongly threw on it. Make assertNotTruncatedToolCall provider-aware: skip providers that deliver tool calls atomically. Streaming-arg providers (Anthropic, Bedrock, OpenAI) still fail fast on truncated tool calls. * fix: detect truncation for Anthropic streaming and OpenAI Responses Two provider shapes the detector missed: - Anthropic streaming stores the terminal stop_reason on AIMessageChunk.additional_kwargs (response_metadata only gets model_provider/context_management), so a streamed Claude tool call cut off at max_tokens slipped past the guard. - OpenAI Responses API signals truncation via response_metadata.status 'incomplete' + incomplete_details.reason 'max_output_tokens', not a stop/finish field. Read additional_kwargs.stop_reason and incomplete_details.reason in getTruncationStopReason, and normalize max_output_tokens. Non-token incomplete reasons (e.g. content_filter) still return null. * v3.2.55 * fix: let hosts exclude side-effecting tools from eager execution (#281) * fix: stop eager tool execution from corrupting large streamed args Two related eager-execution defects surfaced by create_file writing a large (multi-KB) Python file to the sandbox: 1. mergeToolCallArgsText's overlap-dedup heuristic stripped any >=8-char sequence where the accumulated text's suffix matched an incoming chunk's prefix. Indented code (8 spaces of double-indent, repeated tokens) trips this constantly, silently deleting hundreds of characters from the eager args. The corrupted eager args then (a) get written to disk and (b) diverge from the final LangChain-collapsed args, tripping the 'changed after eager execution' guard -> the model is told the write failed and loops. Fix: only dedupe an overlap when doing so yields a MORE complete parse than a plain append (a genuine partial resend recovers structure a doubled seam would break); otherwise concatenate and keep every character. Preserves the existing resend/cumulative dedup tests. 2. Side-effecting tools should not be executed speculatively at all: a speculative write can land even when the turn is superseded. Add EagerEventToolExecutionConfig.excludeToolNames so hosts can opt specific tools (create_file/edit_file) out of eager execution; excluded calls fall through to normal ToolNode execution with final args. Adds tests: large indented args accumulate without dropping characters, and excludeToolNames suppresses prestart. * refactor: withdraw mergeToolCallArgsText heuristic; rely on eager exclusion The parse-completeness heuristic can't disambiguate a genuine string-internal resend from a coincidental overlap — they're byte-identical patterns with opposite correct merges (Codex Finding 1). Rather than trade one corruption for another, revert the merge to its original behavior and rely solely on excludeToolNames: excluding side-effecting tools (create_file/edit_file) from eager execution routes them through the normal ToolNode path, which uses the final LangChain-collapsed args — correct for incremental-delta providers like Bedrock — with no speculative write and no merge guessing. * fix: apply eager exclusion after batch-level direct-tool guards Filtering excluded tools before createEagerToolExecutionPlan hid them from hasDirectToolCallInBatch, so an excluded tool that is also a direct tool could let a sibling event tool prestart in a mixed direct-tool batch. Move the exclusion into the plan's candidate selection, after the batch-level guards run against the full batch. Adds a regression test. * v3.2.56 * ci: parallelize summarization tests by provider (#282) src/specs/summarization.test.ts is a single ~3.7k-line file, and Jest runs tests within one file sequentially (--maxWorkers only parallelizes across files). The summarization-tests job therefore ran every provider E2E suite (Anthropic, Bedrock, OpenAI) plus all local suites back-to-back on one runner. Split the job into a fail-fast:false matrix that runs each provider group and the hermetic "no API keys" group as independent parallel jobs, selected via Jest -t name patterns: - anthropic: Anthropic E2E + Token accounting audit (both hit the Anthropic API) - openai: OpenAI E2E - bedrock: Bedrock E2E - local: all "(no API keys)" suites Keeping each provider in its own serial job avoids cross-job rate-limit contention while cutting wall-clock time to the slowest group. The patterns partition all 25 tests with no gaps or overlaps. * feat: let host tools share the code-execution session (codeSessionToolNames) (#283) * feat: let host tools share the code-execution session (codeSessionToolNames) Files written by a host sandbox tool (e.g. LibreChat's create_file) were invisible to later bash_tool/execute_code calls because the shared code session (sessions[EXECUTE_CODE]) is only updated for built-in CODE_EXECUTION_TOOLS. A host authoring tool wrote to one exec session; the next code tool started its own — so 'create_file then run it' failed with file-not-found. Add RunConfig.codeSessionToolNames: a host-declared set of tool names that write to the sandbox. Their successful results fold the returned exec session_id into the shared code session, so subsequent code tools reuse the same sandbox. Kept name-scoped (not a blanket artifact opt-in) so only host-declared tools can influence the shared session; threaded RunConfig -> Graph -> ToolNode like eagerEventToolExecution. Adds ToolNode.session tests: a declared host tool stores its exec session; the same tool undeclared does not. * fix: attach code session to declared host tools + wire traditional ToolNode Two gaps in the codeSessionToolNames wiring (Codex review): - Read direction: codeSessionContext was attached to requests only for built-in code tools, skill, and read_file. A declared host tool (create_file/ edit_file) called when a session already exists got no session_id/files, so it wrote into a fresh sandbox and lost prior in-session state. Now gated on participatesInCodeSession in the event request builder and the eager stream planner (getCodeSessionContext). - The non-event-driven ToolNode in initializeTools didn't receive codeSessionToolNames, so the option was ignored on the traditional path. Pass it there too. Adds a test: a declared host tool request receives the existing code session. * fix: inject session for direct host tools + auto-exclude session tools from eager Two more gaps (Codex review): - The direct runTool pre-invocation injection (session_id/_injected_files) was gated on built-in CODE_EXECUTION_TOOLS, so a declared host tool on the traditional path got no current session and wrote to a fresh sandbox. Gate on participatesInCodeSession. - A codeSessionToolNames tool writes to the shared sandbox, so it is side-effecting and must not be eagerly prestarted. isEagerExecutionExcludedTool now treats codeSessionToolNames members as excluded, so hosts don't have to duplicate the name in excludeToolNames. * v3.2.57 * ci: make anthropic stream spec immune to dual SDK type identity (#286) The spec imported Stream from @anthropic-ai/sdk/streaming while the base method's identity resolves through the fork chain. The SDK ships dual .d.ts/.d.mts types, so the two can resolve to different identities depending on ts-jest compile order — flaky TS2416 in sharded CI (seen twice on the same shard in agents PR 285 while passing elsewhere). Derive the mock's types from the base method instead. * fix: web search efficiency - rerank timeout, topResults wiring, configurable chunking (#284) * fix: bound rerank API calls with a timeout and wire topResults to the reranker Rerank requests (Jina/Cohere) were the only network calls in the web search pipeline without a timeout, so a hung rerank API could stall the whole tool; they now default to 10s, configurable via the new `rerankerTimeout` option on `SearchToolConfig`. Also passes the configured `topResults` through to `getHighlights` — previously it was destructured but never forwarded, so the reranker always used its default of 5 highlights per source regardless of config. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SZgqWvb3opKJp67YNPgbPg * feat: configurable reranker chunk size for web search highlights Scraped content is split into 150-char chunks (50 overlap) before reranking, so a source at the 50,000-char cap becomes ~500 rerank documents — the dominant per-search rerank cost (6 Cohere search units per source). Highlights are then expanded ±300-450 chars anyway, so the tiny chunks buy little precision. Adds `chunkSize`/`chunkOverlap` options (env: `SEARCH_CHUNK_SIZE`, `SEARCH_CHUNK_OVERLAP`) so hosts can tune this; measured on real scrape fixtures, 500/100 cuts rerank documents ~3.5x and Cohere billing to 2 units per source. Defaults are unchanged (150/50) pending a live quality comparison. Overlap is clamped below chunk size, which the splitter would otherwise reject. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SZgqWvb3opKJp67YNPgbPg --------- Co-authored-by: Claude <noreply@anthropic.com> * feat: add Keenable as a search provider (#285) * feat(search): add Keenable search provider Add Keenable to the provider subsystem, modeled on tavily-search.ts. createKeenableAPI maps /v1/search results into organic sources, so it reuses the shared scraping, reranking and highlighting pipeline. Keyless by default: falls back to the public endpoint and only sends X-API-Key when a key is set. Supports a site filter and client-side result count. No video search. * fix(search): map Keenable date filters * fix(search): skip unsupported sub-searches for Keenable Keenable is organic-only; its API ignores the type parameter, so image/news sub-searches spent rate limit and merged nothing. Gate them like videos (supportsImages/supportsNews). --------- Co-authored-by: Ilya Bogin <ilya.bogin@keenable.ai> * feat: host-suppliable AgentInputs.graphTools for in-graph direct tools (#289) * feat: host-suppliable AgentInputs.graphTools for in-graph direct tools Hosts running event-driven (definitions-only) tool execution had no way to make a specific tool execute IN-PROCESS inside the graph's ToolNode: the event partition routes purely by directToolNames, and the host-side ON_TOOL_EXECUTE handler runs outside the Pregel task frame — where a tool body calling LangGraph interrupt() (e.g. one built on the SDK's askUserQuestion() helper) throws 'Called interrupt() outside the context of a graph.' - AgentInputs.graphTools: host-supplied tool instances that join the existing graph-managed direct path — bound to the model alongside schema-only event tools, added to the ToolNode toolMap, and marked direct (Graph already wires agentContext.graphTools this way for handoff/subagent tools) - AgentContext.fromConfig copies the array (never aliases — the SDK pushes subagent tools into it later and must not mutate host input) - buildChildInputs clears graphTools for subagent children (including the self-spawn shape, which shallow-spreads the parent's _sourceInputs): child graphs compile without a checkpointer, so an interrupt-capable direct tool would deterministically fail with 'No checkpointer set' there Tests: direct tool in event-driven mode raises ask_user_question from its body (event dispatch never sees the call) and resumes with the answer as its ToolMessage; fromConfig copy + event-mode binding; child-input clearing. * fix: Codex round-1 findings — scoped child scrub, executable-only graphTools, honest tool count - buildChildInputs scrubs graphTools ONLY from self-spawn configs (whose agentInputs are a shallow spread of the parent's _sourceInputs); an explicit child config that lists its own graphTools is a deliberate host choice and keeps them (P2) - AgentInputs.graphTools narrowed from GraphTools to GenericTool[]: the wider union admits schema-only shapes (BindToolsInput / Google tool objects) that initializeTools cannot register in the ToolNode direct map — the model would bind a tool advertised as in-process but unexecutable (P2) - getToolCount now includes graphTools: graph-managed + host-supplied direct tools are bound and token-accounted, so omitting them under-reported the run's public tool surface (P3) * fix: seed traditional toolMap from base tools when graphTools force a merge ToolNode treats a supplied toolMap as authoritative (it only derives one from `tools` when the param is undefined), so the merged map built for graphTools must seed an absent currentToolMap from the BASE tools — otherwise ordinary tools stay bound to the model but vanish from the execution map, and every call to them fails as an unknown tool. Smoke test pins both names present. * feat: reshape Langfuse observation tree (#288) * feat: reshape Langfuse traces per Langfuse team feedback - Drop noise spans: langgraph __start__ seeds and anonymous RunnableLambda pass-throughs are no longer exported - Strip the ephemeral agent id (provider__model) from agent=/tools= node names so observation names stay stable across model switches - Rename tool node spans to the actual pending tool name(s) and scope their input to the tool-call args instead of the full chat history - Set root span and trace input/output to the user question and assistant response so the session view reads as a conversation - Rename title-chain runNames (ExtractTitle -> ParseTitleFromResponse, etc.) for clearer observation names * test: update title observation name in langfuse routing spec * v3.2.58 * feat: stateful sandbox sessions via toolExecution.sandbox sub-config (#291) * feat: stateful sandbox sessions via toolExecution.sandbox sub-config Surfaces the Code API's best-effort stateful runtime sessions without a new ToolExecutionEngine value (the remote sandbox tools are host- constructed with closure-held auth/files, and the backend speaks the same /exec protocol — so a sub-config, not a transport swap). - ToolExecutionConfig.sandbox { statefulSessions, runtimeSessionHint }; statefulSessions factory param on the 4 remote tools (prompt text only). - ToolNode injects _runtime_session_hint into config.toolCall (explicit hint else configurable.thread_id), independent of the transient exec-session block, on both the direct and event-driven paths. - execute_code + bash_tool send runtime_session_hint on the request and get hedged 'best-effort' descriptions (usually persists, may reset, only /mnt/data is durable); bash wording is filesystem-tier. PTC/BashPTC plumb the wire hint on the initial request only but keep their stateless prompt in v1 (flipping the 'fresh interpreter' contract is the biggest behavior change; gate it separately once server sessions are proven). - Artifacts + ExecuteResult echo runtime_session_id / runtime_status. Fully additive: stateless servers ignore the field; the flag is prompt-only and never hits the wire. Verified end-to-end with a real Anthropic model (claude-sonnet-4-5) driving execute_code across two turns against a session-mode runner: turn 1 wrote /mnt/data/answer.txt, turn 2 read it back (new->reused), every request carried runtime_session_hint. Unit: 30/30 ToolNode session, 6/6 CodeExecutor stateful+wire, 9/9 BashExecutor. * fix: carry runtimeSessionHint through event-driven request planning buildToolExecutionRequestPlan dropped runtimeSessionHint: ToolExecutionPlanCall didn't declare it and the builder never copied it onto the ToolCallRequest, so event-driven and eager execute_code/bash_tool calls never carried the hint and couldn't reuse the configured stateful runtime (only the direct ToolNode path did). Add the field to the plan call + request, extract a shared resolveRuntimeSessionHint used by ToolNode and the stream.ts eager planner, and resolve+pass the hint in the eager path. Regression tests on the builder + resolver. * fix: harden stateful sandbox against speculative + model-controlled hints Addresses Codex review on 5f95acf1: - Never eagerly prestart execute_code/bash when statefulSessions is on: the eager path is speculative, and a revised/discarded turn would leave writes applied to the durable warm workspace. isEagerExecutionExcludedTool now excludes CODE_EXECUTION_TOOLS under stateful (stateless keeps the throwaway-VM optimization). The eager planner no longer attaches runtimeSessionHint at all. - Strip model-supplied runtime_session_hint from raw tool args in CodeExecutor and BashExecutor: the hint is host-controlled and must only come from ToolNode's injected _runtime_session_hint, else a tool call could opt itself into / pick a stateful runtime with statefulSessions off. - Remove the no-op statefulSessions option from the PTC factory params: PTC is stateless in v1 and never read it, so it was a misleading public knob. Regression tests: model-hint stripping + injected-hint-wins in CodeExecutor.stateful. * docs: clarify the two-gate stateful-sandbox contract Codex flagged that enabling run-scoped toolExecution.sandbox.statefulSessions without the tool-factory statefulSessions param leaves the model told the environment is stateless while the backend runs statefully. The dual gate is intentional: tool descriptions bind to the LLM at construction time (before the run config is applied inside the graph), so the run config cannot retroactively change what the model was shown — only the factory param can. Spell that out on both fields and the required 'set both from one flag' pairing (non-corrupting if they drift: the model just won't exploit persistence). * v3.2.59 * feat(hitl): declare multiSelect on AskUserQuestionRequest (#293) * fix: tool error completion contract across interrupt/resume passes (#292) * fix: tool error completion contract across interrupt/resume passes A direct (graphTools) tool that fails fast — e.g. a zod schema reject — on the RESUME pass of an interrupted batch errors before the rebuilt graph has registered run steps for the replayed calls. handleToolCallErrorStatic threw ('No config provided') in that state, surfacing a scary 'Error in errorHandler' log on every such resume even though the error itself was already handled (the model receives the error ToolMessage regardless). - handleToolCallErrorStatic now returns whether it dispatched the error completion instead of throwing on missing config/stepId/runStep; the unused config precondition is dropped (the dispatch never consumed it). - ToolNode records calls whose errorHandler did NOT dispatch (returned false or threw) and lets its own completion loop dispatch for exactly those — the skip-when-errorHandler-exists shortcut previously assumed the handler always succeeded, stranding the client's tool-call part without a terminal event whenever it didn't. - Net contract: the error completion dispatches exactly once, on the pass where the call's run step is live; resume re-executions of an already-reported error stay silent toward the client (no duplicate cards), and the model keeps receiving the error ToolMessage on every pass. Repro (LibreChat #14139 field report): ask_user_question sibling batch where one call exceeded the 12-option schema cap; verified with a dist probe (pause -> resume on a fresh instance) and covered by src/specs/tool-error-resume.test.ts. * test: fix strict-tool assertion to match the schema error wrapper The strict tool used z.number().max(1), whose zod message is 'Number must be less than or equal to 1' — the 'at most 1' assertion never matched (the suite can't load locally due to the pre-existing mistralai ESM issue, so this was missed pre-push). Switch to an array-cap schema (z.array().max(1)), which faithfully mirrors the field repro — an ask_user_question sibling whose options array exceeded the 12-cap — and assert on the version-independent LangChain wrapper 'did not match expected schema'. Verified locally with the mistralai import stubbed. * fix: consume undispatched-error markers instead of accumulating them Codex #292 review: ids added to `undispatchedToolErrors` (when errorHandler reports it could not dispatch) were never removed — an add-only set. Now the completion loop DELETEs the id when it takes over the dispatch (consume), and `run()` clears the set at batch entry so a marker left behind by an invocation that interrupted before its completion loop can't survive into a resume re-execution. Probed the specific double-dispatch scenario (same-instance resume re-entering the same tool_call_id): NOT reproducible in the current SDK — the marker is only added on a fresh rebuilt instance whose step replay hasn't registered the id yet, and that instance never re-enters with the same id succeeding. So this is defensive hardening against a future dispatch-ordering change plus removing the add-only-set growth, not a live-bug fix. Existing tool-error-resume spec (exactly-once across resume) still green; verified no regression via rebuilt- and same-instance dist probes. * fix: drop the racing run()-entry clear of undispatched-error markers Codex #292 re-review: clearing the shared `undispatchedToolErrors` set at the start of every run() races with a concurrent/overlapping invocation on the same ToolNode — a second batch's clear wipes the first batch's markers before its completion loop runs, so the first error ToolMessage hits the errorHandler-skip path and never dispatches a terminal completion (the very missing-completion case this PR fixes). Remove the clear. The consume-and-delete in the completion loop is the sole mechanism now, and it is concurrency-safe on its own: markers are keyed by globally-unique tool_call_id, so concurrent batches touch disjoint entries and never each other's. Each marker is reclaimed when its completion dispatches; the only residual is a marker left by an invocation that interrupted before its loop ran, which rides a short-lived rebuilt instance and is GC'd with it. Both dist probes (rebuilt- and same-instance resume) stay exactly-once. * fix: tighten the error-completion dispatch contract (Codex #292) Two P2s on the boolean contract: 1. Graph.ts handleToolCallErrorStatic returned true even when NO ON_RUN_STEP_COMPLETED handler was registered (the optional-chained dispatch was a no-op). Hosts that wire completions through callback-based custom events instead of a registered handler would have the error completion silently dropped. Now look up the handler first and return false when none exists, so the ToolNode runs its fallback dispatch. 2. ToolNode runTool marked a THROWN errorHandler as undispatched. But a throw is not proof nothing dispatched: the built-in session handler emits tool.completed BEFORE invoking a user ON_RUN_STEP_COMPLETED callback, so a throw there has already dispatched — re-marking it made the fallback loop emit a DUPLICATE completion (double stream event / UI card). Only an explicit false return now means 'nothing dispatched'; a throw is just logged. Both dist probes stay exactly-once; tsc clean. * feat(hitl): guard sibling double-execution when ask_user_question interrupts mid-batch (#294) * feat(hi…
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds opt-in, best-effort stateful runtime sessions for the remote Code API sandbox, so a conversation's
execute_code/bashcalls can reuse one warm per-session workspace instead of a fresh sandbox per call. The transport is unchanged; this only sends a stable session hint and hedges the model-facing tool descriptions.How
toolExecution.sandboxsub-config (SandboxExecutionConfig { statefulSessions?, runtimeSessionHint? }). This is intentionally not a new engine value: the host constructs the sandbox tools with closure-held auth/files, soengine: 'sandbox'stays the default and dispatch is untouched.ToolNodestamps_runtime_session_hintonto the tool call only whensandbox.statefulSessionsis true (explicitruntimeSessionHint, elseconfigurable.thread_id).runtime_session_hinton the initial request (PTC/BashPTC on the initial request only; continuations bind server-side).statefulSessionstool-factory param adds hedged descriptions forexecute_code+bash_tool(usually share one runtime, may reset at any time, anything durable must be written to/mnt/data). PTC keeps its stateless prompt in v1.runtime_session_id/runtime_statusfor future UI.Safety
Fully additive and off by default. Old-SDK + new-server and new-SDK + old-server both degrade to stateless. The server derives the real session id as
hash(tenant, user, hint), so the hint is never a trust boundary.Tests
45 tests across
ToolNode.session,CodeExecutor.stateful, andBashExecutor.