Skip to content

fix(web): group image views like other tool calls - #9597

Merged
juliusmarminge merged 6 commits into
mainfrom
t3code/fix-image-tool-grouping
Sep 4, 2026
Merged

fix(web): group image views like other tool calls#9597
juliusmarminge merged 6 commits into
mainfrom
t3code/fix-image-tool-grouping

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Sep 4, 2026

Copy link
Copy Markdown
Member

Image views were forced out of tool groups and settled-turn folds by #9126. After #9460 hid their previews until expansion, bare image paths still split otherwise ordinary tool groups.

Remove that grouping exception. Image views group and fold like other reads, including in the live activity row. Expanding a tool row still uses the existing image-preview renderer.

All six providers now share a concise <runtime_info> block identifying T3 Code and their harness, ending with: "You can embed images and videos in your response using Markdown with absolute file paths."

Codex receives it in developer instructions, Claude in its system-preset append, and OpenCode in each prompt's system field. Cursor, Grok, and Antigravity receive a separate ACP text block after the user content, without changing the stored message. Per-turn context includes the model when known. Claude omits model/effort from its session-level prompt to avoid stale values; OpenCode variants are not assumed to be reasoning effort.

Before

Base 2675e3c70, the PR merge base. The same copied thread, expanded turn, scroll anchor, dark theme, and 1280 × 1000 viewport are used in both captures.

Before: image-view paths split the surrounding tool groups

After

Captured at 73c468825; the later prompt-only revision does not change the UI.

After: image views join surrounding tool groups as file reads

Verification

  • On fcffb6e6f, all 307 focused tests pass across the shared runtime instructions, Codex session runtime, and Claude, OpenCode, Cursor, Grok, and Antigravity adapters. Grok and Cursor tests verify runtime context stays out of saved prompts. Server typecheck, targeted lint, and diff checks pass. CI, Cursor Bugbot, and Macroscope correctness and Effect conventions checks pass on this revision; both review threads are resolved.
  • At 73c468825, 294 focused tests passed across CodexSessionRuntime.test.ts, OpenCodeAdapter.test.ts, MessagesTimeline.logic.test.ts, MessagesTimeline.test.tsx, and ChatMarkdown.test.tsx.
  • Preview deployments are label-gated and skipped. Mobile native static analysis is skipped because no native files changed; the native-change detection job passes. PR-size definition sync is inapplicable to this event, and Codesmith is not enabled.
  • Macroscope approvability is neutral and requests human review for changes to default behavior. No human approval is recorded. This PR remains unmerged pending explicit authorization.
  • Browser verification covers the shared web/desktop timeline. Mobile is unchanged; its Markdown renderer already supports these image paths. No wire-contract or connection-mode changes. Existing user documentation describes Markdown media and remote path resolution.
  • Removed the old standalone-image assertions. No replacement grouping regression guards were added, as requested.

Note

Medium Risk
Changes default chat timeline layout and per-turn agent instructions across every harness; mistakes could mislead models or alter how tool activity is shown, though behavior is covered by adapter and timeline tests.

Overview
Image views in the chat timeline no longer bypass tool grouping or settled-turn folding. The special-case logic that kept image-preview work rows standalone is removed, so image views behave like other tool reads in groups, folds, and the live activity strip; expanded rows still use the existing image preview renderer.

All six provider adapters now share buildRuntimeInstructions for a consistent <runtime_info> block (T3 Code + harness, optional model/reasoning effort, Markdown image/video embed hint). Codex folds it into developer instructions; Claude appends it to the claude_code system preset; OpenCode sends it on each prompt’s system field. Cursor, Grok, and Antigravity append an extra ACP text block after user content without changing stored thread prompts. Provider docs describe this wiring.

Reviewed by Cursor Bugbot for commit fcffb6e. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Group image views like other tool calls and add shared provider runtime context

  • Removes special-case image-preview handling in the message timeline so image work entries follow the same grouping, folding, and active-work logic as other tool calls.
  • Adds a shared buildRuntimeInstructions formatter in RuntimeInstructions.ts that emits a runtime-info block with harness identity, optional model/reasoning-effort metadata, and Markdown media-embedding instructions using absolute file paths.
  • Integrates the shared runtime context into all six provider adapters (Codex, Antigravity, Claude, Cursor, Grok, OpenCode) so each turn sends current harness/model context alongside the user prompt, without storing it in thread history.
  • Documents the runtime-instruction contract and per-adapter prompt channels in providers.md.
  • Behavioral Change: settled-turn folding can now hide image-preview entries that were previously kept as standalone rows; image-preview entries are also eligible for active-tool grouping in MessagesTimeline.logic.ts.

Macroscope summarized fcffb6e.

Implemented with Claude Fable 5.1 via Claude Code in T3 Code. Preparation resumed and verified with vega-alpha via the Codex harness in T3 Code.

Image views were forced out of tool groups and the settled-turn fold by
leaving a bare path label that split every group for no benefit. Revert
both so an image view is an ordinary read.

Tell Codex and Claude to embed result images as markdown in their final
response instead, which chat already renders inline. ACP providers and
OpenCode expose no system-prompt input, so they are unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.6 KiB 13.7 KiB +194 B (+1.4%) 15.1 KiB
Codex Thread snapshot wire 7.0 KiB 7.0 KiB 0 B (0.0%) 7.3 KiB
Codex Live turn WebSocket wire 6.5 KiB 6.7 KiB +194 B (+2.9%) 7.8 KiB
Codex Live turn WebSocket decoded 57.0 KiB 58.5 KiB +1.5 KiB (+2.6%) 66.4 KiB
Codex Live turn messages 9 10 +1 (+11.1%) 21
Claude Total thread wire 13.8 KiB 13.6 KiB −188 B (−1.3%) 15.1 KiB
Claude Thread snapshot wire 7.0 KiB 7.0 KiB +4 B (+0.1%) 7.3 KiB
Claude Live turn WebSocket wire 6.7 KiB 6.5 KiB −192 B (−2.8%) 7.8 KiB
Claude Live turn WebSocket decoded 59.3 KiB 57.8 KiB −1.5 KiB (−2.5%) 66.4 KiB
Claude Live turn messages 10 9 −1 (−10.0%) 21

Baseline: b34ff8f · PR result: fcffb6e · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.8 KiB
  • Claude decoded thread snapshot: 114.5 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@macroscopeapp

macroscopeapp Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The PR changes default chat grouping/folding behavior and adds runtime directives to every provider's normal prompt path. These cross-cutting default behavior changes warrant human review despite the focused intent and adapter tests.

You can add or adjust custom eligibility rules. Learn more.

juliusmarminge and others added 3 commits September 3, 2026 22:03
… text

OpenCode appends a per-prompt `system` string after its own agent and
provider prompts, so it can carry the same instruction as Codex and Claude.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Comment thread apps/server/src/provider/Layers/ClaudeAdapter.ts Outdated
@github-actions github-actions Bot added size:L 100-499 changed lines (additions + deletions). and removed size:M 30-99 changed lines (additions + deletions). labels Sep 4, 2026
Comment thread apps/server/src/provider/Layers/GrokAdapter.ts Outdated
@juliusmarminge
juliusmarminge merged commit 61a91b6 into main Sep 4, 2026
26 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/fix-image-tool-grouping branch September 4, 2026 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant