Compress tool output to reduce token usage

[meganrogge]

Fixes microsoft/vscode#315376

Adds a post-processing compression layer for tool results, inspired by ztk. When the model invokes a tool, we now have a chance to filter the result text before it reaches the model — same information, fewer tokens.

What

• New IToolResultCompressor service in src/extension/tools/common/toolResultCompressor.ts with a per-tool filter registry. Filters are pure functions over text parts. The service preserves stderr and exit codes, skips outputs under 80 bytes, and drops any filter that throws (ztk's "never make it worse" rule).
• Three initial filters for run_in_terminal in src/extension/tools/node/compressors/terminalOutputCompressor.ts:
  • git diff / git show — collapses unchanged context, drops index/similarity index headers, omits lockfile and snapshot diff bodies, preserves +/-/@@ lines verbatim.
  • ls -l / ls -la — keeps just file names (with / for dirs), drops perm/owner/size columns and total N.
  • npm install / yarn / pnpm install — drops progress bars, npm warn deprecated, funding/audit nag lines.
• Wired into ToolsService.invokeTool as a single post-process step right after vscode.lm.invokeTool resolves. Works for any tool — including the core run_in_terminal — without core changes.
• New setting github.copilot.chat.tools.compressOutput.enabled, off by default.
• New telemetry event toolResultCompressed reporting { toolName, filters, beforeBytes, afterBytes } so we can quantify savings.

Why

The terminal tool routinely dumps tens of thousands of raw tokens (a single git diff HEAD~5 can be 90KB+). Most of that is metadata, unchanged context, lockfile churn, or progress noise. Compressing this client-side before it hits the model is a cheap, predictable token win that doesn't change tool semantics.

Tests

12 unit tests in src/extension/tools/node/test/terminalOutputCompressor.spec.ts covering command parsing, each filter's match logic, and the actual compression behavior. All passing.

Scope notes

• Lives entirely in copilot-chat — no core API changes.
• Default off; opt-in via setting (and later, experiment).
• Easy to extend: add a new IToolResultFilter and register it. Future filters: test-runner output, read_file skeleton mode, grep_search per-file match cap, session-level dedup cache.

[Copilot]

Pull request overview

This PR introduces an opt-in post-processing layer that compresses tool results (initially focused on run_in_terminal) before they are forwarded to the language model, aiming to reduce token usage while preserving tool semantics where possible.

Changes:

• Added a new IToolResultCompressor service with a filter registry and telemetry reporting of compression savings.
• Implemented initial terminal output compressors for git diff/show, ls -l/-la, and npm/yarn/pnpm install, plus unit tests for the terminal filters.
• Wired compression into ToolsService.invokeTool and exposed an (off-by-default) setting to enable it.

Show a summary per file

File	Description
src/platform/configuration/common/configurationService.ts	Adds a new config key for enabling tool result compression.
src/extension/tools/vscode-node/toolsService.ts	Integrates compression into tool invocation and registers terminal compressors.
src/extension/tools/common/toolResultCompressor.ts	Introduces the compressor service, filter interfaces, and compression + telemetry logic.
src/extension/tools/node/compressors/terminalOutputCompressor.ts	Adds initial compression filters for common terminal commands.
src/extension/tools/node/test/terminalOutputCompressor.spec.ts	Adds unit tests covering terminal filter matching and transformations.
src/extension/extension/vscode-node/services.ts	Registers the new compressor service in the node DI container.
package.nls.json	Adds localized setting description text.
package.json	Contributes the new setting to VS Code configuration.

Copilot's findings

• Files reviewed: 8/8 changed files
• Comments generated: 5

[unveroleone]

Nice work! Maybe @roblourens or @meganrogge can look at my PR?

[roblourens]

This is cool, some thoughts:
Have you considered doing this on the other end- modifying the git diff command to produce fewer context lines, instead of parsing the output? Same with ls.

If you remove context lines, then the line number reference is wrong, should we patch that so the agent doesn't get confused about which lines it's looking at?

How about telling it that the output is truncated, and giving it a way to read the non-truncated output? What if it actually needed to see the stuff that was removed, or what if there's a bug?

[meganrogge]

Thanks! All three are fair - quick thoughts on each, but TL;DR I'd like to land this behind the experiment first and see whether it actually moves the needle before investing in the deeper versions of any of these.

Re modifying the command instead of parsing output: considered it. Works for git diff -U1 and npm install --no-progress, but ls -l has no flag to drop the perms/owner/size columns, and rewriting arbitrary user commands means parsing pipes/redirects/aliases/already-set flags. Output filtering generalizes to future targets (cargo build, pytest, docker build) without per-tool flag specs, so I'll keep it as the backbone and consider command-rewrite as an additive optimization later.

Re line numbers off after dropping context - good catch. The hunk header counts (@@ -10,7 +10,7 @@) won't match the emitted body, so it's no longer a valid unified diff. I'll rewrite the counts to match in this PR.

Re tell it it's truncated + give it a way to read the original: gitDiffFilter already emits ... N unchanged context lines omitted ... markers (you can see in the screenshot above), but ls/npm filters drop silently and none of them announce which filter ran. I'll add a per-filter header at minimum.

The "let it fetch the original" piece is more involved — would need run_in_terminal to always return a terminal id and get_terminal_output to serve completed-session output (it only does for timed out or async currently) — so I'd rather wait on the exp results before going down that path. If compression turns out to be worth keeping, we can consider it.

[meganrogge]

[meganrogge]

Apparently this is the old repo. IDK where the new one is.

[meganrogge]

Needs to be moved to core