Skip to content

agent: Skip redundant thread saves while a response streams - #65244

Merged
smitbarmase merged 12 commits into
mainfrom
sb/skip-redundant-thread-saves
Oct 7, 2026
Merged

smitbarmase merged 12 commits into
mainfrom
sb/skip-redundant-thread-saves

Conversation

@smitbarmase

Copy link
Copy Markdown
Member

Closes #64823

While the agent replied, every streamed chunk saved the whole thread: serializing it to JSON, compressing it, and rewriting the SQLite row. The streamed text isn't saved until its message completes, so nearly all of these writes were identical.

Now the thread is not saved while a message is streaming, unless a message has completed or a setting, the title, or the draft prompt changed since the last save.

Measured on a 16.6 MB thread (0.76 MB compressed) while the agent replied to one prompt for ~40s:

Before After
Full-thread saves during the reply 836 (~21/s) 2
Zed's memory (Activity Monitor) while streaming 1.34 GB → ~2.2 GB stays flat
CPU while streaming ~75% of a core ~35% of a core

Before:

Initial:
CleanShot 2026-10-06 at 8 36 34 PM@2x

Streaming:
CleanShot 2026-10-06 at 8 37 20 PM@2x

After:

Initial:
CleanShot 2026-10-06 at 8 41 22 PM@2x

Streaming:
CleanShot 2026-10-06 at 8 38 20 PM@2x

Release Notes:

  • Improved memory and CPU usage while the agent responds in long threads.

@smitbarmase
smitbarmase requested a review from osiewicz October 6, 2026 15:14
@cla-bot cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Oct 6, 2026
@zed-community-bot zed-community-bot Bot added the staff Pull requests authored by a current member of Zed staff label Oct 6, 2026
Comment thread crates/agent/src/agent.rs Resolved
Comment thread crates/agent/src/agent.rs Resolved

@osiewicz osiewicz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM sans the submitted comments.

@smitbarmase
smitbarmase requested a review from osiewicz October 7, 2026 10:24
@smitbarmase
smitbarmase added this pull request to the merge queue Oct 7, 2026
Merged via the queue into main with commit 98e1682 Oct 7, 2026
37 checks passed
@smitbarmase
smitbarmase deleted the sb/skip-redundant-thread-saves branch October 7, 2026 10:55
@gianpaj

gianpaj commented Oct 18, 2026

Copy link
Copy Markdown

Great work!
One question. What happens now if the connection drops or the stream stops for any other reason? Are we saving the thread? Imagine I'm on the train; I lose Wi-Fi, and the thread stops. I close the laptop, and the battery runs out. Will the last message in the thread, or a partial message, be saved?

@smitbarmase

Copy link
Copy Markdown
Member Author

Thanks! This PR only changes how often we save, not what we save. That part hasn't changed, so finished messages get saved right away, and the one that's still streaming lives in memory until it's done.

So for your train case, it depends on what happens to that last message:

  • If the wifi drop makes the request fail, Zed keeps what it got so far and saves it then, so closing the laptop after that is fine.
  • If Zed goes away while it's still streaming (battery dies, crash, or even just quitting normally), that partial message is gone, along with any thinking or tool calls in it.

That second one is a known gap, tracked in #65464.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The user has signed the Contributor License Agreement staff Pull requests authored by a current member of Zed staff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The whole thread is re-serialized and written to SQLite several times per second while a response streams

3 participants