Skip to content

Steering messages don't appear in chat history #17

Description

@JoshuaRowePhantom

Summary

Steering messages — user messages injected mid-turn at tool-result boundaries to redirect agent behavior — are silently consumed by the LLM call stack but never recorded in AgentChat's visible chat history. After a turn completes, there is no trace of any steering input the user provided.

Root Causes

There are four interlocking causes, covering both the non-Copilot path (ToolResultSteeringMiddleware) and the Copilot path (CopilotSdkChatClient).

1. AppendUserMessagesToHistory is called before steering can arrive

In AgentChat.RunProcessLoopAsync, history is updated at line 682 — before StartRun is called at line 709. Only messages from the initial drain of the queue are visible at that point. Any messages enqueued by the user while the tool loop is executing are not yet in the queue and are therefore never passed to AppendUserMessagesToHistory.

2. ToolResultSteeringMiddleware injects messages with no callback to AgentChat

ToolResultSteeringMiddleware.InjectQueuedIfToolResult (file: Phantom.Workspaces.Llm.Core\ToolResultSteeringMiddleware.cs) dequeues steering messages from AgentInputQueueManager and appends them to the LLM message list silently. There is no event, callback, or return value that signals AgentChat that these messages were injected. They reach the model but are never written to history.

3. CopilotSdkChatClient bypasses the message framework entirely

For the GitHub Copilot provider, CopilotSdkChatClient.OnQueueChanged (file: Phantom.Workspaces.Llm.Core\CopilotSdkChatClient.cs) handles steering by extracting the raw text from the message and forwarding it via session.SendAsync(Mode="immediate"). The ChatMessage object is never returned to the agent framework — nothing is ever added to history.

4. The design explicitly deferred this work

docs/design/steerable-chat-implementation.md explicitly states "AgentChat does not change its processing loop" (line 18) and contains no mention of history at all. Chat history tracking for steering messages was never implemented.

Recommended Fix

The minimal fix is to add notification callbacks at each injection site and wire them up in AgentChat to call the existing AppendUserMessagesToHistory method.

Part 1 — ToolResultSteeringMiddleware.cs: Add an event + GetService override

// NEW: fires when steering messages are injected into the LLM call.
internal event Action<IReadOnlyList<ChatMessage>>? MessagesInjected;

// NEW: expose this instance so AgentChat can find and subscribe to it.
public object? GetService(Type serviceType, object? serviceKey = null)
{
    if (serviceKey is null && serviceType == typeof(ToolResultSteeringMiddleware))
        return this;
    return this.inner.GetService(serviceType, serviceKey);
}

// In InjectQueuedIfToolResult — collect injected messages and fire the event:
List<ChatMessage>? injected = null;
while (this.queueManager.TryDequeueNextImmediateOrQueued(out var item))
{
    augmented ??= [.. messageList];
    foreach (var message in item.Messages ?? [])
    {
        augmented.Add(message);
        (injected ??= []).Add(message);  // NEW
    }
}
if (injected is not null)
    this.MessagesInjected?.Invoke(injected);  // NEW

Part 2 — CopilotSdkChatClient.cs: Add an event + fire before forwarding

// NEW: fires when a steering message is forwarded to the Copilot session.
internal event Action<ChatMessage>? SteeringMessageForwarded;

// In OnQueueChanged, fire the event before session.SendAsync:
this.SteeringMessageForwarded?.Invoke(message);  // NEW
_ = session.SendAsync(new MessageOptions { Prompt = text, Mode = "immediate" }, CancellationToken.None);

Also add a GetService override to expose CopilotSdkChatClient for discovery.

Part 3 — AgentChat.cs: Subscribe after client is resolved

After this.client = resolvedClient (around line 145 of AgentChat.cs):

if (this.client.GetService(typeof(ToolResultSteeringMiddleware))
        is ToolResultSteeringMiddleware steeringMiddleware)
{
    steeringMiddleware.MessagesInjected += injected =>
        this.AppendUserMessagesToHistory(injected);
}

if (this.client.GetService(typeof(CopilotSdkChatClient))
        is CopilotSdkChatClient copilotClient)
{
    copilotClient.SteeringMessageForwarded += message =>
        this.AppendUserMessagesToHistory([message]);
}

Affected Files

  • Phantom.Workspaces.Llm.Core\AgentChat.cs
  • Phantom.Workspaces.Llm.Core\ToolResultSteeringMiddleware.cs
  • Phantom.Workspaces.Llm.Core\CopilotSdkChatClient.cs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingfailed-verificationBug failed automated verification

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions