Skip to content

.NET: fix: preserve agent continuation token when wrapped with UseOpenTelemetry - #8265

Open
Atharva Vichare (atty57) wants to merge 2 commits into
microsoft:mainfrom
atty57:atty57-issue-8243
Open

.NET: fix: preserve agent continuation token when wrapped with UseOpenTelemetry#8265
Atharva Vichare (atty57) wants to merge 2 commits into
microsoft:mainfrom
atty57:atty57-issue-8243

Conversation

@atty57

Copy link
Copy Markdown
Contributor

Motivation & Context

Wrapping an agent with UseOpenTelemetry breaks background responses. The continuation token returned from a background RunAsync is the underlying provider's raw token rather than the agent's own ChatClientAgentContinuationToken, so an opaque ToBytes/FromBytes save-and-restore followed by a resume throws:

System.ArgumentException: Failed to create ChatClientAgentContinuationToken from provided token because it is not of the correct type. (Parameter 'token')
   at Microsoft.Agents.AI.ChatClientAgentContinuationToken.FromToken(ResponseContinuationToken token)
   at Microsoft.Agents.AI.ChatClientAgent.CreateConfiguredChatOptions(AgentRunOptions runOptions)

The same flow works when OpenTelemetry is not in the pipeline. This blocks the scenario where background responses are started by one process and polled or resumed by another, since that requires persisting a serialized session and token while still emitting telemetry for logging and traceability.

OpenTelemetryAgent delegates to the inner agent through OpenTelemetryChatClient and a ForwardingChatClient, so every response makes an AgentResponse -> ChatResponse -> AgentResponse round trip. AgentResponseExtensions.AsChatResponse returns the inner agent's own ChatResponse directly whenever that is the response's raw representation, which is exactly the case for ChatClientAgent. The wrapped continuation token and AgentId live only on the AgentResponse, so both were discarded on the way back out and RunCoreAsync rebuilt a response from the provider's ChatResponse, carrying the provider's raw token. RunStreamingAsync lost them the same way through AsChatResponseUpdate.

Description & Review Guide

  • What are the major changes?

    • ForwardingChatClient no longer uses AsChatResponse/AsChatResponseUpdate. It now builds the ChatResponse/ChatResponseUpdate that is handed to OpenTelemetryChatClient with the inner agent's own response as the RawRepresentation. RunCoreAsync and RunCoreStreamingAsync already looked for an AgentResponse/AgentResponseUpdate there, so they now return the inner agent's instance unchanged instead of reconstructing a lossy copy.
    • ConversationId is copied across from the inner ChatResponse/ChatResponseUpdate so the chat span keeps emitting gen_ai.conversation.id.
    • Three tests in OpenTelemetryAgentTests: continuation token preservation for RunAsync and RunStreamingAsync, and the reported scenario end to end - serialize the token with ToBytes, restore it with FromBytes, resume, and assert the chat client receives its own token back. All three fail without the source change.
  • What is the impact of these changes?

    • Background responses, AgentId, and any other agent-level response state now survive a UseOpenTelemetry wrapper. Callers no longer need workarounds to reconstruct the agent token.
    • No public API change, and no behavior change for callers that do not use background responses. The fix is confined to the private ForwardingChatClient; AsChatResponse/AsChatResponseUpdate keep their documented short-circuit behavior and are untouched.
  • What do you want reviewers to focus on?

    • Whether copying ConversationId from the raw representation is the right way to keep the chat span intact, or whether the span should take it from the request options instead.
    • Whether other DelegatingAIAgent implementations that perform the same round trip should share a helper rather than keeping this private to OpenTelemetryAgent.

Related Issue

Fixes #8243

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

OpenTelemetryAgent routes runs through OpenTelemetryChatClient, so every
response makes an AgentResponse -> ChatResponse -> AgentResponse round trip.
AsChatResponse/AsChatResponseUpdate return the inner agent's own
ChatResponse when that is the response's raw representation, so the round
trip dropped agent-level state: for a ChatClientAgent the caller received the
provider's raw continuation token instead of the ChatClientAgentContinuationToken,
and resuming with it threw. AgentId was lost the same way.

Wrap the inner agent's response in a ChatResponse that keeps the response
itself as the raw representation, so RunCoreAsync/RunCoreStreamingAsync hand
back the original instance unchanged.

Fixes microsoft#8243

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Both new wrappers omit ModelId, causing response-model telemetry loss for regular and streaming calls.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Fixes #8243 by preserving agent-level continuation state through OpenTelemetry wrappers.

Changes:

  • Preserves original agent responses and streaming updates.
  • Adds continuation-token serialization/resumption tests.
  • Retains conversation IDs for telemetry.
File summaries
File Description
dotnet/src/Microsoft.Agents.AI/OpenTelemetryAgent.cs Wraps responses while retaining agent state.
dotnet/tests/Microsoft.Agents.AI.UnitTests/OpenTelemetryAgentTests.cs Tests token preservation and resumption.
Review details

Suppressed comments (1)

dotnet/src/Microsoft.Agents.AI/OpenTelemetryAgent.cs:360

  • The streaming wrapper similarly omits ChatResponseUpdate.ModelId. OpenTelemetryChatClient consumes the model ID from updates for chunk metrics and from the assembled response for gen_ai.response.model, so streamed calls lose that telemetry after this change. Preserve the raw update's model ID here as well.
                ConversationId = (update.RawRepresentation as ChatResponseUpdate)?.ConversationId,
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread dotnet/src/Microsoft.Agents.AI/OpenTelemetryAgent.cs
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Streaming conversion omits ModelId, causing response-model telemetry loss.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

dotnet/src/Microsoft.Agents.AI/OpenTelemetryAgent.cs:361

  • ModelId is omitted from the streaming wrapper. Providers such as FoundryChatClient set ChatResponseUpdate.ModelId, and OpenTelemetryChatClient reads it to emit the response-model span and metric tags. Because this new wrapper is what telemetry processes, streaming runs now lose that metadata while the non-streaming helper preserves it. Copy the raw update's ModelId as well.
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Usage: [Issues, PRs], Target: .Net

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: [Bug]: UseOpenTelemetry breaks background continuation tokens

2 participants