[nagilson]

Notable decisions:

• 5s was my selected shutdown time for CI
• 300ms is the bare minimum for NAOT failures to make it with relative consistency tested from my machines - so for failures, it enacts a 300ms shutdown delay. I viewed those as more important to capture.
• 600ms is the bare minimum for Managed code failures to make it with relative consistency tested from my machines, however, 300ms is at the edge of UX acceptable thresholds. If [FEATURE REQ] Allow Custom SubDirectory in Azure OTel Exporter Azure/azure-sdk-for-net#62997 is implemented and the two can share a sub-directory then the AOT side can cover for the managed side. The managed side is just plain slower.
• My testing has limitations and bias. More testing for those with suboptimal networks should be done. We saw previously that the TCP Handshake/DNS represented a substantial amount of the network time to POST and that the instrumented packet is actually rather small, so we'd hope that the numbers somewhat translate.
• Due to this issue with OTel, [bug] .lock file Lease Incorrectly Blocks Subsequent Executables open-telemetry/opentelemetry-dotnet-contrib#5210 runs across 3 minutes from one another actually don't get reported unless you invoke the SDK again within 2d (the blob validity lifetime)
• To reduce exit latency on dotnet commands, the drain is set to never run for non-CI environments outside of failure contexts.
• The after-launch drainer for persisted data launches after 50ms in Azure's new implementation, which may mean short-lived commands such as dotnet --info may still not provide sufficient time to drain telemetry. This is an accepted risk. See EagerDrainDelayMilliseconds in https://raw.githubusercontent.com/Azure/azure-sdk-for-net/Azure.Monitor.OpenTelemetry.Exporter_1.9.0/sdk/monitor/Azure.Monitor.OpenTelemetry.Exporter/src/Internals/TransmitFromStorageHandler.cs.
• Telemetry storage created using the old version formats of the OTel Exporter before the fixes will not be deployed to the production telemetry database.

Background

This PR introduced a new telemetry implementation via a custom open telemetry exporter. Reliable CLI telemetry via a persist-then-drain OpenTelemetry exporter by baronfel · Pull Request #55211 · dotnet/sdk

The reason we did this was recorded within the PR descirption as well as OpenTelemetry with Azure Monitor Exporter seems to not be designed for CLI Applications · Issue #55184 · dotnet/sdk.
You should probably also read [FEATURE REQ] Better support CLI applications in OTel Exporter by enabling manual control over blob storage & POST flush · Issue #60838 · Azure/azure-sdk-for-net.

In short, Azure Monitor Exporter had limitations where it did not typically wait long enough or actually enforce that telemetry storage persisted on short lived applications.

The team however made improvements to address our concerns in [Monitor OpenTelemetry Exporter] Persist telemetry on shutdown for short-lived applications by rajkumar-rangaraj · Pull Request #61818 · Azure/azure-sdk-for-net. And then [Monitor Exporter] Make the shutdown drain budget configurable by rajkumar-rangaraj · Pull Request #62340 · Azure/azure-sdk-for-net.

We coordinated over Outlook and they were great in designing a thoughtful solution for us that met our needs together. For context, using ForceFlush in CI would still not result in correct telemetry delivery but solving that is only possible at the OTel layer, not the exporter layer.

Why

There are many reasons, but really we do not want to maintain a long term fork of their code. This is quite an expensive cost and also introduces several risks such as desynced code or CVE Coordination Disclosures.

Timeframe

The target of this PR is Sept 14th - we must get this out before GA for RC2 to ensure that we have quality data coming in to the .NET platform which informs key decisions. Sept 22nd is the wiggle room but we want validation before RC2 goes out.

Tests

Different shutdown forceFlush budgets

A 'retained blob' means it didn't get sent.

Budget	AOT Failure	Managed Failure
200 ms	Blob retained—all 4 events	Blob retained—all 4 events
300 ms	No blobs remain	Blob retained—all 4 events
400 ms	No blobs remain	Blob retained—all 4 events
450 ms	No blobs remain	Blob retained—all 4 events
500 ms	No blobs remain	Blob retained—all 4 events
600 ms	No blobs remain	No blobs remain
800 ms	No blobs remain	No blobs remain
1 second	No blobs remain	No blobs remain
2 seconds	No blobs remain	No blobs remain

Showing whether waiting 3m10s helped resend failed blobs

Error Shutdown Budget	Failure Payload Removed Immediately	Entire Storage Empty Immediately After Failure	Failure Payload Gone After Success
500 ms	1/2	1/2	2/2
600 ms	2/2	1/2	2/2
800 ms	2/2	0/2	2/2
1 second	2/2	1/2	2/2
2 seconds	2/2	2/2	2/2

AOT vs Managed Failure Telemetry

Budget	AOT shutdown elapsed	AOT remaining blobs	Managed shutdown elapsed	Managed remaining blobs
500 ms	376.51 ms	0	504.40 ms	1, all 4 expected events retained
600 ms	324.32 ms	0	618.24 ms	0
800 ms	376.87 ms	0	670.80 ms	0
1000 ms	393.11 ms	0	673.91 ms	0
2000 ms	384.74 ms	0	750.98 ms	0

Without extra shutdown budget:

Command	CI Observed	Local Observed
dotnet --version	3/3 — 100%	0/3 — 0%
dotnet --info	3/3 — 100%	0/3 — 0%
dotnet tool search dotnet	3/3 — 100%	0/3 — 0%
dotnet --list — intentional invalid command	3/3 — 100%	0/3 — 0%
Total	12/12 — 100%	0/12 — 0%

[azure-pipelines]

Azure Pipelines:
Successfully started running 2 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[nagilson]

Big sad, I based this off main but main is net 12 now

[Copilot]

🟡 Changes recommended

Two moderate review findings remain unresolved: CI-path coverage is missing, and live endpoint tests default to a shared resource.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Replaces the custom telemetry persistence/export pipeline with Azure Monitor OpenTelemetry Exporter 1.9.0 for managed and Native AOT CLI paths.

Changes:

• Removes obsolete custom telemetry implementation and tests.
• Configures exporter persistence and shutdown behavior.
• Updates dependencies, lifecycle coverage, and documentation.

File summaries

File	Summary
test/Microsoft.DotNet.Cli.Telemetry.Tests/RealEndpointTelemetryE2ETests.cs	Adds live endpoint coverage. Moderate (3 votes): require explicit opt-in instead of the default connection string.
test/Microsoft.DotNet.Cli.Telemetry.Tests/PersistentStorageTelemetryUploaderTests.cs	Removes obsolete uploader tests.
test/Microsoft.DotNet.Cli.Telemetry.Tests/Microsoft.DotNet.Cli.Telemetry.Tests.csproj	Updates telemetry test dependencies.
test/Microsoft.DotNet.Cli.Telemetry.Tests/HttpTelemetryUploadTransportTests.cs	Removes obsolete HTTP transport tests.
test/Microsoft.DotNet.Cli.Telemetry.Tests/FileSystemTelemetryBlobStorageTests.cs	Removes obsolete storage tests.
test/Microsoft.DotNet.Cli.Telemetry.Tests/BreezePartialContentTests.cs	Removes obsolete Breeze tests.
test/Microsoft.DotNet.Cli.Telemetry.Tests/AzureMonitorTelemetrySerializerTests.cs	Removes obsolete serializer tests.
test/Microsoft.DotNet.Cli.Telemetry.Tests/AzureMonitorLogSerializerTests.cs	Removes obsolete log serializer tests.
test/Microsoft.DotNet.Cli.Telemetry.Tests/AzureMonitorExporterDifferentialTests.cs	Adds exporter differential coverage.
test/Microsoft.DotNet.Cli.Telemetry.Tests/AzureMonitorConnectionStringTests.cs	Removes obsolete connection-string tests.
test/Microsoft.DotNet.Cli.Telemetry.Tests/AzureExporterTestSupport.cs	Adds isolated Azure exporter test support.
test/Microsoft.DotNet.Cli.Telemetry.Tests/AzureExporterLifecycleTests.cs	Tests exporter persistence and shutdown behavior.
test/dotnet.Tests/TelemetryTests/TelemetryClientTests.cs	Updates CLI telemetry tests.
test/dotnet.Tests/TelemetryTests/PersistentStorageTelemetryE2ETests.cs	Tests managed and AOT telemetry persistence.
test/dotnet-aot.Tests/dotnet-aot.Tests.csproj	Removes obsolete telemetry references.
src/Cli/Microsoft.DotNet.Cli.Telemetry/README.md	Updates telemetry component documentation.
src/Cli/Microsoft.DotNet.Cli.Telemetry/PersistentStorageTracerProviderBuilderExtensions.cs	Removes custom tracer-provider extensions.
src/Cli/Microsoft.DotNet.Cli.Telemetry/PersistentStorageTraceExporter.cs	Removes the custom trace exporter.
src/Cli/Microsoft.DotNet.Cli.Telemetry/PersistentStorageTelemetryOptions.cs	Removes custom telemetry options.
src/Cli/Microsoft.DotNet.Cli.Telemetry/PersistentStorageLoggerOptionsExtensions.cs	Removes custom logger extensions.
src/Cli/Microsoft.DotNet.Cli.Telemetry/PersistentStorageLogExporter.cs	Removes the custom log exporter.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Microsoft.DotNet.Cli.Telemetry.csproj	Updates the telemetry project.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/TelemetryUploadResult.cs	Removes custom upload result types.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/TelemetryResourceContextFactory.cs	Removes custom resource-context creation.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/TelemetryResourceContext.cs	Removes custom resource context.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/PersistentStorageTelemetryUploader.cs	Removes the custom telemetry uploader.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/ITelemetryUploadTransport.cs	Removes the custom upload transport abstraction.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/ITelemetryBlobStorage.cs	Removes the custom blob-storage abstraction.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/HttpTelemetryUploadTransport.cs	Removes the custom HTTP transport.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/FileSystemTelemetryBlobStorage.cs	Removes the custom filesystem storage.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/BreezeWriter.cs	Removes the custom Breeze writer.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/BreezeSchema.cs	Removes the custom Breeze schema.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/BreezePartialContent.cs	Removes custom partial-content handling.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/BreezeEnvelope.cs	Removes the custom Breeze envelope.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/AzureMonitorTelemetrySerializer.cs	Removes the custom telemetry serializer.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/AzureMonitorLogSerializer.cs	Removes the custom log serializer.
src/Cli/Microsoft.DotNet.Cli.Telemetry/Implementation/AzureMonitorConnectionString.cs	Removes custom connection-string handling.
src/Cli/dotnet/Telemetry/TelemetryClient.cs	Configures Azure exporter lifecycle behavior. Moderate (1 vote): add real CI-path subprocess coverage.
src/Cli/dotnet/dotnet.csproj	Updates CLI telemetry dependencies.
src/Cli/dotnet-aot/dotnet-aot.csproj	Adds Azure exporter support for AOT.
sdk.slnx	Removes the deleted telemetry project.
eng/Versions.props	Updates Azure Monitor and OpenTelemetry versions.
documentation/project-docs/telemetry.md	Documents telemetry delivery and shutdown behavior.

Review details

Suppressed comments (1)

src/Cli/dotnet/Telemetry/TelemetryClient.cs:135

• The CI-specific exporter switches added here are only reproduced in ExporterSettingsScope; no test instantiates the production TelemetryClient with CI detection enabled. The CLI E2E test explicitly removes CI markers and the host-matrix tests disable Azure export, so a typo in these switch names/values or in the CI FlushProviders path can pass all tests. Add an isolated subprocess test that exercises the real CLI CI path and verifies it waits for an upload and retains retryable data.

            AppContext.SetSwitch("Azure.Monitor.OpenTelemetry.Exporter.DisablePersistOnShutdown", s_isCIEnvironment);
            AppContext.SetData("Azure.Monitor.OpenTelemetry.Exporter.ShutdownDrainBudgetMilliseconds", 0); //  background upload will occur next time - reduce exit latency. This only impacts storage persistThenDrain, so not CI.
            AppContext.SetSwitch("Azure.Monitor.OpenTelemetry.Exporter.PersistOnForceFlush", !s_isCIEnvironment);
            s_tracerProviderBuilder.AddAzureMonitorTraceExporter(options =>

• Files reviewed: 43/43 changed files
• Comments generated: 1
• Review effort level: Lite

[azure-pipelines]

Azure Pipelines:
Successfully started running 2 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[nagilson]

The failing test is a dotnet test flake, so merging this.