fix(otel): select the supported entry of OTEL_*_EXPORTER lists by MilanGarnier · Pull Request #21596 · DataDog/dd-trace-py

MilanGarnier · 2026-11-09T12:54:34Z

Description

The OpenTelemetry spec says OTEL_TRACES_EXPORTER, OTEL_METRICS_EXPORTER and OTEL_LOGS_EXPORTER can potentially be comma-separated lists of exporters. Upstream python OTel supports it
ddtrace only compared the raw value against otlp and none, so a list such as otlp,zipkin was treated as unsupported and ignored.

ddtrace still exports each signal to a single exporter. This aimes to have a consistent fall back to the supported entry (otlp) instead of discarding the whole value:

otlp anywhere in the list selects OTLP export, even when combined with none.
Otherwise none disables the signal.
Entries are trimmed and matched ignoring case, and blank entries are skipped.
Other entries are ignored with a warning, and counted in the otel.env.unsupported telemetry metric when the list still selects an exporter.
An empty value, or a list without otlp or none, behaves like an unset variable (a list with only unsupported entries still gets the existing "not supported" warning).

Parsing lives in select_exporter in _otel_remapper.py and is shared by the traces OTLP check in _agent.py, the config remapping and the metrics validation in the telemetry module. OTEL_METRICS_EXPORTER is now validated at most once in Config, so its warnings and telemetry are not duplicated.

Testing

Added tests in tests/opentelemetry/test_config.py, tests/tracer/test_agent.py and tests/telemetry/test_writer.py covering lists, blank values and the ignored-entry telemetry for each signal.

scripts/lint checks passes. I could not run the test suites locally: the Docker test runner ran out of disk space while building the environments, so the new tests have not been run yet and rely on CI.

Risks

Low. Values that were previously ignored as unsupported (lists containing otlp or none) are now honored. Single-value behavior is unchanged.

Additional Notes

The commits are split as tests, list parsing, telemetry, release note.

cit-pr-commenter-54b7da · 2026-11-09T12:56:08Z

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/internal/settings/_agent.py                                     @DataDog/apm-core-python
ddtrace/internal/settings/_config.py                                    @DataDog/python-guild @DataDog/apm-sdk-capabilities
ddtrace/internal/settings/_otel_remapper.py                             @DataDog/apm-sdk-capabilities
ddtrace/internal/telemetry/__init__.py                                  @DataDog/apm-python
ddtrace/opentelemetry/__init__.py                                       @DataDog/apm-sdk-capabilities
releasenotes/notes/otel-traces-exporter-list-e3334060c1d529f4.yaml      @DataDog/apm-python
tests/opentelemetry/test_config.py                                      @DataDog/apm-sdk-capabilities
tests/telemetry/test_writer.py                                          @DataDog/apm-core-python @DataDog/apm-python
tests/tracer/test_agent.py                                              @DataDog/apm-sdk-capabilities

cit-pr-commenter-54b7da · 2026-11-09T12:56:16Z

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 83 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 83 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=120)
ddtrace.llmobs._integrations.base -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)
ddtrace.llmobs._integrations.litellm -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)
ddtrace.llmobs._integrations.langchain -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)
ddtrace.llmobs._integrations.bedrock -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:
uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

MilanGarnier · 2026-11-09T13:04:33Z

@codex review

chatgpt-codex-connector · 2026-11-09T13:04:53Z

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-09T13:57:34.012642Z	`9b466e4`	New commits
🔒 Security Review	✅ Completed 2026-11-09T14:00:54.098089Z	`9b466e4`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector · 2026-11-09T13:08:00Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: e193480f95

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e193480f95

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadog-datadog-prod-us1 · 2026-11-09T13:11:26Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 9b466e4 | Docs | View more details | Give us feedback!}

pr-commenter · 2026-11-09T13:39:37Z

Benchmarks

Benchmark execution time: 2026-11-09 14:19:04

Comparing candidate commit 9b466e4 in PR branch milan.garnier/otel-traces-exporter-list with baseline commit d078aad in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 613 metrics, 9 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

unstable execution_time [-701.264ns; +730.104ns] or [-6.539%; +6.808%]

scenario:coreapiscenario-core_dispatch_50_listeners

unstable execution_time [-1880.456ns; +1816.285ns] or [-9.366%; +9.046%]

scenario:coreapiscenario-core_dispatch_exception_listeners

unstable execution_time [-1950.971ns; +1571.050ns] or [-9.977%; +8.034%]

scenario:coreapiscenario-core_dispatch_listeners

unstable execution_time [-354.860ns; +361.873ns] or [-8.166%; +8.327%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

unstable execution_time [-219.550ns; +215.974ns] or [-8.045%; +7.914%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

unstable execution_time [-90.777ns; +87.969ns] or [-6.090%; +5.901%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

unstable execution_time [-4719.720ns; +4714.685ns] or [-9.272%; +9.262%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

unstable execution_time [-986.235ns; +952.201ns] or [-9.140%; +8.824%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

unstable execution_time [-50.061µs; +60.711µs] or [-8.671%; +10.516%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

scenario:packagespackageforrootmodulemapping-cache_off

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 16edd1907a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadog-datadog-prod-us1

More details

Exporter-list selection remains consistent between signal configuration and trace transport. The runtime-metrics guard also preserves disabling via OTEL_METRICS_EXPORTER=none.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 16edd19}

emmettbutler

lgtm

MilanGarnier added 2 commits November 9, 2026 13:38

test(otel): cover exporter lists in OTEL_*_EXPORTER

9afb319

feat(otel): select the supported entry of OTEL_*_EXPORTER lists

2b50d04

chatgpt-codex-connector Bot reviewed Nov 9, 2026

View reviewed changes

Comment thread ddtrace/internal/settings/_config.py Resolved

MilanGarnier force-pushed the milan.garnier/otel-traces-exporter-list branch from e193480 to 909a7ae Compare November 9, 2026 13:11

MilanGarnier added 2 commits November 9, 2026 14:13

feat(telemetry): report ignored OTEL_*_EXPORTER list entries

fe4ef81

docs: add release note for OTEL_*_EXPORTER lists

16edd19

MilanGarnier force-pushed the milan.garnier/otel-traces-exporter-list branch from 909a7ae to 16edd19 Compare November 9, 2026 13:14

MilanGarnier marked this pull request as ready for review November 9, 2026 13:35

MilanGarnier requested review from a team as code owners November 9, 2026 13:35

MilanGarnier requested review from juanjux and mabdinur and removed request for a team November 9, 2026 13:35

chatgpt-codex-connector Bot reviewed Nov 9, 2026

View reviewed changes

Comment thread ddtrace/internal/telemetry/__init__.py Resolved

Comment thread ddtrace/internal/telemetry/__init__.py Resolved

datadog-datadog-prod-us1 Bot reviewed Nov 9, 2026

View reviewed changes

MilanGarnier added 2 commits November 9, 2026 14:48

fix(telemetry): treat blank OTEL_*_EXPORTER lists as unset in telemetry

37fefc5

Merge branch 'main' into milan.garnier/otel-traces-exporter-list

9b466e4

emmettbutler approved these changes Nov 9, 2026

View reviewed changes

Conversation

MilanGarnier commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Risks

Additional Notes

Uh oh!

cit-pr-commenter-54b7da Bot commented Nov 9, 2026

Codeowners resolved as

Uh oh!

cit-pr-commenter-54b7da Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency direction analysis

⚠️ Existing dependency direction violations

Uh oh!

MilanGarnier commented Nov 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Nov 9, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

datadog-datadog-prod-us1 Bot commented Nov 9, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Unstable benchmarks

scenario:coreapiscenario-context_with_data_listeners

scenario:coreapiscenario-core_dispatch_50_listeners

scenario:coreapiscenario-core_dispatch_exception_listeners

scenario:coreapiscenario-core_dispatch_listeners

scenario:coreapiscenario-core_dispatch_no_args_listeners

scenario:coreapiscenario-core_dispatch_with_results_1_listener

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

scenario:coreapiscenario-core_dispatch_with_results_listeners

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

Known flaky benchmarks

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

datadog-datadog-prod-us1 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

emmettbutler left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MilanGarnier commented Nov 9, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Nov 9, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Nov 9, 2026 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Nov 9, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

pr-commenter Bot commented Nov 9, 2026 •

edited

Loading