fix(otel): default OTLP metrics and logs to HTTP/protobuf by Mascarpone · Pull Request #21593 · DataDog/dd-trace-py

Mascarpone · 2026-11-09T09:30:45Z

Description

Default OTLP metrics and logs to http/protobuf, matching traces, with local endpoints on port 4318. Explicit protocol settings retain their precedence.

Update system-tests to include SDK default-protocol discovery.

Testing

OpenTelemetry suites, older-exporter compatibility tests, system-tests, and lint pass. Startup benchmark timings are comparable.

Risks

gRPC collectors now require OTEL_EXPORTER_OTLP_PROTOCOL=grpc or a signal-specific protocol override. Documentation and an upgrade note cover the migration.

Additional Notes

Jira: APMAPI-2711

Use HTTP/protobuf as the shared OTLP default, matching traces, while preserving explicit global and signal-specific protocols. Update configuration metadata, export coverage, and migration documentation. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>

Mascarpone · 2026-11-09T09:31:13Z

@codex review

chatgpt-codex-connector · 2026-11-09T10:14:46Z

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-09T13:33:26.233569Z	`4764890`	Draft marked ready
🔒 Security Review	✅ Completed 2026-11-09T13:35:18.954552Z	`4764890`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector · 2026-11-09T10:17:03Z

Codex Review: Didn't find any major issues. 👍

Reviewed commit: de5265c0c8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-11-09T10:17:39Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: de5265c0c8

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

cit-pr-commenter-54b7da · 2026-11-09T10:26:59Z

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

.github/workflows/system-tests.yml                                      @DataDog/python-guild @DataDog/apm-core-python
.gitlab-ci.yml                                                          @DataDog/python-guild @DataDog/apm-core-python
ddtrace/internal/settings/_opentelemetry.py                             @DataDog/apm-sdk-capabilities
ddtrace/opentelemetry/__init__.py                                       @DataDog/apm-sdk-capabilities
releasenotes/notes/otlp-default-http-protobuf-1dd8653f7d63b745.yaml     @DataDog/apm-python
supported-configurations.json                                           @DataDog/apm-python
tests/opentelemetry/test_config.py                                      @DataDog/apm-sdk-capabilities
tests/opentelemetry/test_logs.py                                        @DataDog/apm-sdk-capabilities
tests/opentelemetry/test_metrics.py                                     @DataDog/apm-sdk-capabilities

cit-pr-commenter-54b7da · 2026-11-09T10:27:21Z

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 83 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 83 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=120)
ddtrace.llmobs._integrations.mcp -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)
ddtrace.llmobs._integrations.openai_agents -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)
ddtrace.llmobs._integrations.llama_index -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)
ddtrace.llmobs._integrations.crewai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=118)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:
uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

datadog-official · 2026-11-09T10:38:06Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 4764890 | Docs | View more details | Give us feedback!}

pr-commenter · 2026-11-09T10:49:29Z

Benchmarks

Benchmark execution time: 2026-11-09 11:28:50

Comparing candidate commit 4764890 in PR branch codex/apmapi-2711-otlp-protobuf with baseline commit e44a5f7 in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 1 performance regressions! Performance is the same for 620 metrics, 9 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:errortrackingmonitoring-production_handler

🟥 max_rss_usage [+23.928MB; +24.005MB] or [+108.664%; +109.014%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

unstable execution_time [-716.680ns; +685.728ns] or [-6.662%; +6.374%]

scenario:coreapiscenario-core_dispatch_50_listeners

unstable execution_time [-1772.685ns; +1910.527ns] or [-8.838%; +9.525%]

scenario:coreapiscenario-core_dispatch_exception_listeners

unstable execution_time [-2028.770ns; +1491.554ns] or [-10.353%; +7.612%]

scenario:coreapiscenario-core_dispatch_listeners

unstable execution_time [-393.205ns; +326.238ns] or [-9.033%; +7.495%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

unstable execution_time [-219.819ns; +216.202ns] or [-8.044%; +7.912%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

unstable execution_time [-102.643ns; +80.481ns] or [-6.825%; +5.352%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

unstable execution_time [-4924.132ns; +4562.551ns] or [-9.625%; +8.918%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

unstable execution_time [-1079.212ns; +859.293ns] or [-9.956%; +7.927%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

unstable execution_time [-60.021µs; +50.593µs] or [-10.279%; +8.664%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

scenario:packagespackageforrootmodulemapping-cache_off

Use the first system-tests revision that observes each SDK's default OTLP transport while retaining explicit-protocol and payload checks. Keep GitHub and GitLab test revisions aligned. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>

Mascarpone · 2026-11-09T11:08:04Z

@codex review

chatgpt-codex-connector · 2026-11-09T11:10:57Z

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 47648909b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-11-09T11:12:25Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 47648909b9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

datadog-official

More details

The HTTP/protobuf default uses existing protocol-aware exporter paths and preserves explicit global and signal-specific protocol precedence. gRPC-only collectors require an explicit protocol setting, as documented in the upgrade note.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 4764890}

Mascarpone marked this pull request as ready for review November 9, 2026 13:31

Mascarpone requested review from a team as code owners November 9, 2026 13:31

Mascarpone requested review from brettlangdon, mabdinur, vpellan and wantsui and removed request for a team November 9, 2026 13:31

datadog-official Bot reviewed Nov 9, 2026

View reviewed changes

Mascarpone mentioned this pull request Nov 9, 2026

fix(otel): recover from invalid OTLP protocol and compression values #21599

Draft

mabdinur approved these changes Nov 9, 2026

View reviewed changes

Conversation

Mascarpone commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Risks

Additional Notes

Uh oh!

Mascarpone commented Nov 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Nov 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 9, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

cit-pr-commenter-54b7da Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codeowners resolved as

Uh oh!

cit-pr-commenter-54b7da Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency direction analysis

⚠️ Existing dependency direction violations

Uh oh!

datadog-official Bot commented Nov 9, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

scenario:errortrackingmonitoring-production_handler

Unstable benchmarks

scenario:coreapiscenario-context_with_data_listeners

scenario:coreapiscenario-core_dispatch_50_listeners

scenario:coreapiscenario-core_dispatch_exception_listeners

scenario:coreapiscenario-core_dispatch_listeners

scenario:coreapiscenario-core_dispatch_no_args_listeners

scenario:coreapiscenario-core_dispatch_with_results_1_listener

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

scenario:coreapiscenario-core_dispatch_with_results_listeners

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

Known flaky benchmarks

Uh oh!

Mascarpone commented Nov 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 9, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

datadog-official Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Mascarpone commented Nov 9, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Nov 9, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Nov 9, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Nov 9, 2026 •

edited

Loading

datadog-official Bot commented Nov 9, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

pr-commenter Bot commented Nov 9, 2026 •

edited

Loading