Skip to content

ci(instrumentation): enforce contrib manifest name matches TracedPackage - #5831

Open
dd-apm-ecosystems[bot] wants to merge 2 commits into
mainfrom
automated/lint-manifest-name-traced-package-2026-11-05
Open

dd-apm-ecosystems[bot] wants to merge 2 commits into
mainfrom
automated/lint-manifest-name-traced-package-2026-11-05

Conversation

@dd-apm-ecosystems

Copy link
Copy Markdown

What does this PR do?

Adds TestManifestNameMatchesTracedPackage to instrumentation/contrib_test.go. It walks every contrib/**/manifest.yaml, reads the top-level name, and fails if that value is not the TracedPackage of a package registered in instrumentation/packages.go.

The manifest name identifies the integration (the instrumented library). packages[].name lists the published packages and is intentionally allowed to differ, so only the top-level name is checked. The test also fails if no manifests are found, so a wrong walk root can't silently pass.

Motivation

Reviewers have repeatedly had to ask contributors to change a new manifest's top-level name to the integration's TracedPackage, "like every other manifest". Both times the contributor agreed and pushed a fix commit. The check is a pure structural lookup, so it shouldn't depend on reviewer attention.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • New code is free of linting errors. You can check this by running make lint locally.
  • New code doesn't break existing tests. You can check this by running make test locally.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • All generated files are up to date. You can check this by running make generate locally.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. Make sure all nested modules are up to date by running make fix-modules locally.

Self-verified:

  • gofmt -l instrumentation/ is clean.
  • The pushed blob hash (4a91f43c…) matches the locally tested file.
  • I ran the new test body in an isolated scratch module against the real contrib/ tree and TracedPackage values from packages.go at 1bdbf936. It passes: all 71 current manifests already satisfy the rule, so no existing file needs changing.
  • I temporarily set contrib/envoyproxy/go-control-plane/manifest.yaml name to github.com/envoyproxy/go-control-plane/envoy and confirmed the test fails, naming the manifest. I then reverted the change locally; it is not in this diff.
  • Not verified: I could not run go test ./instrumentation/ or make lint end to end. The sandbox's module proxy returned 502 for the otelc dependency, so the root module wouldn't build. CI will run the real thing.

Automation Rationale (context for reviewers)

(a) Contributing occurrences (all new this run; human-raised, human-accepted):

  • manifest-name-traced-package|pr5654|4358986357: PR chore(contrib): backfill AppSec processor manifests #5654 (author dd-evgenii-vasilev). Reviewer rarguelloF flagged that name should be the TracedPackage. The author replied "Good point, updated" (comment 4359881237) and pushed commit 0b036cfc.
  • manifest-name-traced-package|pr5656|4358986654: PR chore(contrib/aws/datadog-lambda-go): add instrumentation manifest #5656 (author dd-evgenii-vasilev). Reviewer rarguelloF asked why name differed from TracedPackage, noting "All the other manifests use the TracedPackage value". The author replied "Good point, updated" (comment 4359882056) and pushed commit 21041828.

Table C had no do_not_implement or implement_differently rows, so no prior maintainer feedback applies.

(b) Why this is low-false-positive and mechanical: The check is a set-membership test on one YAML scalar against a fixed set of Go string constants. It needs no semantic judgment. All 71 existing manifests pass today, so adoption causes zero failures. The legitimate difference between name and packages[].name is left unchecked.

(c) Tool and CI wiring: I added a native Go test next to the existing convention tests (TestNoSetTagServiceName, TestNoTracerServiceName) in instrumentation/contrib_test.go. It runs with go test ./instrumentation/.... instrumentation/** is claimed by the core component in .github/ci-components.yml (gate @everything), so it runs in the merge-gating test jobs. No new workflow, script, or golangci-lint config was needed, because manifests are YAML and packages.go is an in-package map that a generic linter can't cross-reference. The test uses go.yaml.in/yaml/v3, which is already a direct dependency in the root go.mod.

(d) Verification command: go test ./instrumentation/ -run TestManifestNameMatchesTracedPackage -v

(e) Not shaped by an implement_differently row; none exist.

Related automation PRs for other themes: #5612 (naming map), #5774 (typed ext.Component) and #5708 (configinverter) are still open and independent of this change.

@dd-apm-ecosystems
dd-apm-ecosystems Bot requested a review from a team as a code owner November 5, 2026 22:10
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T22:12:53.025913Z 7a2342d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7a2342d19c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread instrumentation/contrib_test.go Outdated
Comment on lines +379 to +380
if _, ok := traced[m.Name]; !ok {
offending = append(offending, fmt.Sprintf("%s: name %q", relPath, m.Name))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject names belonging to another integration

When a contributor copies an existing manifest while adding a new integration and forgets to update its top-level name, this membership check still passes because the copied name is already present in traced. The manifest therefore need not match its own integration's TracedPackage, defeating the stated check in a likely creation workflow; associate each manifest with its corresponding package entry or at least reject duplicate manifest names.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 808df61. The check now looks up TracedPackage by contrib path (so a copied name fails against that integration's own package entry) and also rejects duplicate manifest names, including the case where TracedPackage was copied too.

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static verification confirmed that all 71 existing manifest names satisfy the new check, while packages[].name remains unconstrained. The targeted Go test remains unverified because the module proxy returned HTTP 502 for otelc.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 7a2342d

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Pipelines  Tests

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 3 Pipeline jobs failed

OTelc | Integration Test (windows | oldstable) — 🔄 Retry may pass, looks flaky

View more details · View in GitHub Actions

Check Pull Request CI Status | all-jobs-are-green

View more details · View in GitHub Actions

OTelc | OTelc Integration Tests

View more details · View in GitHub Actions

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 73.49% (-0.28%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 808df61 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-11-09 15:10:13

Comparing candidate commit 808df61 in PR branch automated/lint-manifest-name-traced-package-2026-11-05 with baseline commit 1bdbf93 in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 415 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:BenchmarkContextWithSpan/detach-no-snapshot

  • unstable execution_time [-1.604ns; +1.528ns] or [-5.458%; +5.199%]

A set-membership check still passes when a new manifest copies another
integration's name. Require the name to match that contrib path's
TracedPackage, and reject duplicate names.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant