Skip to content

chore(aiguard): add backend-provided tags to AI Guard spans - #21145

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
mainfrom
avara/APPSEC-70768-aiguard-backend-extra-tags
Oct 21, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
mainfrom
avara/APPSEC-70768-aiguard-backend-extra-tags

Conversation

@avara1986

@avara1986 avara1986 commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Description

Jira: APPSEC-70768

The AI Guard evaluate response can now carry an optional extra_tags object (data.attributes.extra_tags), so new evaluation details reach AI Guard spans without a tracer change per tag.

  • Scalar values (str, int, float, bool) are set as tags on the ai_guard span with the key unchanged. Booleans are written as "true"/"false", matching the other AI Guard tags.
  • Structured values (dict, list) are added to the ai_guard meta_struct under their key.
  • Values the tracer computes are never overridden: ai_guard.action, reason, target, tool_name, blocked, redacted, event, the anomaly-detection copies, and the meta_struct keys messages, attack_categories, sds, tag_probs.
  • Keys that Span.set_tag treats as span controls are also reserved: manual.keep, manual.drop, service.name, service.version, _dd.measured, version, env.
  • Backend tags may override customer tags passed through the tags= parameter.
  • Tags are applied before the block decision, so they land on both the allow and blocking paths.
  • Fail-safe: a malformed payload (not a dict, non-string or empty keys, unsupported value types) is skipped with a debug log and never changes the evaluation result or telemetry.

No public API change.

Testing

  • New tests/aiguard/api/test_extra_tags.py (15 tests): scalars, structured values, reserved tracer and control keys, anomaly-tag reservation, customer-tag override, blocking path, malformed and null payloads, absent field, and an exception raised while applying tags.
  • ai_guard_api suite passes on py3.9, 3.11 and 3.13; ai_guard_openai suite passes on py3.13.
  • Lint (format, style, typing, spelling, security), plus import cycle and layering checks: no new findings.

Risks

Low. The new path only runs when the response contains extra_tags, and it is wrapped so that it cannot fail the evaluation. The response field name follows the expected backend contract; if that changes, the parsing is a one-line update.

Additional Notes

Backend values are not size-capped; they come from decoded JSON returned by the AI Guard service.

🤖 Generated with Claude Code

The evaluate response may carry an extra_tags object. Scalar values are
set as tags on the ai_guard span and structured values are added to the
ai_guard meta_struct, so new evaluation details reach spans without a
tracer change per tag. Tags computed by the tracer and span control keys
are never overridden, and malformed payloads are skipped without
affecting the evaluation.

APPSEC-70768

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/aiguard/_api_client.py                                          @DataDog/asm-python
tests/aiguard/api/test_extra_tags.py                                    @DataDog/asm-python
tests/aiguard/utils.py                                                  @DataDog/asm-python

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 163 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 163 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=121)
ddtrace.llmobs._integrations.openai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=119)
ddtrace.llmobs._integrations.bedrock_agents -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=119)
ddtrace.llmobs._utils -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=119)
ddtrace.llmobs._integrations.langgraph -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=119)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@avara1986

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T13:00:44.589988Z 4f05433 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 16b5696a5b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@avara1986 avara1986 added the changelog/no-changelog A changelog entry is not required for this PR. label Oct 21, 2026
@avara1986 avara1986 changed the title feat(aiguard): add backend-provided tags to AI Guard spans chore(aiguard): add backend-provided tags to AI Guard spans Oct 21, 2026
Backend-provided tags are an internal mechanism to add span tags without a
tracer release, so no customer-facing release note is needed.

APPSEC-70768

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@avara1986
avara1986 marked this pull request as ready for review October 21, 2026 12:25
@avara1986
avara1986 requested a review from a team as a code owner October 21, 2026 12:25
@avara1986
avara1986 requested review from christophe-papazian and removed request for a team October 21, 2026 12:25

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Reserved span controls and tracer-owned AI Guard fields remain protected, and extra tags are applied before the block decision. Validation used source inspection and isolated checks; the test suite was not run.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 247e644

@florentinl florentinl left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

Just a tiny optional typing nit.

Comment thread ddtrace/aiguard/_api_client.py Resolved
Type the span as SpanData and write extra tags through _set_attribute,
and type the payload as object so the isinstance checks narrow it.

APPSEC-70768

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 779fee5 into main Oct 21, 2026
438 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the avara/APPSEC-70768-aiguard-backend-extra-tags branch October 21, 2026 14:37
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Oct 27, 2026
…alue lists (#21252)

## Description

Jira: [APPSEC-70768](https://datadoghq.atlassian.net/browse/APPSEC-70768)

Follow-up to #21145, which has not been released yet. The AI Guard evaluate response now carries backend-provided span data in two separate attributes, each a list of key/value objects. The keys are dotted (for example `ai_guard.evaluate_id`), which are not valid JSON:API member names, so they cannot be object keys:

```json
"extra_tags":       [{ "key": "ai_guard.evaluate_id", "value": "<same UUID as data.id>" }],
"extra_metastruct": [{ "key": "policy_match", "value": { "rule": "credit_card", "categories": ["pii"] } }]
```

Today the response only sends `ai_guard.evaluate_id` in `extra_tags`; `extra_metastruct` is not sent yet (the entry above only illustrates its shape).

- `extra_tags` (string values): each entry is set as a tag on the `ai_guard` span under its key exactly as returned. Non-string values are skipped with a debug log; structured values here no longer go to the meta_struct.
- `extra_metastruct` (any value): each non-null entry is added to the `ai_guard` meta_struct under its key exactly as returned, scalars included.
- A payload that is not a list, or an entry that is not an object or has a missing, non-string or empty `key`, is skipped with a debug log. On duplicate keys the last entry wins.
- Unchanged from #21145: reserved tracer and span-control keys are never overridden, the tracer adds no `ai_guard.` prefix, and a malformed payload never changes the evaluation result or telemetry.

No public API change.

## Testing

- `tests/aiguard/api/test_extra_tags.py` updated for the list shape: tags on the span, `extra_metastruct` values (structured and scalar) in the meta_struct, a full response document with `ai_guard.evaluate_id` in `extra_tags` only, equal to `data.id`, next to attributes the tracer does not read, duplicate keys, structured `extra_tags` kept out of the meta_struct, reserved keys for both fields, malformed containers and entries, and null payloads for each field.
- `ai_guard_api` suite passes on py3.13 (430 tests); `test_extra_tags.py` passes on py3.9.
- Lint (format, typing, spelling): clean.

## Risks

Low. Neither field has shipped in a release, and both code paths only run when the response contains them.

## Additional Notes

No release note: #21145 is unreleased and this does not change customer-facing behaviour.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


[APPSEC-70768]: https://datadoghq.atlassian.net/browse/APPSEC-70768?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ


Co-authored-by: alberto.vara <alberto.vara@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog A changelog entry is not required for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants