Skip to content

feat(telemetry): report the content-safety provider and categories (#1183) - #1187

Merged
tommasobolis merged 1 commit into
developfrom
feat/1183-content-safety-provider-span
Oct 20, 2026
Merged

tommasobolis merged 1 commit into
developfrom
feat/1183-content-safety-provider-span

Conversation

@tommasobolis

Copy link
Copy Markdown
Collaborator

Closes #1183

Summary

  • Exception: ContentSafetyBlocked.provider is a stable slug: azure-content-safety, bedrock-guardrails, google-model-armor or crowdstrike-aidr. The slug sits next to the display name in _CONTENT_SAFETY_VENDORS, and content_safety_reject() returns (provider, vendor, categories). The message text is unchanged.
  • Span: refusal spans gain donkey.policy.provider (the slug) and donkey.policy.categories (a string array of flagged categories). The constants are DONKEY_POLICY_PROVIDER and DONKEY_POLICY_CATEGORIES in core/telemetry.py, added to __all__. build_genai_attributes takes them as policy_provider and policy_categories.
  • Generic naming: _policy_detail in transport/policy.py keys on the attribute, not the class. Any refusal that carries provider reports it, with categories when they're non-empty. When the federated-guardrail types (refusals: type the Akamai federated guardrail verdict #313, refusals: detect the F5 Guardrails block, which returns HTTP 200 #1177) set those two fields, they get the same span attributes with no transport change. A PII or budget refusal emits neither attribute.
  • Ratchet: errors.py (1031) and telemetry.py (630) stay at their ceilings. I offset the growth by rewrapping docstrings and comments to the 100-column limit, so no wording was dropped.
  • Docs: updated telemetry.mdx (the attribute list plus a paragraph on grouping refusals by provider) and errors.mdx, then regenerated website/public.

Tests

  • One per reject.content-safety-* fixture (Azure, Bedrock, Model Armor, CrowdStrike AIDR):
    • .provider and .categories on the exception;
    • both span attributes through DonkeyAsyncClient.
  • A PII refusal emits no donkey.policy.provider or donkey.policy.categories. PIIDetected has no .provider.
  • build_genai_attributes covers the new keys and leaves them out when unset.
  • Gate: nox -s typecheck-and-lint test-3.11 new-dependencies commit-identities passes locally.

Follow-up

The live acceptance telemetry group asserts only donkey.policy.type=content_safety. It still passes, and asserting donkey.policy.provider live is a small follow-up there.

🤖 Generated with Claude Code

…1183)

ContentSafetyBlocked gains .provider, a stable slug (azure-content-safety,
bedrock-guardrails, google-model-armor, crowdstrike-aidr) carried in
_CONTENT_SAFETY_VENDORS beside the display name. The refusal span gains
donkey.policy.provider and donkey.policy.categories, emitted only for a
violation that carries a provider, so a PII or budget refusal omits both.
The names are generic so the federated-guardrail types (#313, #1177)
reuse them.

errors.py and telemetry.py stay at their ratchet ceilings: the growth is
offset by reflowing docstrings and comments, with no wording dropped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the enhancement New feature or request label Oct 20, 2026
@tommasobolis
tommasobolis merged commit 1f38163 into develop Oct 20, 2026
48 checks passed
@tommasobolis
tommasobolis deleted the feat/1183-content-safety-provider-span branch October 20, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report the content-safety provider and categories on the exception and span

1 participant