Skip to content

Trace exporter ignores capture_content — prompts/replies written to NDJSON/app log; Langfuse cached-token double-count #4454

Closed

Description

@senamakel

Part of the v0.58.7 → HEAD TinyAgents-migration audit (see parent issue). Introduced by the Langfuse tracing work (#4434 / commit 288e642).

Problem

The capture_content privacy gate is enforced only on the Langfuse push path (src/openhuman/agent/progress_tracing/langfuse.rs:228). The collector attaches the full user prompt and model reply to the turn span unconditionally (src/openhuman/agent/progress_tracing.rs:627-645, fed by the unconditional TurnContent emit at src/openhuman/agent/harness/session/turn/core.rs:1126-1130), and the local exporter spans_to_ndjson/export_spans (progress_tracing.rs:723-782) serializes the whole span with no gate:

  • with export_path set → plaintext prompts/replies appended to the NDJSON file;
  • with export_path unset → printed at info level into the application log (progress_tracing.rs:771-780).

This contradicts the module's own doc ("the exporter still gates transmission behind capture_content", lines 146-147), the config's documented metadata-only default, and the project rule "Never log secrets or full PII". Tests set capture_content: false but never assert content absence.

Related (same PR): Langfuse usageDetails double-counts cached tokens — langfuse.rs:184-196 emits input (inclusive of cached) plus cache_read_input_tokens; Langfuse treats keys as disjoint, so aggregation over-counts and the explicit total disagrees with the component sum (src/openhuman/agent/cost.rs:171 shows cached is a subset of input).

Fix plan

  1. Gate content at the span-storage level: only attach input/output to spans when capture_content is true (single choke point, protects all exporters). Alternatively strip in export_spans when the flag is false — but storage-level is safer for future exporters.
  2. Never log span content at info; if no export_path, either skip content entirely or log metadata-only at debug.
  3. Fix usageDetails: emit input = input_tokens - cached_input_tokens so components are disjoint and sum to total.
  4. Add assertions to the existing tests: with capture_content: false, the NDJSON/log output contains no prompt/reply text.

Acceptance criteria

  • With capture_content: false (the default), no prompt or reply text appears in the NDJSON export file, the application log, or the Langfuse payload.
  • Langfuse usageDetails components are disjoint and sum to total.

Activity

  1. added
    subtaskSubtask of a larger tracked effort.
    agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.
    securitySecurity, encryption, approvals, credentials, and trust boundaries.
    on Jul 3, 2026
  2. M3gA-Mind commented on Jul 7, 2026

    @M3gA-Mind
    Collaborator

    Resolved by merged PR #4495 ("TinyAgents migration parity & bug fixes (#4451–#4469)") + follow-up #4504 — both present in main. Part of the completed #4470 cluster; closing as done. (Reopen if a residual remains.)

  3. added theissue type on Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.bugpriority: highBreaks a specific featuresecuritySecurity, encryption, approvals, credentials, and trust boundaries.subtaskSubtask of a larger tracked effort.

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions