Wire PPT generation and file attachments into agent definitions and error handling

Summary

Update the orchestrator and relevant agent definitions to advertise the generate_presentation tool and file-attachment capability. Add structured error variants for unsupported file types, missing runtime dependencies, and generation failures so the agent surfaces clear, actionable messages instead of generic sandbox errors.

Problem / Context

The tool exists but agents will not use it unless their definitions grant access. Likewise, without explicit error types the agent falls back to opaque sandbox failures. This wiring sub-task closes the loop between backend capabilities and the user-facing agent experience.

This sub-task is part of the implementation of #1535 (Add multimodal support for files and PPT generation).

Scope

IN scope: update agent YAML definitions in src/openhuman/agent/agents/orchestrator/ (and code_executor if applicable) to include generate_presentation in tool lists; update src/openhuman/agent/harness/tool_filter.rs to allow the new tool under appropriate policies; add error variants to tools error enum for UnsupportedFileType, MissingRuntime, GenerationFailed; update tool_loop.rs error-mapping to surface these as user-visible messages; add integration test that runs orchestrator with a PPT-generation prompt and verifies tool invocation + artifact output. NOT in scope: new tool implementation, UI changes, multimodal parsing changes, Composio integration.

Acceptance criteria

Orchestrator agent definition includes generate_presentation in its tool scope
Tool filter policy allows generate_presentation for orchestrator and code_executor agents
UnsupportedFileType error includes the rejected MIME type and a list of supported types
MissingRuntime error names the missing dependency and suggests an install command
GenerationFailed error includes the Python stderr output truncated to 500 chars
Integration test sends a create slides prompt through orchestrator and asserts artifact is produced
Error paths are covered by unit tests for each new variant
Diff coverage >= 80%
Diff coverage ≥ 80% — the implementing PR meets the changed-lines coverage gate.

Add: pre-presentation grounding requirement

Surfaced during live dev:app verification of #2779 (PR #3017): asking the agent for "5 slides on current AI trends" results in the LLM (resolved_model=reasoning-v1) inventing slide bullets, body text, and speaker notes from its training data with zero verification — no web_search, no memory.search, no grounding call before generate_presentation dispatch. Real hallucination risk for accuracy-sensitive decks.

Trace from the agent loop:

[agent] dispatching 1 tool(s): ["generate_presentation"]
[agent_loop] tool start name=generate_presentation
[agent_loop] tool finish name=generate_presentation elapsed_ms=225 success=true

One tool call. No retrieval prior.

Proposed scope addition to this issue

Wire the orchestrator's pre-presentation flow so the agent must establish a grounding context before generate_presentation is dispatched:

Orchestrator agent.toml — extend the orchestrator's system prompt with a rule: when the user requests a presentation, FIRST call web_search and / or memory.search to gather sources for any factual claims; THEN call generate_presentation with bullet text drawn from those sources. Skip grounding only when the user explicitly supplies source material in their prompt (e.g. pasted text, attached document, summarized chat earlier in the thread).
Source-tracking field on SlideSpec (optional, deferred to a follow-up if scope balloons) — add a sources: Vec<String> field so the model can attach the retrieval URLs / memory ids it used per slide. Surfaces in speaker notes as a 'Sources:' footer.
Acceptance criterion (add to existing AC checklist):
- Pre-presentation grounding — when the user requests a deck without supplying source material, the agent dispatches at least one grounding tool (web_search / memory.search) before generate_presentation. Verified by a Vitest+RTL or Rust integration test that asserts the tool sequence in a recorded session.

Out of scope here (intentionally)

Fact-check classifier ranking grounding-confidence per bullet — separate concern; would warrant its own issue.
Per-slide source attribution / citation rendering in the final pptx — depends on python-pptx layout work; track via the optional sources field above if added.

Navigation Menu

Repository navigation

Wire PPT generation and file attachments into agent definitions and error handling #2780

Summary

Problem / Context

Scope

Acceptance criteria

Related

Add: pre-presentation grounding requirement

Proposed scope addition to this issue

Out of scope here (intentionally)

Related

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Wire PPT generation and file attachments into agent definitions and error handling #2780

Description

Summary

Problem / Context

Scope

Acceptance criteria

Related

Activity

oxoxDev commented on May 30, 2026

Add: pre-presentation grounding requirement

Proposed scope addition to this issue

Out of scope here (intentionally)

Related

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions