Skip to content

Fix video agent API evidence collection and answer quality - #22

Merged
devhims merged 3 commits into
mainfrom
fix/video-agent-answer-quality
Sep 5, 2026
Merged

Fix video agent API evidence collection and answer quality#22
devhims merged 3 commits into
mainfrom
fix/video-agent-answer-quality

Conversation

@devhims

@devhims devhims commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Problem

A production request for “suggest the top use cases of gpt 6 astra” exhausted the research window without usable transcript evidence and returned promotional discovery text as a partial answer. Local reproductions also exposed extra search planning, transcript selection in separate waves, and invalid structured model output.

Changes

  • Execute the classifier's search query immediately, preserving the one-search budget and compatibility with persisted routes.
  • Batch selected transcripts with independent persistence and billing. Allow two concurrent analysts for focused research and four for comparative research; inspection retains two.
  • Return an explicit NO_CONTENT_EVIDENCE fallback when content analysis fails, instead of synthesizing recommendations from titles and descriptions. Track transcript attempts synchronously so deadline aborts cannot race failure recording.
  • Guide analysts and synthesis toward three practical, source-attributed recommendations, with relevant examples and caveats.
  • Expose classifier and answer tool parameters as top-level objects while retaining conditional route and citation validation.
  • Document scheduling, fallback behavior, and local quality checks.

The 60-second deadline and 20-second finalization reserve remain. The proposed public response redesign is separate from this fix.

Validation

  • 334 unit tests across 50 files passed, including batch concurrency, partial failures, all-transcript timeout recovery, one-search enforcement, and conditional schema validation.
  • 10 runtime integration tests passed.
  • TypeScript and generated documentation checks passed; diff whitespace checks passed.
  • Local live GLM research on the exact reported query analyzed four videos and returned a three-option answer in approximately 51 seconds, without partial-evidence or coverage-shortfall warnings.
  • Local live GLM inspect regression completed in approximately 39 seconds with transcript and storyboard evidence.

The final timeout race guard was validated with a regression test after the live runs. These timings demonstrate successful local runs, not a latency guarantee. Source claims remain attributed rather than independently verified. Production verification follows merge and deployment.

@mintlify

mintlify Bot commented Sep 5, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
video2ctx 🟢 Ready View Preview Sep 5, 2026, 5:44 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
video2ctx-web Ready Ready Preview Sep 5, 2026 5:43pm UTC

@devhims
devhims merged commit 47b4e3c into main Sep 5, 2026
7 checks passed
@devhims
devhims deleted the fix/video-agent-answer-quality branch September 5, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant