Repository navigation
Add generic multimodal request preparation to the harness - #21
Merged
Merged
Conversation
Update the disable sentinel test to assert that a disabled limit keeps zero files rather than being clamped up to one, matching the intended behaviour that files_disabled exists to preserve. Auto-committed-on: dragonfly
… limits `FileLimits::effective` now returns zero for `max_files` instead of clamping it up to one, so the hard-disable sentinel survives the clamp and callers no longer need to check `files_disabled` first. Documentation was updated to reflect that the sentinel is preserved rather than lifted. Auto-committed-on: dragonfly
Expose a new request module with media request configuration, model, source resolver, and image readout types so callers can build and resolve multimodal media requests. Auto-committed-on: dragonfly
Add tests covering multimodal request construction and handling. Auto-committed-on: dragonfly
…xtraction Test helpers now construct ModelRequest with a struct literal and default fields, and the two fallback tests extract rendered text directly from the user message content blocks instead of relying on a removed helper. This keeps the tests compiling after the request type and text accessor changed. Auto-committed-on: dragonfly
The effective file limit no longer special-cases zero, so a configured max_files of 0 now clamps to 1 instead of disabling uploads. Tests were updated to compare serialized requests and to use owned scope strings. Auto-committed-on: dragonfly
The effective file limit now returns zero when max_files is zero instead of clamping it up to one, so a zero setting keeps disabling file attachments rather than silently allowing a single file. The remaining changes are formatting and import reordering with no behaviour change. Auto-committed-on: dragonfly
Remote attachment sources are now rejected before resolution when the relevant allow_remote_fetch setting is off, and max_files == 0 is kept at zero by effective() so a source asking for no attachments cannot admit one. The deferred fetch call is also forwarded to the inner model. Auto-committed-on: dragonfly
Adds the request module to the architecture overview and file table, covering MediaRequestModel and the resolver/readout seams the host supplies. Rewrites the limits note to reflect that max_files == 0 now stays zero in FileLimits::effective and is checked before any source read, and clarifies which responsibilities remain with the host. Auto-committed-on: dragonfly
…disabled The hard-zero check only inspected the latest user message, so attachments referenced earlier in the conversation slipped past it. It now scans every message before rejecting, and tests cover the historical-media case plus request immutability on resolution errors and cancellation. Auto-committed-on: dragonfly
Historical media blocks are now resolved newest-first under a configurable budget, and blocks outside it are replaced with host-rendered placeholders instead of failing the request. A new MediaHistoryFallback trait lets hosts decide which old resolution failures are recoverable, and the scope callback now receives the disposable request copy so hosts can consume trusted metadata without mutating the canonical transcript. Auto-committed-on: dragonfly
…lbacks Add tests for per-call metadata scope consumption under concurrency, historical media budget bounding reads and replacing older items, and safe degradation when historical media is missing while latest media failures stay fatal. Auto-committed-on: dragonfly
Adds the ImageRef import needed by the multimodal request tests so the image reference type is in scope. Auto-committed-on: dragonfly
…matting Export MediaHistoryFallback from the multimodal module and reflow the request wrapper and its tests to satisfy rustfmt. The doc comment now notes that scope may come from trusted host request metadata in addition to model state. Auto-committed-on: dragonfly
Update the multimodal README to describe the bounded historical-media budget, the MediaHistoryFallback host seam, and selected-model ModelProfile facts, and adjust the request tests to use NoTextExtractor instead of a default Arc. Auto-committed-on: dragonfly
The module docs now describe the generic pipeline as leaving source access, scope, selected-model facts, and other host policy behind explicit seams, and the host-owned table lists the source resolver, model profile, and placeholder policies alongside the existing entries. The count-before-resolving rule now points at MediaRequestModel, which enforces turn caps and the history budget against raw markers before any resolver runs. Auto-committed-on: dragonfly
The archive fallback test now carries a cfg attribute for the multimodal feature so it is only compiled when that feature is enabled, matching the other tests in the module. Auto-committed-on: dragonfly
Historical text blocks now pass a historical flag into text preparation so image and file resolution failures fall back to placeholders individually instead of failing the whole block. This keeps recovered history intact when only some attachments are unavailable. Auto-committed-on: dragonfly
Add a test that verifies historical text recovers each failed media marker without losing surrounding prose or valid attachments, and that the unavailable hook receives the modality, source, and error. The request module only picks up rustfmt reflow of the recover_history calls. Auto-committed-on: dragonfly
Rename the historical request binding for clarity and assert that a source failure in the latest user turn stays fatal while the unavailable history entry is still recorded. Auto-committed-on: dragonfly
Rename the local request variable to historical_request and reformat the invoke call so the test reads more clearly. No behaviour change. Auto-committed-on: dragonfly
The history budget now selects individual image and file references inside a text block instead of dropping the whole block, so a block with more markers than remaining budget keeps its newest references and replaces only the rest with omission notices. The trusted scope callback also runs before limit checks and history selection so those checks see the normalized request, and typed media blocks are counted Auto-committed-on: dragonfly
Attachment text from file fallbacks, image readouts, and archive extraction was previously passed through unbounded, so large attachments could blow past the configured context budget. A new limit_context helper now truncates each of these strings to the effective file context limit and appends a truncation marker. Auto-committed-on: dragonfly
Add a test that exercises image and archive fallback context with a long unicode readout and a zip listing of many long names, asserting both contexts are truncated and remain within the configured character limit. Auto-committed-on: dragonfly
The config field and its documentation now count individual media items rather than blocks, since each media marker and typed media block is resolved separately. The scope callback docs were also expanded to note that it runs before media counting, must not perform source I/O, and is invoked even for requests with no user messages. Auto-committed-on: dragonfly
Reformat assertions and chained calls in the multimodal request tests to satisfy rustfmt line-width rules. No test behaviour or coverage changed. Auto-committed-on: dragonfly
…n test The assertion now checks messages and metadata separately instead of relying on whole-struct equality, so the test no longer depends on the request type's PartialEq implementation. Auto-committed-on: dragonfly
Clarify the request model's budget and fallback behavior, note the synchronous trusted-request callback's constraints, and add the request test module to the file table. Also document that image readout and archive fallback context is capped by max_extracted_text_chars and marked when truncated. Auto-committed-on: dragonfly
File payloads and their fallback text are now composed into a separate context block that is passed through the context limiter before being appended to the message, so large file attachments no longer bypass the configured context budget. Auto-committed-on: dragonfly
The comment above the latest-turn media count still referred to deriving scope, which no longer happens at that point. It now says the count is taken before calling any source resolver, matching the code. Auto-committed-on: dragonfly
Media resolution now returns a ResolvedMedia wrapper that pairs the attachment with optional host-authored annotations, so safe workspace hints or recovered-source labels can be preserved next to native media and text fallbacks. Annotations are emitted before native blocks and merged into fallback text, while failed resolutions still cannot produce them. Auto-committed-on: dragonfly
Media source resolvers can now attach annotations to resolved media via a ResolvedMedia type, and those annotations are joined into the fallback text alongside the existing readout. This lets resolvers surface provenance or recovery context that would otherwise be lost when an attachment is inlined. Auto-committed-on: dragonfly
File payloads, annotations, and archive text are now combined into a single context string per attachment before being appended to the text output, instead of accumulating payloads and fallback text separately and merging them at the end. This keeps each document's annotations and archive text adjacent to its payload and applies the context limit once per attachment. Auto-committed-on: dragonfly
Add a test asserting that the extracted-text character limit applies to each attachment independently rather than being shared across a turn. Two files whose combined text exceeds the configured limit must both still be rendered in full. Auto-committed-on: dragonfly
Introduce a RequestScopeFactory type alias for the scope closure stored on MediaRequestModel, and rework the multimodal request tests to build ModelRequest values with struct update syntax and to scope captured-request locks in blocks so they are released before later assertions. Auto-committed-on: dragonfly
Clarify that a resolver returns ResolvedMedia so host-authored safe path or recovery annotations stay beside native media and in fallbacks. Auto-committed-on: dragonfly
Document the host adoption map for issue 482, laying out the adoption path and current status so the plan is captured in the repository. Auto-committed-on: dragonfly
Correct the path to the attachment provider in the request preparation row so the adoption map points at the actual host file. Auto-committed-on: dragonfly
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a generic
ChatModelrequest decorator for multimodal marker and typed media. Hosts provide an immutable per-call scope, authorized source resolver, safe historical fallback, selected model profile, optional image readout, and the existingTextExtractor; TinyAgents prepares a disposable request, checks limits before reads, bounds historical items, preserves safe resolver annotations, and renders capability-aware fallbacks.max_files == 0is preserved as a hard disable. Historical media failures recover per reference without losing surrounding prose or successful media, and fallback context is Unicode-safe and bounded per attachment. Tests cover concurrent scopes, request normalization, invoke/stream history immutability, source-read counts, history recovery/budget, native media, image/archive fallback, PDF extraction, annotations, and truncation.Validation:
cargo test -p tinyagents-harness --features multimodal multimodal::request::tests(18 passed)cargo check -p tinyagents-harnessandcargo check -p tinyagents-harness --features multimodalcargo clippy -p tinyagents-harness --all-targets -- -D warningscargo clippy -p tinyagents-harness --features multimodal --all-targets -- -D warningscargo fmt --all -- --checkRefs tinyhumansai#482
Part of tinyhumansai/openhuman#7929