Skip to content

file transfer (ts): parse images/files, wire directive sink to Rust parity - #346

Merged
brentrager merged 1 commit into
mainfrom
ft-ts
Aug 11, 2026
Merged

file transfer (ts): parse images/files, wire directive sink to Rust parity#346
brentrager merged 1 commit into
mainfrom
ft-ts

Conversation

@brentrager

@brentrager brentrager commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Problem

The file-transfer protocol contract merged in #342 (send_message.files[] + images[], and the send_file directive convention on eventual_response.directive) was spec-only. The Rust server already implements images/directive; the TypeScript server parsed message only, built the user message text-only, and never emitted a directive.

Solution — TS server to Rust parity

  • Regen protocol types (typescript/src/generated/types.ts via scripts/generate.ts): send_message gains images[]/files[]; eventual_response gains directive.
  • Images → model: parsed in the dispatcher and attached to the turn's user message as OpenAI image_url content parts. The published @smooai/smooth-operator-core engine takes a plain-string message and reuses it for retrieval/memory, so a new withUserImages() wrapper rewrites the outgoing request body's current user message ("text" → [{type:'text'},{type:'image_url'}]) — the exact shape Rust's with_user_images produces — while leaving text-based retrieval untouched.
  • Files → tool context: parsed and surfaced on a new per-turn ToolContext (parallel to images), never sent to the model — mirroring the Rust ToolProviderContext.
  • Directive sink: a new optional toolProvider seam (mirrors Rust ToolProvider::tools_for) hands host tools the per-turn context. A host tool writes ctx.directive; the dispatcher drains it after the turn and passes it to the eventual_response builder (last-write-wins, omitted when none).
  • All attachment parsing is fail-soft (a malformed entry is dropped, never rejects the turn).

Notes

  • The TS core engine (1.7.1) has no multimodal seam; the request-body wrapper is the workaround, with a ponytail: upgrade path to a core withUserImages when the engine gains one (as Rust has).
  • Additive + back-compat: no attachments / no provider ⇒ behaviour byte-for-byte unchanged. Existing conformance/scenario-parity suites stay green with no edits.

Tests / gates

  • New withUserImages + parseImages/parseFiles unit tests, plus end-to-end (real WS): images-attach, files-on-context, tool → directive → eventual_response, back-compat omit, and fail-soft. 12 new tests.
  • Green: typecheck + test + build for @smooai/smooth-operator and @smooai/smooth-operator-server (245 server + 41 SDK tests pass).
  • Changeset added (minor on both packages).

🤖 Generated with Claude Code

@brentrager @claude
file transfer (ts): parse images/files, wire directive sink to Rust p…
bbd1c0f 
…arity

Implement the PR #342 file-transfer contract in the TypeScript smooth-operator
server, to parity with the Rust reference:

- Regenerate the client SDK protocol types so send_message gains images[]/files[]
  and eventual_response gains directive.
- images[] are attached to the turn's user message as OpenAI image_url content
  parts. The published core takes a plain-string message and reuses it for
  retrieval, so a withUserImages() wrapper rewrites the outgoing request body's
  current user message instead (retrieval/memory stay text-based).
- files[] are surfaced on a new per-turn ToolContext (never sent to the model),
  mirroring the Rust ToolProviderContext.
- New optional toolProvider seam (mirrors Rust ToolProvider): host tools bound to
  the turn's context can read the attachments and write ctx.directive; the
  dispatcher drains it onto eventual_response.directive (last-write-wins).
- All attachment parsing is fail-soft.
- Tests: withUserImages/parse unit tests + end-to-end images-attach,
  files-on-context, tool->directive->eventual_response, and fail-soft.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: bbd1c0f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@smooai/smooth-operator-server Minor
@smooai/smooth-operator Minor
@smooai/smooth-operator-web-chat-example Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@brentrager
brentrager merged commit d41643e into main Aug 11, 2026
1 check passed
@brentrager brentrager mentioned this pull request Aug 14, 2026
Merged
brentrager added a commit that referenced this pull request Aug 14, 2026
@brentrager @claude
release: bump the lockstep anchor so the .NET server actually ships #348
15dab99 
 + #352 (#354)

SmooAI.SmoothOperator.Server is stamped from the lockstep anchor
(@smooai/smooth-operator, typescript/package.json) by sync-versions.mjs.
#348's changeset named only @smooai/smooth-operator-server — which is the
TYPESCRIPT server npm package — so release #349 bumped that to 1.8.0 and left
the anchor at 1.39.0. The .NET NuGet was therefore never republished, and
1.39.0 consumers see no TurnContext, no directive sink and no images[]/files[]
ingest despite main carrying all of it since 2026-08-11. #352 (skill
resolution) inherited the same mistake by copying #348's changeset header.

The sibling TS PR #346 named BOTH packages, which is why the TS lane shipped
and the .NET lane silently did not.

Reported by a downstream consumer who checked the published package rather
than main — the report was right about the artifact and wrong about the code.


Claude-Session: https://claude.ai/code/session_012iM1Q8JC1H83H2FXQVQNs9

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 14, 2026
Merged
Merged
brentrager added a commit that referenced this pull request Aug 14, 2026
@brentrager @claude
th-54fc00: anchor guard — fail a PR that changes a lockstep-stamped t…
7557a3e 
…ree without naming the anchor (#356)

Every artifact ships at one shared version held in @smooai/smooth-operator
(typescript/package.json). Changesets versions only npm packages, so
sync-versions.mjs stamps that number onto every other published manifest. A
changeset that does not name the anchor therefore republishes nothing outside
npm — silently, with a green release.

That is how #348 (.NET file transfer) and #352 (.NET skill resolution) both
landed on main and sat unpublished: their changesets named
@smooai/smooth-operator-server, which is the TYPESCRIPT server package sitting
one word away from the anchor. A downstream consumer found it two days later by
reading the NuGet and filed a request to build what we had already shipped.

Rule: touching a stamped tree WITH a changeset requires one changeset naming the
anchor. Conditioned on having a changeset at all, so docs/test-only PRs stay
quiet — a guard that cries wolf gets ignored, and then it protects nothing.

sync-versions.mjs is now importable (stamping runs only when invoked directly)
so the guard derives the stamped trees from the very list that does the
stamping. A new target cannot be added without the guard learning about it.

Verified by replay against real history: the guard FIRES on #348's actual commit
and PASSES #346's (the TS sibling that correctly named both packages).


Claude-Session: https://claude.ai/code/session_012iM1Q8JC1H83H2FXQVQNs9

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@brentrager brentrager mentioned this pull request Aug 14, 2026
Merged
brentrager added a commit that referenced this pull request Aug 14, 2026
@brentrager @claude
th-ebe27d: skill resolution (ts) — resolve send_message.skill server-…
caa3678 
…side, Rust #338 parity (#357)

Second language in the fan-out after .NET (#352). The TS server had the field on
the wire and ignored it, exactly as its own changelog admitted.

- skills.ts: isValidSkillName / stripFrontmatter / skillSection / resolveSection
- SkillResolver seam via serve({ skillResolver }); DirSkillResolver over
  SMOOTH_SKILLS_DIR with explicit-wins-then-env, mirroring Rust's
  install_skill_resolver_from_env
- Fail-CLOSED, and resolved BEFORE the 202 ack so a client never gets
  "accepted" for a turn that will never run
- Body appended LAST to the system prompt; the persisted user message stays
  exactly what the user typed

Name validation makes traversal unrepresentable rather than filtered.

Changeset names BOTH the TS package and the lockstep anchor, per #346 — the
omission of the anchor is what stranded the .NET work in #348/#352.

Tests: five Rust skills.rs tests ported under their Rust names + over-the-socket
fail-closed / placement / blank-as-absent. 254 green (245 baseline + 9).


Claude-Session: https://claude.ai/code/session_012iM1Q8JC1H83H2FXQVQNs9

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@brentrager