Skip to content

feat(session): add structured output from Zod schemas - #37

Merged
corasan merged 3 commits into
fm27/generation-optionsfrom
fm27/structured-output
Oct 16, 2026
Merged

corasan merged 3 commits into
fm27/generation-optionsfrom
fm27/structured-output

Conversation

@corasan

@corasan corasan commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Guided generation is the main feature of Foundation Models. The model is constrained to a schema, so its output always parses. Before this change, the library used it only for tool arguments. Apps that wanted typed output had to prompt for JSON and parse text.

Scope

  • respond and streamResponse accept schema in the options object from layer 2. TypeScript overloads pick the return type:

    const Recipe = z.object({
      title: z.string(),
      difficulty: z.enum(['easy', 'medium', 'hard']),
      ingredients: z.array(z.object({ name: z.string(), quantity: z.string() })),
    })
    
    const recipe = await session.respond('A quick pasta recipe', { schema: Recipe })
    //    ^? z.infer<typeof Recipe>
    
    const final = await session.streamResponse(
      'A quick pasta recipe',
      partial => render(partial), // DeepPartial<z.input<typeof Recipe>>
      { schema: Recipe, temperature: 0.2 },
    )

    Without schema, the signatures and behavior are the same as before. A non-object schema (for example z.string()) is a type error and also a runtime SCHEMA_CREATION_ERROR.

  • The final value is parsed with the Zod schema, so defaults and transforms apply. A value that does not parse rejects with a new stable code, RESPONSE_VALIDATION_ERROR. details.issues holds the Zod issues and details.response holds the raw value. This happens when the schema has checks the model cannot see, such as .refine().

  • Streamed partial objects are passed through without validation.

  • One schema pipeline for tools and responses. The TypeScript side moves out of tool-utils.ts into generation-schema.ts as toGenerationSchema(schema, errorPathRoot). Tools use the root arguments and responses use response, so error paths read response.address.city. On the Swift side, ToolSchemaBuilder becomes GenerationSchemaBuilder with toolParameters(from:) (JSON Schema or the legacy flat map) and responseSchema(from:) (object JSON Schema only). Tool value conversion, which responses do not need, moves to ToolContent. createTool and HybridTool call the new names. SCHEMA_CREATION_ERROR messages say "Failed to create schema" instead of "Failed to create tool schema".

  • Native: respond and streamResponse take an optional schema: AnyMap. There are no separate schema methods. With a schema, Swift calls respond(to:schema:options:) and streamResponse(to:schema:options:) and returns GeneratedContent.jsonString. With no includeSchemaInPrompt argument, these resolve without ambiguity on the iOS 26 and iOS 27 SDKs, and the default is true on both. So no #if compiler branch is needed here. Schema requests go through the same session check, option mapping, availability check, busy lock, and layer 1 error mapping and context-overflow recovery as text requests.

  • TypeScript: each call builds one request descriptor (native schema, chunk decoder, final parser), so respond and streamResponse each have one path. ObjectSchema is z.ZodObject.

  • Hooks: useLanguageModel and useStreamingResponse still return text only. Their state is response: string. A schema-aware hook needs a generic over the schema and typed state per call. That is a new hook, not an option, so it is not in this PR. The docs tell users to call the session methods directly.

  • Example: a new "Structured" tab streams a recipe (nested object array, enum, bounded integer, string array). It shows the fields as they fill in, the latest raw snapshot, and the parsed value at the end.

What a partial snapshot looks like

I checked this at runtime before I designed the partial path. The first probe was a Swift script against the macOS 27 host model. The second was the iOS 27.0 simulator through the new Nitro method, with every native string recorded.

  • Every snapshot's jsonString is valid JSON of the object so far. It is never truncated JSON. On the simulator, 19 of 19 raw snapshots parsed with JSON.parse.
  • Keys appear in the order the model writes them, which is not the schema order. For a contact schema declared name, role, address, skills, the model wrote address first. Key order also changes between snapshots.
  • Strings grow in place: "quantity": "2", then "200g". An enum or a new string starts as "". A new array element starts as {} or "".
  • Numbers appear only when complete. Consecutive snapshots can be equal.

So JS runs JSON.parse on each snapshot and does not need a partial-JSON parser. DeepPartial<T> widens every string type, including enums and literals, to string, because "" is a real partial value for an enum. If a snapshot is not JSON, the stream rejects with RESPONSE_VALIDATION_ERROR and does not blame the callback. The observed data says this does not happen.

Raw simulator snapshots (excerpt):

{"address": {}}
{"address": {"city": "Lisbon"}}
{"name": "", "address": {"country": "Portugal", "city": "Lisbon"}}
{"name": "Ana Ruiz", "address": {"city": "Lisbon", "country": "Portugal"}}
{"address": {...}, "role": "designer", "name": "Ana Ruiz"}
{"name": "Ana Ruiz", "role": "designer", "address": {...}, "skills": ["Figma", "typography", ""]}
{"role": "designer", "name": "Ana Ruiz", "address": {...}, "skills": ["Figma", "typography", "prototyping"]}

Tradeoffs

  • Schema conversion runs on each call. It is small next to a model call, and a cache keyed on the Zod object could hold a stale schema.
  • With a schema, a blank prompt goes to the model (it returned a valid object on the simulator). The text methods still return "" for a blank prompt, because an empty string is not a valid structured value.

Blast Radius

  • Calls without schema do not change. KEYWORDS and UNSUPPORTED_HINTS are still exported from the package root. They now live in generation-schema.ts.
  • The SCHEMA_CREATION_ERROR message text changes from "Failed to create tool schema: ..." to "Failed to create schema: ...". The code does not change.
  • The Nitro spec gained an optional schema argument on respond and streamResponse, so apps must rebuild native code. Swift files were renamed and added, so run pod install again.
  • ToolSchemaBuilder is internal Swift. No public API refers to it.

Verification

  • bun run test: 72 pass. New tests cover the schema sent to native for a nested object, array, enum, and default. They also cover a parsed value with a default applied, a Zod mismatch with the issues in details, non-JSON output, an unsupported schema and a non-object root rejected before native, a typed native failure (CONTEXT_EXCEEDED) through the schema path, streamed partials passed through unvalidated (including an "" enum and a {} element), a final-value mismatch after streaming, and a non-JSON snapshot.
  • bun run test:swift: all suites pass. New cases check the response root name, enum and array encoding, rejection of a flat tool-style map as a response schema, and the response.id error path. The tool tests run against the renamed GenerationSchemaBuilder and ToolContent.
  • bun run typecheck now also checks package/tests (tsconfig.test.json with @types/bun), and the session mock is typed from the Nitro spec. bun run lint, bun run format:check, and example tsc pass. I checked the overloads with a scratch file that was not committed. respond(p) and respond(p, { temperature }) are string. respond(p, { schema }) is the inferred object, and assigning it to string is an error. The stream callback gets typed partial fields. respond(p, { schema: z.string() }) is a type error.
  • iOS 27.0 simulator (iPhone 18 Pro), real model, Xcode 27.1 build:
    • The Structured tab streamed "A quick pasta recipe" in 31 snapshots. The frames at 6, 17, and 26 snapshots show difficulty first, then minutes, then ingredients ("quantity": "2" grows to "200g"), then steps. The last frame reads "Validated after 31 snapshots", with the title filled in last. The dessert prompt gave a 6-ingredient, 6-step recipe with difficulty: "hard", minutes: 180.
    • debugger-evaluate: respond with a nested contact schema (object, enum, bounded array) resolved to {"name":"Ana Ruiz","role":"designer","address":{"city":"Lisbon","country":"Portugal"},"skills":["Figma","typography"]}.
    • A .refine(v => v.length > 200) field rejected with RESPONSE_VALIDATION_ERROR and issues: [{ code: "custom", path: ["fruit"] }].
    • A union schema rejected with SCHEMA_CREATION_ERROR ("Property 'response.id' uses unsupported schema feature: anyOf ...") before the request reached the model.
    • A prompt of about 900 sentences streamed with a schema rejected with CONTEXT_EXCEEDED, and wasContextReset was true afterwards. The layer 1 recovery works on the schema path.
    • After the final rebuild, a createTool tool was called with {"city":"Lisbon"} and the answer used its result. The migrated tool path works.

Closes #22

@corasan
corasan added this pull request to stack #36 September 23, 2026 19:49
@corasan corasan changed the title feat: add structured output from Zod schemas feat(session): add structured output from Zod schemas Sep 23, 2026
@corasan
corasan force-pushed the fm27/structured-output branch from c18462a to 5590502 Compare September 24, 2026 22:59
respond and streamResponse take an optional Zod object schema. With a
schema, the model is constrained to it, the final value is parsed with
Zod, and streamed chunks are partial objects.

The Zod to JSON Schema to GenerationSchema pipeline is now shared by
tools and responses: generation-schema.ts on the TypeScript side and
GenerationSchemaBuilder on the Swift side. Tool value conversion moves
to ToolContent.

Closes #22
The native spec had four request methods that differed only by a schema
argument. respond and streamResponse now take an optional AnyMap schema,
and respondWithSchema and streamResponseWithSchema are gone.

LanguageModelSession builds one request descriptor per call (native
schema, chunk decoder, final parser), so each method has one path. The
any-typed chunk callback and the MaybeStructuredOptions alias are gone,
and ObjectSchema is z.ZodObject.

Blank-prompt short-circuit stays on text requests only. A schema request
has no empty value that satisfies the schema, so it reaches the model.

Tests now typecheck: the session mock is typed from the Nitro spec, and
bun run typecheck also checks package/tests with @types/bun.
@corasan
corasan force-pushed the fm27/structured-output branch from 2dbe1f1 to cb9c1af Compare October 16, 2026 23:04
@corasan
corasan merged commit 03a5c85 into main Oct 16, 2026
6 checks passed
@corasan
corasan deleted the fm27/structured-output branch October 16, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support structured output with Zod schemas

1 participant