Repository navigation
feat(session): add structured output from Zod schemas - #37
Merged
Merged
Conversation
corasan
force-pushed
the
fm27/structured-output
branch
from
September 24, 2026 22:59
c18462a to
5590502
Compare
respond and streamResponse take an optional Zod object schema. With a schema, the model is constrained to it, the final value is parsed with Zod, and streamed chunks are partial objects. The Zod to JSON Schema to GenerationSchema pipeline is now shared by tools and responses: generation-schema.ts on the TypeScript side and GenerationSchemaBuilder on the Swift side. Tool value conversion moves to ToolContent. Closes #22
The native spec had four request methods that differed only by a schema argument. respond and streamResponse now take an optional AnyMap schema, and respondWithSchema and streamResponseWithSchema are gone. LanguageModelSession builds one request descriptor per call (native schema, chunk decoder, final parser), so each method has one path. The any-typed chunk callback and the MaybeStructuredOptions alias are gone, and ObjectSchema is z.ZodObject. Blank-prompt short-circuit stays on text requests only. A schema request has no empty value that satisfies the schema, so it reaches the model. Tests now typecheck: the session mock is typed from the Nitro spec, and bun run typecheck also checks package/tests with @types/bun.
corasan
force-pushed
the
fm27/structured-output
branch
from
October 16, 2026 23:04
2dbe1f1 to
cb9c1af
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Guided generation is the main feature of Foundation Models. The model is constrained to a schema, so its output always parses. Before this change, the library used it only for tool arguments. Apps that wanted typed output had to prompt for JSON and parse text.
Scope
respondandstreamResponseacceptschemain the options object from layer 2. TypeScript overloads pick the return type:Without
schema, the signatures and behavior are the same as before. A non-object schema (for examplez.string()) is a type error and also a runtimeSCHEMA_CREATION_ERROR.The final value is parsed with the Zod schema, so defaults and transforms apply. A value that does not parse rejects with a new stable code,
RESPONSE_VALIDATION_ERROR.details.issuesholds the Zod issues anddetails.responseholds the raw value. This happens when the schema has checks the model cannot see, such as.refine().Streamed partial objects are passed through without validation.
One schema pipeline for tools and responses. The TypeScript side moves out of
tool-utils.tsintogeneration-schema.tsastoGenerationSchema(schema, errorPathRoot). Tools use the rootargumentsand responses useresponse, so error paths readresponse.address.city. On the Swift side,ToolSchemaBuilderbecomesGenerationSchemaBuilderwithtoolParameters(from:)(JSON Schema or the legacy flat map) andresponseSchema(from:)(object JSON Schema only). Tool value conversion, which responses do not need, moves toToolContent.createToolandHybridToolcall the new names.SCHEMA_CREATION_ERRORmessages say "Failed to create schema" instead of "Failed to create tool schema".Native:
respondandstreamResponsetake an optionalschema: AnyMap. There are no separate schema methods. With a schema, Swift callsrespond(to:schema:options:)andstreamResponse(to:schema:options:)and returnsGeneratedContent.jsonString. With noincludeSchemaInPromptargument, these resolve without ambiguity on the iOS 26 and iOS 27 SDKs, and the default istrueon both. So no#if compilerbranch is needed here. Schema requests go through the same session check, option mapping, availability check, busy lock, and layer 1 error mapping and context-overflow recovery as text requests.TypeScript: each call builds one request descriptor (native schema, chunk decoder, final parser), so
respondandstreamResponseeach have one path.ObjectSchemaisz.ZodObject.Hooks:
useLanguageModelanduseStreamingResponsestill return text only. Their state isresponse: string. A schema-aware hook needs a generic over the schema and typed state per call. That is a new hook, not an option, so it is not in this PR. The docs tell users to call the session methods directly.Example: a new "Structured" tab streams a recipe (nested object array, enum, bounded integer, string array). It shows the fields as they fill in, the latest raw snapshot, and the parsed value at the end.
What a partial snapshot looks like
I checked this at runtime before I designed the partial path. The first probe was a Swift script against the macOS 27 host model. The second was the iOS 27.0 simulator through the new Nitro method, with every native string recorded.
jsonStringis valid JSON of the object so far. It is never truncated JSON. On the simulator, 19 of 19 raw snapshots parsed withJSON.parse.name, role, address, skills, the model wroteaddressfirst. Key order also changes between snapshots."quantity": "2", then"200g". An enum or a new string starts as"". A new array element starts as{}or"".So JS runs
JSON.parseon each snapshot and does not need a partial-JSON parser.DeepPartial<T>widens every string type, including enums and literals, tostring, because""is a real partial value for an enum. If a snapshot is not JSON, the stream rejects withRESPONSE_VALIDATION_ERRORand does not blame the callback. The observed data says this does not happen.Raw simulator snapshots (excerpt):
Tradeoffs
""for a blank prompt, because an empty string is not a valid structured value.Blast Radius
schemado not change.KEYWORDSandUNSUPPORTED_HINTSare still exported from the package root. They now live ingeneration-schema.ts.SCHEMA_CREATION_ERRORmessage text changes from "Failed to create tool schema: ..." to "Failed to create schema: ...". The code does not change.schemaargument onrespondandstreamResponse, so apps must rebuild native code. Swift files were renamed and added, so runpod installagain.ToolSchemaBuilderis internal Swift. No public API refers to it.Verification
bun run test: 72 pass. New tests cover the schema sent to native for a nested object, array, enum, and default. They also cover a parsed value with a default applied, a Zod mismatch with the issues in details, non-JSON output, an unsupported schema and a non-object root rejected before native, a typed native failure (CONTEXT_EXCEEDED) through the schema path, streamed partials passed through unvalidated (including an""enum and a{}element), a final-value mismatch after streaming, and a non-JSON snapshot.bun run test:swift: all suites pass. New cases check the response root name, enum and array encoding, rejection of a flat tool-style map as a response schema, and theresponse.iderror path. The tool tests run against the renamedGenerationSchemaBuilderandToolContent.bun run typechecknow also checkspackage/tests(tsconfig.test.jsonwith@types/bun), and the session mock is typed from the Nitro spec.bun run lint,bun run format:check, and exampletscpass. I checked the overloads with a scratch file that was not committed.respond(p)andrespond(p, { temperature })arestring.respond(p, { schema })is the inferred object, and assigning it tostringis an error. The stream callback gets typed partial fields.respond(p, { schema: z.string() })is a type error."quantity": "2"grows to"200g"), then steps. The last frame reads "Validated after 31 snapshots", with the title filled in last. The dessert prompt gave a 6-ingredient, 6-step recipe withdifficulty: "hard",minutes: 180.debugger-evaluate:respondwith a nested contact schema (object, enum, bounded array) resolved to{"name":"Ana Ruiz","role":"designer","address":{"city":"Lisbon","country":"Portugal"},"skills":["Figma","typography"]}..refine(v => v.length > 200)field rejected withRESPONSE_VALIDATION_ERRORandissues: [{ code: "custom", path: ["fruit"] }].SCHEMA_CREATION_ERROR("Property 'response.id' uses unsupported schema feature: anyOf ...") before the request reached the model.CONTEXT_EXCEEDED, andwasContextResetwastrueafterwards. The layer 1 recovery works on the schema path.createTooltool was called with{"city":"Lisbon"}and the answer used its result. The migrated tool path works.Closes #22