Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

65 changes: 61 additions & 4 deletions docs/docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,16 @@ Generates a complete response from the language model.

```typescript
respond(prompt: string, options?: GenerationOptions): Promise<string>
respond<S extends ObjectSchema>(
prompt: string,
options: StructuredGenerationOptions<S>,
): Promise<z.infer<S>>
```

**Parameters**:
- `prompt: string` - The user's message
- `options?: GenerationOptions` - Sampling, temperature, and length limits for this request. See [`GenerationOptions`](#generationoptions)
- `options.schema?: ObjectSchema` - A Zod object schema. When set, the response is constrained to the schema and the promise resolves with the parsed value. See [Structured output](#structured-output)

```typescript
const answer = await session.respond('Name a color', {
Expand All @@ -51,12 +56,39 @@ streamResponse(
onChunk: (responseSoFar: string) => void,
options?: GenerationOptions,
): Promise<string>
streamResponse<S extends ObjectSchema>(
prompt: string,
onChunk: (partial: DeepPartial<z.input<S>>) => void,
options: StructuredGenerationOptions<S>,
): Promise<z.infer<S>>
```

**Parameters**:
- `prompt: string` - The user's message
- `onChunk: (responseSoFar: string) => void` - Called with the full streamed response so far
- `onChunk` - Called with the full streamed response so far. With `options.schema`, called with the partial object generated so far
- `options?: GenerationOptions` - Sampling, temperature, and length limits for this request. See [`GenerationOptions`](#generationoptions)
- `options.schema?: ObjectSchema` - A Zod object schema. See [Structured output](#structured-output)

#### Structured output

```typescript
const Contact = z.object({
name: z.string(),
role: z.enum(['engineer', 'designer', 'manager']),
address: z.object({ city: z.string(), country: z.string() }),
skills: z.array(z.string()).max(4),
});

const contact = await session.respond(text, { schema: Contact });

const final = await session.streamResponse(text, setDraft, { schema: Contact });
```

- The schema must have `z.object` at the root. It follows the same rules as tool arguments. See [`createTool`](#createtooldefinition). An unsupported schema rejects with `SCHEMA_CREATION_ERROR` before the request reaches the model. Error paths start with `response`, for example `response.address.city`.
- The final value is parsed with the schema, so defaults and transforms apply. If it does not parse, the request rejects with `RESPONSE_VALIDATION_ERROR`. `details.issues` holds the Zod issues and `details.response` holds the raw value. This can happen with checks the model cannot see, such as `.refine()`.
- Each streamed snapshot is the complete object generated so far, as valid JSON. Snapshots are not validated. Fields appear in the order the model writes them, which is not always the schema order. A string can be incomplete. An enum value can be `''` before the model picks one. An array can hold an empty placeholder element (`{}` or `''`) before the model fills it in. Consecutive snapshots can be equal.
- Context overflow and the other generation errors have the same codes as for text responses.
- `useLanguageModel` and `useStreamingResponse` return text only.

#### `tokenCount(prompt)`

Expand Down Expand Up @@ -207,6 +239,30 @@ type ToolCallingMode = 'allowed' | 'required' | 'disallowed';
- `samplingMode` - How the model picks each token. `greedy` always picks the most likely token. `randomTopK` samples from the `top` most likely tokens (`top` is an integer of 1 or more). `randomProbabilityThreshold` samples from the smallest set of tokens whose probabilities add up to `probabilityThreshold` (greater than 0, at most 1). `seed` is an optional non-negative integer that makes random sampling repeatable.
- `toolCallingMode` - Whether the model may (`allowed`), must (`required`), or must not (`disallowed`) call tools. iOS 27 and later only. iOS 26 ignores it. With `required`, the model calls a tool at every step. On the iOS 27.0 simulator it kept calling the tool and the request did not end normally, so prefer `allowed` unless your tool loop has an exit.

### `StructuredGenerationOptions`

```typescript
type ObjectSchema = z.ZodObject;

interface StructuredGenerationOptions<S extends ObjectSchema> extends GenerationOptions {
schema: S;
}
```

### `DeepPartial`

The type of a streamed partial object. Every field is optional, and every string type (including enums and literals) widens to `string`, because a partial value can be incomplete.

```typescript
type DeepPartial<T> = T extends string
? string
: T extends readonly (infer Element)[]
? DeepPartial<Element>[]
: T extends object
? { [Key in keyof T]?: DeepPartial<T[Key]> }
: T;
```

### `LanguageModelSessionConfig`

```typescript
Expand Down Expand Up @@ -247,9 +303,10 @@ class AppleAIError extends Error {
- `SESSION_NOT_INITIALIZED` - Session is not ready
- `TOOL_CALL_ERROR` - Tool call failed
- `TOOL_EXECUTION_ERROR` - Tool execution failed
- `SCHEMA_CREATION_ERROR` - Failed to create tool schema
- `SCHEMA_CREATION_ERROR` - A tool or response schema uses a feature the model does not support
- `ARGUMENT_PARSING_ERROR` - Failed to parse tool arguments
- `RESPONSE_PARSING_ERROR` - Failed to parse tool response
- `RESPONSE_VALIDATION_ERROR` - A structured response does not match its schema. `details.issues` holds the Zod issues
- `UNKNOWN_TOOL_ERROR` - Unknown tool referenced
- `SESSION_STREAMING_ERROR` - Streaming failed
- `SESSION_RESPONSE_ERROR` - Response failed for a reason with no specific code
Expand Down Expand Up @@ -293,15 +350,15 @@ The generation fields apply to that stream. The callbacks are not sent to the mo
Helper function to create type-safe tools with Zod schema validation.

```typescript
function createTool<T extends ZodObjectSchema>(definition: {
function createTool<T extends ObjectSchema>(definition: {
name: string;
description: string;
arguments: T;
handler: (params: z.infer<T>) => Promise<AnyMap>;
}): ToolDefinition
```

The Zod schema you pass is the exact contract the model sees. Supported schema features:
The Zod schema you pass is the exact contract the model sees. The same rules apply to the `schema` of a structured response. Supported schema features:

- `z.string()`, `z.number()`, `z.number().int()`, `z.boolean()`
- `z.object()` and `z.array()`, nested to any depth
Expand Down
5 changes: 5 additions & 0 deletions example/app/_layout.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ export default function RootLayout() {
<NativeTabs.Trigger.Label>Hook</NativeTabs.Trigger.Label>
</NativeTabs.Trigger>

<NativeTabs.Trigger name="structured-demo">
<NativeTabs.Trigger.Icon sf="list.bullet.rectangle" />
<NativeTabs.Trigger.Label>Structured</NativeTabs.Trigger.Label>
</NativeTabs.Trigger>

<NativeTabs.Trigger name="+not-found" hidden />
</NativeTabs>
</ThemeProvider>
Expand Down
225 changes: 225 additions & 0 deletions example/app/structured-demo.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
import { useCallback, useState } from 'react'
import { ActivityIndicator, ScrollView, StyleSheet, TouchableOpacity } from 'react-native'
import {
type AppleAIError,
type DeepPartial,
isAppleAIError,
LanguageModelSession,
parseNativeError,
} from 'react-native-foundation-models'
import { useSafeAreaInsets } from 'react-native-safe-area-context'
import { z } from 'zod'
import { Text, useThemeColor, View } from '@/components/Themed'

const Recipe = z.object({
title: z.string(),
difficulty: z.enum(['easy', 'medium', 'hard']),
minutes: z.number().int().min(1).max(240).describe('Total time in minutes'),
ingredients: z
.array(z.object({ name: z.string(), quantity: z.string() }))
.min(2)
.max(8),
steps: z.array(z.string()).min(1).max(6),
})

type Recipe = z.infer<typeof Recipe>

const PROMPTS = [
'A quick pasta recipe',
'A vegan breakfast bowl',
'A hard French dessert',
]

const session = new LanguageModelSession({
instructions: 'You write short, practical recipes.',
})

export default function StructuredDemoScreen() {
const insets = useSafeAreaInsets()
const mutedColor = useThemeColor({}, 'muted')
const borderColor = useThemeColor({}, 'border')
const cardColor = useThemeColor({}, 'card')
const tintColor = useThemeColor({}, 'tint')
const dangerColor = useThemeColor({}, 'danger')

const [partial, setPartial] = useState<DeepPartial<Recipe>>({})
const [snapshots, setSnapshots] = useState(0)
const [recipe, setRecipe] = useState<Recipe>()
const [isStreaming, setIsStreaming] = useState(false)
const [error, setError] = useState<AppleAIError>()

const generate = useCallback(async (prompt: string) => {
setPartial({})
setSnapshots(0)
setRecipe(undefined)
setError(undefined)
setIsStreaming(true)
try {
const result = await session.streamResponse(
prompt,
next => {
setPartial(next)
setSnapshots(count => count + 1)
},
{ schema: Recipe },
)
setRecipe(result)
} catch (err) {
setError(isAppleAIError(err) ? err : parseNativeError(err))
} finally {
setIsStreaming(false)
}
}, [])

const shown = recipe ?? partial
const status = error
? error.code
: recipe
? `Validated after ${snapshots} snapshots`
: isStreaming
? `Streaming, ${snapshots} snapshots`
: 'Pick a prompt'

return (
<ScrollView
contentContainerStyle={[styles.content, { paddingTop: insets.top + 12 }]}
testID="structured-scroll"
>
<Text style={[styles.eyebrow, { color: mutedColor }]}>ZOD SCHEMA OUTPUT</Text>
<Text style={styles.title}>Structured</Text>

<View style={styles.prompts}>
{PROMPTS.map(prompt => (
<TouchableOpacity
key={prompt}
accessibilityRole="button"
disabled={isStreaming}
onPress={() => generate(prompt)}
style={[
styles.chip,
{ borderColor: tintColor, opacity: isStreaming ? 0.4 : 1 },
]}
>
<Text style={{ color: tintColor }}>{prompt}</Text>
</TouchableOpacity>
))}
</View>

<View style={[styles.card, { borderColor, backgroundColor: cardColor }]}>
<View style={[styles.statusRow, { backgroundColor: 'transparent' }]}>
<Text
testID="structured-status"
style={[styles.cardLabel, { color: error ? dangerColor : mutedColor }]}
>
{status.toUpperCase()}
</Text>
{isStreaming ? <ActivityIndicator size="small" /> : null}
</View>

{error ? <Text style={{ color: dangerColor }}>{error.message}</Text> : null}

<Text style={styles.recipeTitle}>{shown.title || ' '}</Text>
<Field label="Difficulty" value={shown.difficulty} mutedColor={mutedColor} />
<Field
label="Minutes"
value={shown.minutes === undefined ? undefined : String(shown.minutes)}
mutedColor={mutedColor}
/>

<Text style={[styles.cardLabel, { color: mutedColor }]}>INGREDIENTS</Text>
<Text style={styles.list}>
{(shown.ingredients ?? [])
.map(({ quantity = '', name = '' }) => `• ${quantity} ${name}`.trimEnd())
.join('\n')}
</Text>

<Text style={[styles.cardLabel, { color: mutedColor }]}>STEPS</Text>
<Text style={styles.list}>
{(shown.steps ?? []).map((step, index) => `${index + 1}. ${step}`).join('\n')}
</Text>
</View>

<View style={[styles.card, { borderColor, backgroundColor: cardColor }]}>
<Text style={[styles.cardLabel, { color: mutedColor }]}>
{recipe ? 'PARSED VALUE' : 'LATEST SNAPSHOT'}
</Text>
<Text testID="structured-json" style={styles.json}>
{JSON.stringify(shown, null, 2)}
</Text>
</View>
</ScrollView>
)
}

function Field({
label,
value,
mutedColor,
}: {
label: string
value?: string
mutedColor: string
}) {
return (
<View style={[styles.statusRow, { backgroundColor: 'transparent' }]}>
<Text style={{ color: mutedColor }}>{label}</Text>
<Text>{value || '…'}</Text>
</View>
)
}

const styles = StyleSheet.create({
content: {
paddingHorizontal: 20,
paddingBottom: 120,
gap: 16,
},
eyebrow: {
fontSize: 13,
fontWeight: '600',
letterSpacing: 0.5,
},
title: {
fontSize: 34,
fontWeight: '700',
},
prompts: {
flexDirection: 'row',
flexWrap: 'wrap',
gap: 8,
backgroundColor: 'transparent',
},
chip: {
borderWidth: 1,
borderRadius: 16,
paddingHorizontal: 12,
paddingVertical: 6,
},
card: {
borderWidth: StyleSheet.hairlineWidth,
borderRadius: 16,
padding: 16,
gap: 8,
},
cardLabel: {
fontSize: 12,
fontWeight: '600',
letterSpacing: 0.5,
},
statusRow: {
flexDirection: 'row',
justifyContent: 'space-between',
alignItems: 'center',
},
recipeTitle: {
fontSize: 22,
fontWeight: '600',
},
list: {
lineHeight: 24,
},
json: {
fontFamily: 'Menlo',
fontSize: 12,
},
})
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"scripts": {
"test": "bun --cwd package test",
"test:swift": "bun --cwd package test:swift",
"typecheck": "bun tsc -p ./package --noEmit",
"typecheck": "bun tsc -p ./package --noEmit && bun tsc -p ./package/tsconfig.test.json",
"lint": "biome lint .",
"format": "biome format --write .",
"format:check": "biome check --formatter-enabled=true --linter-enabled=false --assist-enabled=false .",
Expand Down
Loading
Loading