Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions TESTING_COVERAGE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,36 @@
New-drive sync: WebSocket unit coverage verifies SUB and SYNC wait for a pending genesis acknowledgement, then resume on ResourceSaved. The Local DB-off rendering E2E exercises this ordering with real server persistence.

Cover repositioning: `cover-reposition.spec.ts` uploads a real image and verifies multiple pointer movements update its framing before release (native image dragging previously interrupted the gesture).

Template visibility: `settings-templates.spec.ts` toggles Hide templates through Settings, verifies the loaded New page hides templates across reload, and restores them when unchecked.

AI settings search: browser coverage checks API matches retain provider credentials but hide model, generative, and voice controls; titles matches show only the title toggle within AI.

Model option details: settings browser coverage checks input/output prices per million tokens and the OpenRouter added month inside an option, while preserving selection and dropdown width.

OpenRouter privacy: settings browser coverage filters out models absent from the mocked ZDR endpoint list. The BYOK voice/chat flow asserts provider.zdr is sent on its chat completion; voice audio remains outside this chat-only policy.

Model dropdown geometry: `ai-settings.spec.ts` checks that the open list matches its input width (regression: viewport-wide popover).

Speech settings: browser coverage selects a transcription model, persists the voice toggle across reload, hides the mic when disabled, and verifies the selected model in the personal-key audio request. The managed-credit model remains service-controlled.

AI settings: `ai-settings.spec.ts` exercises navigation from the chat agent menu,
provider controls and default model selection, agent/skill/MCP creation and reload
persistence, and settings search. Provider/MCP APIs are mocked. Agent radios expose
saved selection; voice BYOK regression still passes after the menu refactor.

Voice feedback: unit checks cover audio-level bounds and local-recognition cancellation; the personal-key Chromium flow checks mic scaling with simulated volume and interim word rendering. Real on-device transcription requires browser support and an installed language pack.

Voice with personal OpenRouter key: `ai-voice-byok.spec.ts` verifies the mic remains usable, direct audio requests carry the configured key, and no SaaS voice endpoint is called. Media and model responses are mocked.

Voice chat: `voiceTurn.test.ts` checks PCM WAV serialization and cancellation
without retries. `ai-live.spec.ts` runs the actual chat controls with mocked media
and OpenRouter responses: record, transcribe, hosted AI reply and speech playback,
without a personal API key. SaaS `ai::voice` validates input bounds and model pinning;
shared AI tests cover credit reservations and settlement. Real microphone/provider
quality remains unverified without the SaaS OpenRouter key. The earlier GPT-Live
lifecycle tests remain for its retained, separate backend implementation.

Plugin configuration: hook tests retain a release's validation schema when a save
receipt omits computed metadata, and clear it when the release or installation
changes. The plugin-install E2E checks invalid config after saving valid config.
Expand Down
2 changes: 1 addition & 1 deletion browser/data-browser/src/actions/appActions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ export const appActions: ActionDefinition[] = [
id: 'userSettings',
scope: 'app',
section: 'action',
label: () => 'User settings',
label: () => 'User',
helper: () => 'Open your agent / user settings.',
keywords: ['account', 'agent', 'profile'],
shortcut: shortcuts.userSettings,
Expand Down
68 changes: 68 additions & 0 deletions browser/data-browser/src/chunks/AI/AIConfigurationSections.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import { useState, type ReactNode } from 'react';
import { SettingsSection } from '@components/Settings';
import { Column } from '@components/Row';
import { AgentConfigTab, useAIAgentConfig } from './AgentConfig';
import { SkillsConfigTab } from './SkillsConfigTab';
import { MCPConfigTab } from './MCPConfigTab';

interface EditorProps {
actionPortalElement: HTMLElement | null;
onActionsVisibleChange: (visible: boolean) => void;
}

function EditorSection({
label,
children,
keywords,
}: {
label: string;
keywords?: string;
children: (props: EditorProps) => ReactNode;
}) {
const [actions, setActions] = useState<HTMLDivElement | null>(null);
const [editing, setEditing] = useState(false);

return (
<SettingsSection label={label} childSearchKeywords={keywords}>
<Column>
{children({
actionPortalElement: actions,
onActionsVisibleChange: setEditing,
})}
<div ref={setActions} hidden={!editing} />
</Column>
</SettingsSection>
);
}

export default function AIConfigurationSections() {
const { agents, defaultAgentId, setDefaultAgentId } = useAIAgentConfig();

return (
<Column gap='0'>
<EditorSection
label='Agents'
keywords='agent default system prompt temperature context tools model'
>
{props => (
<Column>
<p>Choose the default agent for new chats.</p>
<AgentConfigTab
{...props}
selectedAgent={
agents.find(agent => agent.id === defaultAgentId) ?? agents[0]
}
onSelectAgent={agent => setDefaultAgentId(agent.id)}
/>
</Column>
)}
</EditorSection>
<EditorSection label='Skills' keywords='skills content references'>
{props => <SkillsConfigTab {...props} />}
</EditorSection>
<EditorSection label='MCP' keywords='mcp server url transport headers'>
{props => <MCPConfigTab {...props} />}
</EditorSection>
</Column>
);
}
87 changes: 0 additions & 87 deletions browser/data-browser/src/chunks/AI/AISettingsDialog.tsx

This file was deleted.

17 changes: 10 additions & 7 deletions browser/data-browser/src/chunks/AI/AgentConfig.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,8 @@ export const AgentConfigTab = ({
</AgentsList>

<CreateButton onClick={handleCreateNewAgent}>
<FaPlus title='' /> Create New Agent
<FaPlus title='' />
<span>Create New Agent</span>
</CreateButton>
</Column>
)}
Expand Down Expand Up @@ -391,22 +392,24 @@ const AgentForm = ({ agent, isDefaultAgent, onChange }: AgentFormProps) => {
checked={agent.canReadAtomicData}
onChange={checked => handleChange('canReadAtomicData', checked)}
/>
Read
<span>Read</span>
</CheckboxLabel>
<CheckboxLabel>
<Checkbox
checked={agent.canWriteAtomicData}
onChange={checked => handleChange('canWriteAtomicData', checked)}
/>
Write
<span>Write</span>
</CheckboxLabel>
<CheckboxLabel>
<Checkbox
checked={agent.ragEnabled}
onChange={checked => handleChange('ragEnabled', checked)}
/>
RAG, Automatically provide context from your knowledge base based on
your prompt.
<span>
RAG, Automatically provide context from your knowledge base based on
your prompt.
</span>
</CheckboxLabel>
</StyledField>
<StyledField label='Tools' multiInput>
Expand Down Expand Up @@ -439,7 +442,7 @@ const AgentForm = ({ agent, isDefaultAgent, onChange }: AgentFormProps) => {
checked={agent.skillsEnabled ?? true}
onChange={checked => handleChange('skillsEnabled', checked)}
/>
Enable skills
<span>Enable skills</span>
</CheckboxLabel>
</StyledField>

Expand All @@ -451,7 +454,7 @@ const AgentForm = ({ agent, isDefaultAgent, onChange }: AgentFormProps) => {
handleChange('model', checked ? undefined : defaultChatModel);
}}
/>
Use Default Model
<span>Use Default Model</span>
</CheckboxLabel>
{agent.model && (
<ModelSelect
Expand Down
2 changes: 2 additions & 0 deletions browser/data-browser/src/chunks/AI/AgentConfigItem.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ export const AgentConfigItem: React.FC<AgentConfigItemProps> = ({
<input
disabled={providerDisabled}
type='radio'
checked={selected}
onChange={() => onSelect(agent)}
id={inputId}
name='agent'
value={agent.id}
Expand Down
30 changes: 30 additions & 0 deletions browser/data-browser/src/chunks/AI/ClientOnlyTransport.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// @wc-ignore-file
import {
generateText,
stepCountIs,
streamText,
type ChatTransport,
Expand All @@ -23,11 +24,13 @@ import { useSettings } from '@helpers/AppSettings';
import { shortenSubject } from '@helpers/subjectRefs';
import { getClassesOnDrive } from './atomicSchemaHelpers';
import { createHostedModel } from '@helpers/managed/ai';
import { hostedVoiceModel } from './hostedVoiceModel';

export type Modalities = 'text' | 'image';

export interface ClientOnlyTransportOptions {
openRouterAPIKey?: string;
openRouterZdr?: boolean;
ollamaURL?: string;
selectedAgent: AIAgent;
model: AIModelIdentifier;
Expand Down Expand Up @@ -61,6 +64,32 @@ export class ClientOnlyTransport implements ChatTransport<AtomicUIMessage> {
this.options = options;
}

/** Use the configured personal provider when present, otherwise SaaS credits. */
public async runLiveTask(
messages: AtomicUIMessage[],
abortSignal: AbortSignal,
) {
const prepared = await this.options.addContextToMessages(
trimToLastSummary(messages),
);
const result = await generateText({
model: this.options.openRouterAPIKey
? this.getModel(this.options.model)
: hostedVoiceModel(),
messages: await modelMessagesWithToolRecovery(prepared),
system: `${await this._prepareSystemPrompt(this.options.selectedAgent.systemPrompt)}\nYou are handling a delegated request from a live voice conversation. Use the latest user request and its corrections. Perform only requested actions. Return a brief factual result suitable for speech, including whether actions actually succeeded.`,
tools: this.options.tools,
abortSignal,
maxRetries: 0,
stopWhen: stepCountIs(8),
});

return (
result.text ||
'The task finished without a spoken result. Check the chat before repeating any action.'
);
}

public async sendMessages({
abortSignal,
...options
Expand Down Expand Up @@ -129,6 +158,7 @@ export class ClientOnlyTransport implements ChatTransport<AtomicUIMessage> {
apiKey: this.options.openRouterAPIKey,
compatibility: 'strict',
extraBody: {
...(this.options.openRouterZdr ? { provider: { zdr: true } } : {}),
modalities,
plugins: [{ id: 'context-compression' }],
},
Expand Down
Loading
Loading