Skip to content

feat: host MCP Apps views for tool calls - #1183

Draft
timonwong wants to merge 3 commits into
LodyAI:mainfrom
timonwong:feat/mcp-apps-host
Draft

timonwong wants to merge 3 commits into
LodyAI:mainfrom
timonwong:feat/mcp-apps-host

Conversation

@timonwong

@timonwong timonwong commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Related issue

Refs #1182

Problem / pressure

When an agent calls an MCP tool that declares an MCP Apps UI (SEP-1865 _meta.ui.resourceUri, a ui:// HTML resource), Codex desktop shows an interactive embedded view, such as "Opened NowledgeMem" with an explorable knowledge graph. Lody shows only Ran mcp.codex_apps.nowledgemem.explore_graph. The data a host needs is dropped twice: the Codex adapter never forwards the app resource, and history-apply keeps neither _meta nor rawOutput for MCP calls.

This PR exceeds the 1000-line community limit and stays a draft until a maintainer decides on #1182, whether to assign it or to split the work differently.

Summary

The PR has three commits, kept separate for review:

  1. feat: route MCP Apps requests to the owning agent, the data plane:
    • Persists only a bounded toolCall.mcpApp descriptor (server, tool, ui:// resourceUri, optional appName and display mode). It is extracted from _meta.lody.mcpApp and survives updates.
    • Advertises the mcpApps client capability from the real session client only.
    • Adds the session/mcp-app Machine RPC on the shared lane with load, resource_read and tool_call ops, plus the renderer facade and useMcpAppHost.
    • Errors are typed: MCP_APP_UNAVAILABLE, MCP_APP_ACCESS_DENIED, MCP_APP_AGENT_ERROR and MCP_APP_RESPONSE_TOO_LARGE. The response cap is 16 MiB − 64 KiB, under the 16 MiB local IPC body limit; a real app's HTML is about 7.1 MB before JSON escaping.
    • When no agent is live, the RPC reports unavailable and never boots one.
    • Bumps acp-extension-core and acp-extension-codex.
  2. feat(components): render MCP Apps views for tool calls:
    • A completed call renders "Opened {app}". The header carries the hover-only disclosure chevron and a full-screen button.
    • The app runs in an opaque-origin, credentialless frame with no preload (see View sandbox). The frame loads the fixed lody-mcp-app://sandbox/ proxy page, which receives the app HTML through the SEP-1865 sandbox-proxy handshake.
    • A per-app CSP is built from _meta.ui.csp, deny-by-default. connectDomains accepts https:// and wss:// origins; other directives accept only https://.
    • The JSON-RPC bridge handles initialize, tool-input and tool-result, tools/call, resources/read, open-link (http/https), size and display mode, host-context changes, and teardown.
    • Full screen moves the same frame into a dialog.
  3. test(e2e): cover MCP Apps view hosting: a synthetic ACP agent and app (@LODY-MCPAPP-001). It also checks that the view's in-memory storage works and never reaches Lody's user-data directory.

Companion PRs: LodyAI/acp-extension-core#17 (contract) and LodyAI/acp-extension-codex#62 (agent side). Their commits currently exist only on the timonwong forks, so CI submodule checkout from the LodyAI remotes will fail until those land and the pointers move. The codex pointer here is the same two commits cherry-picked onto 8689ac3, the commit main pins. That keeps pnpm-lock.yaml unchanged apart from the new ext-apps devDependency; #62 carries the port onto current acp-extension-codex main.

View sandbox

MCP App views run in an opaque origin: sandbox="allow-scripts" plus credentialless, served by the lody-mcp-app://sandbox/ proxy.

  • Before the proxy writes the app document, it installs in-memory localStorage, sessionStorage, and document.cookie.
  • Views that use them work without SecurityError, and nothing reaches disk.
  • State lasts for one view load. IndexedDB and Cache Storage stay unavailable.

A per-app origin with allow-same-origin was considered and rejected. Chromium keeps each load's credentialless storage partition in Lody's LevelDB permanently, and Electron's clearData / clearStorageData cannot remove those entries. Dropping credentialless would instead expose Lody's default-session cookies to views. Persistent per-app storage would need a dedicated in-memory session (<webview> partition), which is left for a follow-up.

_meta.ui.domain is not honored.

Visual explanation

sequenceDiagram
  participant Codex as Codex App Server
  participant Adapter as acp-extension-codex
  participant Daemon as Lody CLI daemon
  participant UI as Renderer card
  participant Frame as lody-mcp-app sandbox frame
  Codex->>Adapter: item mcpToolCall (mcpAppUi.resourceUri)
  Adapter->>Daemon: tool_call _meta.lody.mcpApp
  Daemon->>Daemon: history stores descriptor only
  UI->>Daemon: session/mcp-app load + resource_read
  Daemon->>Adapter: _lody/mcp_apps/load, /resource/read
  Adapter->>Codex: mcpServer/resource/read (threadId, originCallId)
  UI->>Frame: sandbox-resource-ready {html + per-app CSP}
  Frame->>UI: ui/initialize, then initialized
  UI->>Frame: tool-input, tool-result
  Frame->>UI: tools/call explore_graph
  UI->>Daemon: session/mcp-app tool_call
  Daemon->>Adapter: _lody/mcp_apps/tool/call
  Adapter->>Codex: mcpServer/tool/call nowledgemem.explore_graph
Loading
packages/components/src/components/ai-gui/mcp-app/
  mcp-app-tool-call-card.tsx   card, header, load state, frame placement, full-screen dialog
  mcp-app-bridge.ts            host side of the SEP-1865 postMessage JSON-RPC
  mcp-app-document.ts          HTML selection, per-app CSP builder, domain sanitizer
  mcp-app-host.ts              context + facade adapter
apps/electron/src/main/services/mcp-app-sandbox.ts   fixed proxy page, in-memory storage shim, response CSP upper bound

Before / after

Before After
Ran mcp.codex_apps.nowledgemem.explore_graph Inline MCP App view
No way to reach the app from Lody Card header with disclosure chevron and full-screen button

The screenshots come from a real desktop build. The app and its data are synthetic; Lody's styles are unmodified.

Test plan

  • A real end-to-end run on macOS arm64 used an isolated E2E instance, the real acp-extension-codex build, the machine's Codex ChatGPT login, and the real NowledgeMem app (codex_apps). It completed the handshake, rendered the graph, and completed app-initiated tools/call round trips (bare explore_graph forwarded as nowledgemem.explore_graph). It produced no CSP violations, page errors or failed requests. No screenshots are attached because the data is private.
  • The real run caught three interop bugs that mocks missed, now fixed and covered by tests:
    • an incomplete hostContext.toolInfo rejected by the official @modelcontextprotocol/ext-apps schema;
    • bare versus connector-namespaced tool names on codex_apps;
    • JSON-escaped HTML over the first 8 MiB cap.
  • components tests validate the initialize result and host-context-changed against the real ext-apps 2.0.0 schema, as a devDependency.
  • Not covered: Linux and Windows desktop runs, and remote-machine (Loro Streams) traffic.

Context handoff

Original user prompt

Show original prompt
[@f9e5444a-lody.png](file:///home/vscode/.lody/repos/github---timonwong---lody/worktrees/d109fc5f-b4d0-42d4-9aed-249d9594e4a2/.lody/attachments/f9e5444a-lody.png)[@1ad34b12-codex.png](file:///home/vscode/.lody/repos/github---timonwong---lody/worktrees/d109fc5f-b4d0-42d4-9aed-249d9594e4a2/.lody/attachments/1ad34b12-codex.png)第一张图是 lody,第二张图是 codex。
你可以看到 codex 支持把 渲染为 "Opened NowledgeMem",里面就是直接访问 nowledgemem 的知识图谱。

Lody 这边,只能显示一个 "Ran mcp.codex_apps.nowledgemem.explore_graph"

你能否让 Lody 这里的行为跟 Codex 一样?

对了,如果你要执行测试,最好是通过 lody mcp 将任务分发到 timon-m3mac 上,因为你自己的资源(磁盘,cpu,内存)优先,cpu 还被 CFS 死死限制住的

Sharing refusal (verbatim)

不分享,但是把我的 prompt 和关键信息都放进去

Shared conversation

Status: user-declined
Reason: The author chose not to publish the authoring conversation and asked to include the original prompt and its key decisions instead, listed below.

Key decisions from the authoring conversation

  • Delivery: three changes (core contract, Codex adapter, Lody), reviewed on forks first, each opened as a draft.
  • Data strategy: lazy loading through the live agent. Only a small descriptor enters history, never the HTML or results.
  • Interaction scope: full. Apps may call tools/call and resources/read on their originating server, limited to tools visible to apps, and on codex_apps to the originating connector.
  • UI review against real screenshots, decided by the author:
    • the full-screen button moved from the frame overlay into the header;
    • a disclosure chevron was added, hover-only per ai-gui/AGENTS.md;
    • the card's left edge aligns with the text column;
    • the "App unavailable" state stays as is.
  • Validation had to include a real NowledgeMem run, not only mocks.
  • View sandbox: keep the opaque origin and give views in-memory localStorage, sessionStorage, and cookies from the proxy. A per-app allow-same-origin origin was tried and dropped because its credentialless storage persisted in Lody's LevelDB beyond Electron's clearing APIs; _meta.ui.domain stays unsupported.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

@timonwong, this pull request needs updates before review.

It is marked status:needs-pr-attention. Address the findings below by 2026-10-08 04:50:09 UTC. The label and this comment are removed automatically after the PR passes validation.

If the PR remains invalid for 7 days, it will be closed and marked status:pr-policy-expired. Continue afterward by opening a new pull request with the current template.

Policy findings
PR does not meet Lody contribution requirements:

- PR changes 3908 lines; community PRs over 1000 lines require a maintainer assignment on the linked Issue before review.

See `CONTRIBUTING.md` and `.github/PULL_REQUEST_TEMPLATE.md`.

@timonwong
timonwong force-pushed the feat/mcp-apps-host branch 2 times, most recently from 9013178 to 3076a58 Compare October 1, 2026 06:35
@Leeeon233 Leeeon233 self-assigned this Oct 1, 2026
@timonwong
timonwong force-pushed the feat/mcp-apps-host branch 3 times, most recently from 963e337 to 4b8c2b3 Compare October 2, 2026 11:26
Persist a small descriptor for tool calls that open an MCP App and add
the session/mcp-app Machine RPC, so a renderer can load the originating
input and result and proxy the view's resource reads and tool calls
through the live agent. Bump the Codex adapter and core contract.

Model: claude-opus-5-5
Tool calls carrying an MCP Apps descriptor render as "Opened {app}" with
the app in a sandboxed frame. srcdoc/blob/data frames inherit the
renderer CSP and cannot run inline scripts, so the frame loads a fixed
lody-mcp-app:// sandbox proxy that receives the app HTML over the
SEP-1865 proxy handshake. The view keeps an opaque, credentialless
origin, and the proxy gives it in-memory localStorage, sessionStorage
and cookies: allow-same-origin would leave credentialless storage in
the default session's LevelDB, beyond Electron's clearing APIs. The
bridge proxies app tool calls and resource reads to the owning agent.

Model: claude-opus-5-5

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants