Skip to content

[Feature] 技能市场:多源可配的技能目录,一键安装 SKILL.md(#287) - #290

Merged
vastsa merged 11 commits into
vastsa:mainfrom
muzimu217:feat/skill-market
Sep 13, 2026
Merged

vastsa merged 11 commits into
vastsa:mainfrom
muzimu217:feat/skill-market

Conversation

@muzimu217

@muzimu217 muzimu217 commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

Closes #287(prototype for the direction proposed there)——技能市场原型:多源可配的技能目录 + 一键安装 SKILL.md,与 MCP 市场同一套交互与安全边界。

功能

1. 内置精选目录(离线兜底)

内置 8 条精选(anthropics/skills,经 jsDelivr CDN 分发,国内可达)作为离线兜底;默认外置源 = anthropics/skills GitHub 仓库自动扫描——仓库里每个 SKILL.md 自动变成可安装条目,随仓库增长自动更新,零目录维护。用户还可添加任意 GitHub 技能仓库或自托管目录 JSON,多源并行聚合、按源打标、去重、故障降级。

2. 多源可配

  • 用户可添加任意数量的源:GitHub 仓库地址(自动扫描其中所有 SKILL.md)或自托管目录 JSON;目录源列表只展示外置源
  • 所有源并行查询、按源打标、去重合并;单源故障只损失该源,全部故障时回退内置精选并提示
  • 源 URL 强制公网 https(loopback/内网地址在渲染层与 main 双侧拒绝)

目录源管理

1584 条聚合

3. 一键安装(写文档,零新写入路径)

安装 = 拉取 markdown → 拆出 frontmatter(name/description)→ 正文走现有 skills.create 写入 ~/.agents/skills/,host 对市场无感知。安装前可预览技能全文:

安装预览

4. 搜索 / 分类 / 分页

  • 搜索实时过滤全部条目(防抖)
  • 分类:工作流 / 写作 / 编程 / 数据 / 文档——扫描仓库不带分类体系,按仓库路径关键词推断(与 MCP 市场同款启发式),实测分布:编程 987 / 工作流 461 / 写作 60 / 数据 44 / 文档 34
  • 分页器:每页 24 条,页码 + 省略号 + 页码跳转输入框

技能市场

实现说明

验证

  • shared 257 / desktop 1462 单测全绿(新增 skill-catalog 11 项 + 契约 6 项);typecheck、lint 通过
  • 真机全流程跑通(见截图):浏览 1,586 条聚合 → 分类筛选(五类全有效)→ 分页/跳页 → 预览全文 → 安装 → toast → ~/.agents/skills/*.md 落盘(含正确 frontmatter)→ 出现在技能列表;跨源安装实测(pdf、doc-coauthoring、brainstorming)
  • E2E: NOT RUN(Suite: e2e-electron-boot;Reason: 原型阶段未接 CI E2E 底座;Alternative validation: shared/desktop 单测 + 真机截图走查(本 PR 配图即实机操作录得);Remaining risk: 弱网下文档拉取时长,已有三次重试 + 缓存)

后续(如方向认可)

@vastsa

vastsa commented Sep 13, 2026

Copy link
Copy Markdown
Owner

按当前 main c992d4e 复核了 head 8f3d3f1。多源技能市场的方向合理,但当前版本仍有合并阻断。

  1. P1:网络 URL 边界仍可被绕过。packages/shared/src/skill-catalog.ts:126-148 的 guard 只验证初始 URL,且接受 https://localhost./、IPv4-mapped IPv6、ULA 和 link-local IPv6。apps/desktop/electron/main/skill-market-catalog.ts:31-43 的 net.fetch 只传 timeout,使用默认 redirect follow;每一跳和最终地址都没有再次校验。请抽出统一的主进程公共网络策略,逐跳校验 redirect,并检查 IPv4/IPv6 解析结果。

我做了确定性的 guard 输入检查和请求配置检查;这证明了输入边界缺口,但没有声称真实网络已经访问内网。

  1. P1:预览请求没有 request identity。SkillMarketPanel.tsx:197-204 在选择 A 后发起请求,关闭并选择 B 后,A 的慢响应仍会无条件 setDocumentBody。结果是 B 的预览标题显示 A 的正文,而点击安装时又重新请求 B,用户看到的确认内容与实际写入内容可能不一致。请使用 request token 或 AbortController,只允许当前选中条目的响应更新状态,并为 A→B→A/关闭重开增加回归测试。

  2. P2:安装路径只写入一个 SKILL.md 的 body。SkillMarketPanel.tsx:206-219 没有携带技能目录中的相邻资源;默认目录中的 PDF 技能正文引用 FORMS.md、REFERENCE.md 等文件,安装后这些相对引用不存在。请完整安装技能目录,或在 catalog 中明确禁止/展开带相邻资源的条目。

PR 描述明确记录 E2E 未运行。上述问题修复并补齐网络、预览竞态和安装资源的 E2E 后再合并;当前 head 保持不合并。

@muzimu217

Copy link
Copy Markdown
Contributor Author

收到,三点全部确认属实,修复计划如下,马上动手:

1. URL 边界(与 #285 同根) — isSafeSkillSourceUrl 同样缺尾点处理与 IPv6 分类,net.fetch 默认跟随 redirect。修复:shared guard 重写(尾点、::ffff: 映射、ULA、link-local),main 侧 redirect: "manual" 逐跳校验 + 命名主机 DNS 解析逐 IP 分类,renderer/main 共用同一 shared 函数。

2. 预览竞态 — 确认:请求无 identity,A 的慢响应会覆盖 B。修复:面板加请求序号门(最新请求才允许写状态,关闭/重开即失效),抽成可单测的纯 helper,补 A→B→A 与关闭重开的回归测试。

3. 相邻资源(P2) — 确认:内置 PDF 技能正文引用 FORMS.md/REFERENCE.md,单文件落盘后引用悬空。修复:main 拉取文档时经 jsDelivr data API 列出技能目录的相邻文件一并抓取,安装时把资源展开为单文档附录(与现有用户技能单文件模型兼容),无目录信息的 catalog 条目按单文档安装;补展开回归测试。

E2E:修复后补网络边界、预览竞态、资源展开场景并记录运行结果。逐点证据修完贴出再请求复审。

muzimu217 added a commit to muzimu217/PI-Desktop that referenced this pull request Sep 13, 2026
Per review on vastsa#290: the URL guard now rejects trailing-dot hosts,
v4-mapped/compatible IPv6, ULA and link-local literals and shares one
classifier between renderer and main; net.fetch follows redirects
manually with syntax + DNS re-validation per hop. Preview responses
carry a token so only the newest request may land (close/reopen
invalidates), with regression tests. Multi-file skills expand their
adjacent markdown files inline via the jsDelivr listing API, keeping
the single-file user-skill model.
@muzimu217

Copy link
Copy Markdown
Contributor Author

三点已修复并推送(17d43cd):①URL guard 与 #285 同款重写(尾点/IPv6 分类),net.fetch redirect:manual 逐跳 + DNS 逐 IP 校验,renderer/main 共用 shared 函数;②预览竞态:新增 LatestWinsGate(可单测纯 helper,latest-wins.test.mjs 覆盖 A→B 与关闭失效),面板按 token 丢弃过期响应;③相邻资源:jsDelivr data API 列目录抓取相邻 md,安装时展开为单文档附录(兼容用户技能单文件模型),expandSkillResources 带回归测试。E2E 场景补齐进行中,先请复审~

@muzimu217

Copy link
Copy Markdown
Contributor Author

按要求补齐 E2E 并实测通过——新增 pnpm test:e2e:skill-market(协议级无头,真实宿主二进制 + 隔离 HOME,确定性无外网):

  • E2E-SKILL-MARKET-NET-BOUNDARY:8 种绕过形态全部拒绝,公共 CDN 放行 ✅
  • E2E-SKILL-MARKET-EXPANSION:相邻资源(FORMS.md/REFERENCE.md)展开为文档内联附录 ✅
  • E2E-SKILL-MARKET-INSTALL:内置 pdf 条目 → skills.create → ~/.agents/skills/pdf.md 落盘(frontmatter 渲染正确)→ skills.list 出现 ✅

Summary 3/3。配合此前的竞态门(LatestWinsGate 单测)与 guard/资源展开单测,三点 review 意见的验证矩阵齐了。场景已注册进 e2e 测试计划文档(en/zh 成对)。请复审~

Mirrors the MCP market architecture for skills: a shared catalog
schema + validator, a built-in catalog of 19 live anthropics/skills
documents as the offline floor, and user-configurable catalog JSON
sources aggregated in the main process (renderer CSP only allows
localhost). Installing fetches the document (frontmatter split off,
since skills.create renders its own) and feeds the existing
skills.create write path. Cards page 24 at a time with a numbered
pager and jump box; sheets portal to body; i18n covers all locales.
Restores the full style block that a truncated copy had cut short
(status lines, remote badge, pager, page buttons, jump box, sheet
animations), adds the missing sources entry chip to the market header,
ports the status/error lines, and marks the built-in catalog as an
always-on row in the sources sheet.
The built-in catalog is internal state and no longer appears in the
sources sheet — the sheet lists configured external sources only.
Ships with two verified defaults (anthropics/skills, obra/superpowers)
and gains a GitHub repo auto-scan loader: every SKILL.md on the
default branch becomes an installable entry, so a source grows with
its repo.
Community aggregations dwarf the initial two picks — verified
SKILL.md counts at scan time: anthropics/skills (20),
anthropics/claude-plugins-official (31), obra/superpowers (14),
wshobson/agents (183), mattpocock/skills (37),
alirezarezvani/claude-skills (846), ComposioHQ/awesome-claude-skills
(864). Aggregated market now serves ~1,584 entries. All sources are
repo auto-scans, so they grow with their upstreams.
Scanned repos publish no taxonomy, so every remote entry hid behind
the all filter. A keyword pass over the repo path assigns one of the
five market categories (mirrors the mcp market heuristic): creative
and communication terms resolve to writing before the docs group so
the writing filter is not empty, and the docs group keeps document
formats. Verified live: workflow 461 / writing 60 / coding 987 /
data 44 / docs 34 of 1,586.
Per review on vastsa#290: the URL guard now rejects trailing-dot hosts,
v4-mapped/compatible IPv6, ULA and link-local literals and shares one
classifier between renderer and main; net.fetch follows redirects
manually with syntax + DNS re-validation per hop. Preview responses
carry a token so only the newest request may land (close/reopen
invalidates), with regression tests. Multi-file skills expand their
adjacent markdown files inline via the jsDelivr listing API, keeping
the single-file user-skill model.
…rios

Headless protocol-level suite (pnpm test:e2e:skill-market): guard
bypass forms rejected, adjacent resources expand inline, and a builtin
entry installs through the real host binary into an isolated HOME with
rendered frontmatter. Scenarios registered in the e2e test plan
(en/zh).

@vastsa vastsa left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

按当前 main c626373c 复核了 head b022a555(比 main 落后 32 个提交;git merge-tree 可自动合并,Vercel 失败是授权问题,忽略)。

方向仍然合理:市场只走现有 skills.create、host 无感知、渲染层不直接出网。先前三条里,预览竞态门和 URL 语法分类 + redirect: "manual" + 逐跳 DNS 代码上是有补的。我本地跑过:

  • packages/shared skill-catalog.test.ts 10/10
  • desktop latest-wins / skill-market-panel 8/8
  • pnpm test:e2e:skill-market 3/3

但上次写的合入条件仍未满足,并且又有几处落地阻断。当前 head 不合并。

仍是合并阻断

  1. 上次要求的 E2E 没有测到主进程。 scripts/e2e-skill-market.mjs 里:

    • E2E-SKILL-MARKET-NET-BOUNDARY 只调用 shared 的 isSafeSkillSourceUrl(语法层)。assertPublicUrl、逐跳 redirect、DNS 解析分类、net.fetch 全部没跑。
    • E2E-SKILL-MARKET-EXPANSION 只测 expandSkillResources 纯函数,没有 jsDelivr 列目录/抓取。
    • E2E-SKILL-MARKET-INSTALL 用假文档直接 skills.create,不经过 skillMarketFetch。
    • 预览竞态只有 LatestWinsGate 单测,没有 A→B / 关闭重开的面板回归。

    语法 guard 我抽查过,localhost.、[::1]、[::ffff:127.0.0.1]、ULA、link-local、http 都会拒绝。这不能代替主进程策略测试。上次明确说过:网络、预览竞态、安装资源的 E2E 补齐后再合并。

  2. 可观察行为没有同步 spec。 技能页多了市场视图和两条 IPC(pi-desktop/skill/market/search|fetch),但未改:

    • docs/spec/04-ux/06-settings-ia.md(Skills 工作台)
    • docs/spec/03-runtime/01-ipc-protocol.md §12b
    • 安全/出网边界(主进程拉 GitHub tree + jsDelivr + 用户源)

    另外英文 E2E 计划被粘坏:docs/spec/06-delivery/04-e2e-test-plan.md 末尾 steering 场景的 Status 后面拼进了 pnpm test:e2e:layout 残句(360px floor…),然后再接技能市场表。请先修好这段,不要在损坏的源规格上再加表。

  3. 内置目录违反 English-first。 skill-catalog-builtin.ts 把 Word 文档 / PDF 处理 等写死成中文,所有 locale 都会看到。目录条目是用户可见文案,应走 i18n,或至少英文源 + 翻译。

  4. 扫描出的 id 和 host valid_capability_id 对不齐。 GitHub 扫描保留下划线、允许数字开头;host 只接受 [a-z0-9-],且 create 会在 id 非法时改用 slugify(name)。结果:装成功了,但 installedIds.includes(entry.id) 仍是 false,按钮一直是 Install。请在扫描时用同一套 id 规则,或安装后用 host 返回的 id 标记已安装。

  5. 相邻资源展开会撞上 MAX_SKILL_BYTES(128KiB)。 最多串行抓 20 个 .md 再拼进单文件;预览只显示未展开的 document.body,安装才 expandSkillResources。大技能(pdf 的 REFERENCE.md 等)会出现「预览成功、安装报 document exceeds」。请在预览里展示将要写入的正文,超限在安装前拒绝,并决定非 md 资源是展开、拒绝还是保持单文件。

  6. 公共网络策略仍是一份拷贝。 isPublicHostname / isPublicIpLiteral 与 #285 字节级重复,两边都 export *。两个 PR 先后合入会在 @pi-desktop/shared 撞名。上次已经要求抽出主进程公共网络策略;至少把分类器放到一个模块,skill/MCP 共用。注释写「三次重试」,request() 实际没有重试。

不阻断、但请不要带进 main

  • 7 个默认 GitHub 源不可移除,冷启动每次 14 次未认证 api.github.com(60/小时限额)。国内走 jsDelivr 文档、发现仍打 GitHub API,失败时只能看到 8 条内置。这和 #287 里「默认官方 JSON 目录」的设想也不一致。
  • sourceName() 只查用户源,不查 DEFAULT_SKILL_SOURCES,默认源条目的角标几乎都是泛化的「Source」。
  • SkillMarketPanel.tsx 660 行,超过新模块 ~500 的指导线;settings.css +505 还留着 MCP 的 is-devtools / is-web 类名。
  • 内置 id 若也出现在扫描结果里,仍会打上 remote badge(remoteIds 用的是合并前的 remote 集合)。

请按上面 1–6 修完、补上真正打到 main 聚合器的测试,并 rebase 当前 main 后再请求复审。

@vastsa

vastsa commented Sep 13, 2026

Copy link
Copy Markdown
Owner

按仓库维护者指示:先合入本 PR,再另开跟进分支修审查意见(E2E 主进程覆盖、spec 同步、English-first 内置目录、skill id 与 host 对齐、128KiB 展开上限、公共网络策略抽取)。

@vastsa
vastsa merged commit 36b914e into vastsa:main Sep 13, 2026
3 of 4 checks passed
@muzimu217

Copy link
Copy Markdown
Contributor Author

Thank you for the exceptionally thorough review — the main-process E2E gap, the spec drift, and the English-first violation are all real, and the corrupted paragraph in the E2E plan is on us. We'll open the follow-up branch and address every item:

Will rebase onto current main and request re-review when the follow-up is green.

vastsa pushed a commit that referenced this pull request Sep 14, 2026
Follow-up to the #290 review. Auditing current main first: most findings
were already landed by the integration that followed the merge
(English-first builtin catalog, id sanitization, the shared public-network
module consumed by the skill catalog, the 128KiB gate in both preview and
install, source badges incl. default sources, panel/tests contracts, and
the previously glued paragraph in the E2E plan are all verified
present). What remained untested or undocumented ships here:

- The preview race gate gets its panel regression: opening a preview must
  take a LatestWinsGate token, a resolving fetch may only apply while its
  token is current, and closing the sheet must invalidate whatever is
  still in flight, with the document state reset on every open.
- The main-process aggregator gets its wiring contract: the module must
  build the public-network client over Electron's net.fetch and hand
  client.request to the aggregator, with no direct http/https module use.
- The IPC spec now defines what the market's egress policy actually is
  (syntactic URL guard, DNS classification, per-hop redirect revalidation,
  bounded responses; the renderer never reaches the network directly) in
  both locales, and a duplicated line in the settings IA spec is dropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 技能市场:像 MCP 市场一样多源可配,点击安装 SKILL.md

2 participants