Skip to content

feat: install base agent skills by default - #35263

Merged
andrepimenta merged 7 commits into
mainfrom
feat/base-agent-skills-by-default
Sep 8, 2026
Merged

andrepimenta merged 7 commits into
mainfrom
feat/base-agent-skills-by-default

Conversation

@andrepimenta

@andrepimenta andrepimenta commented Aug 25, 2026 •

Copy link
Copy Markdown
Member

Note

Unblocked. MetaMask/skills#135 and MetaMask/skills#154 are merged and @metamask/skills@0.3.1 is published. Ready to review.

Description

Agent skills currently require running yarn skills and picking domains by hand. This makes the base set arrive from a plain yarn install instead, so a fresh clone lands 10 skills with no flags and nothing to read first. Everything else stays opt-in via yarn skills --select.

File Change
package.json @metamask/skills bumped to ^0.3.1; postinstall can no longer fail yarn install
yarn.lock resolves @metamask/skills@npm:0.3.1
scripts/setup.mjs yarn setup reports which skills are on disk
.skills.local.example documents the base-only default and the opt-outs
README.md, AGENTS.md, .gitignore all three described yarn skills as the only way skills arrive

Why 0.3.1 and not 0.3.0. The base-skill default shipped in 0.3.0, but 0.3.0 cannot deliver it to anyone with CONSENSYS_SKILLS_DIR set: tools/sync chose its installer with an -x test, npm strips the executable bit when unpacking, so it fell through to the overlay's copy of tools/install — which predates base: and installed zero skills while exiting 0. Found by testing this branch against the published package; fixed in MetaMask/skills#154 and released as 0.3.1.

Verified against the published 0.3.1 with the exec bits exactly as npm delivers them (tools/install at 0644) and the overlay configured:

yarn skills:postinstall   claude=10 cursor=10 agents=10
yarn skills               claude=39 cursor=39 agents=39

Same setup returned 0/0/0 on 0.3.0.

Why the postinstall guard is || true. Unguarded, an unresolvable @metamask/skills (production-only install, partial node_modules) exits 127 and yarn install exits 1 — measured. That contradicts README.md, which promises "Skipping yarn skills is fine — it only affects agent tooling, not the app build."

It was || echo '...' >&2 first. Measured with yarn 4.14.1 in a scratch workspace, failing postinstall either way:

postinstall install #1 install #2
bare YN0009 + build.log path retries
|| echo … YN0007 only, message printed nowhere never retried

Yarn surfaces build output only for builds it considers failed, so the guard discarded its own message and marked the build successful, which stopped Yarn retrying. || true is the honest form; the README now says plainly that nothing is printed either way.

Why the setup report. Yarn swallows all postinstall output into a build log, so the base set arrived silently and nobody learned that team skills are opt-in. yarn setup runs in the foreground and is run often. The task only reads .claude/skills/; it never runs a sync, so yarn setup does not install twice. Skipped in CI and --node mode, where postinstall already skips installing.

Output:

....
     Found 10 agent skill(s) in .claude/skills.

     The base set installs automatically; yarn skills adds every domain:
      🔎 Pick specific domains:       yarn skills --select
      📖 Inspect one:                 yarn metamask-skills describe <domain>/<skill>
      🧹 Drop ones no longer managed: yarn skills --prune-stale
                                      (add --domain none for the base set only)
....

"Found", not "installed": postinstall returns 0 even when the sync fails, so this is a directory listing rather than a report of what this run did. It names only Claude Code because it only reads .claude/skills — the three targets drift, and captioning it "and Cursor" reported 10 on a checkout where Cursor was loading 39.

The prune hint cannot be one command. With a domain saved, plain --prune-stale keeps it (12 → 12) and --domain none silently discards it (12 → 10); with nothing saved the fallback is all, so plain --prune-stale installs every domain first (10 → 37). Both measured.

One oddity worth knowing. The trailing blank line comes from a ​ constant, not whitespace. Listr trims each captured console.log chunk, so a trailing newline, space, tab, NBSP, or an extra console.log('') is all removed and the next task's output butts against this one. A zero-width space is not in JavaScript's whitespace set, so it survives the trim while rendering as nothing. Written as an escape with a comment rather than a literal, so no invisible byte sits in the file for a formatter to strip.

Behaviour change. Engineers without SKILLS_DOMAINS set go from ~36 skills to 10 on their next dependency change. Nothing is pruned automatically, so existing installs accumulate — yarn skills --prune-stale cleans up once, and SKILLS_DOMAINS=all restores previous behaviour. Opt out entirely with SKILLS_AUTO_UPDATE=0 or SKILLS_SKIP_POSTINSTALL=1.

This also reaches external contributors: @metamask/skills is a devDependency of a public repo, so a drive-by contributor's yarn install will clone the public skills repo into .skills-cache/ and write 10 gitignored skill files. Public content, gitignored output, one clone — flagging it rather than letting it be discovered later.

Changelog

CHANGELOG entry: null

Developer tooling only — no user-facing change.

Related issues

Refs: AIENG-38
Depended on: MetaMask/skills#135 and MetaMask/skills#154 (both merged, released as @metamask/skills@0.3.1)

Manual testing steps

Feature: base agent skills install by default

  Scenario: engineer runs setup and sees which skills are available
    Given a checkout with @metamask/skills at a version that supports `base`
    When the engineer runs `yarn setup:expo`
    Then the "Report agent skills" task lists the number of available skills
    And it points at `yarn skills --select` for opt-in domains
    And the EXPO "Setup complete!" banner remains the final message

  Scenario: a broken skills CLI does not fail the install
    Given @metamask/skills is not resolvable
    When the engineer runs `yarn install`
    Then the install exits 0
    And `yarn skills:postinstall` run by hand still exits non-zero and reports why

  Scenario: CI does not run the report
    Given CI is set
    When setup runs
    Then the "Report agent skills" task is skipped

Screenshots/Recordings

Before

N/A — no UI change. Before this PR, yarn setup printed nothing about agent skills.

After

Terminal output shown in the Description above.

Pre-merge author checklist

Performance checks (if applicable)

  • I've tested on Android
  • I've tested with a power user scenario
  • I've instrumented key operations with Sentry traces for production performance metrics

N/A for all three — this changes a setup script and a package.json script, with no app-runtime code path.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Low Risk
Developer-only install and setup scripting; no app runtime or production dependency paths change beyond a resilient postinstall hook.

Overview
Fresh clones get a default agent skill set without running yarn skills first. The PR bumps @metamask/skills to ^0.3.1 so postinstall syncs only base skills during yarn install / setup; yarn skills still installs every domain. Docs (README, AGENTS.md, .skills.local.example, .gitignore) are updated to match that split and to document SKILLS_DOMAINS, opt-outs (SKILLS_AUTO_UPDATE=0), and when Yarn actually re-runs postinstall.

yarn install no longer fails if the skills hook breaks: postinstall is now yarn skills:postinstall || true, aligning with the promise that agent tooling is optional for the app build.

yarn setup surfaces what’s on disk: a new Report agent skills Listr task (local only, not CI/--node) lists mms-* folders under .claude/skills and points people at yarn install for the base set or yarn skills for all domains, without triggering a second sync.

Reviewed by Cursor Bugbot for commit 51dd8ea. Bugbot is set up for automated code reviews on this repo. Configure here.

Agent skills now arrive from a plain `yarn install` instead of requiring
`yarn skills` and a domain selection. The base set (10 skills) installs for
everyone; everything else stays opt-in via `yarn skills --select`.

- `postinstall` can no longer fail `yarn install`. When @metamask/skills is
  unresolvable — a production-only install, or a partial node_modules — the
  script exited 127 and Yarn failed the workspace build, contradicting the
  README's promise that skipping skills does not affect the app build.
- `yarn setup` reports which skills are available. Yarn swallows postinstall
  output into a build log, so without this the base set arrives silently and
  nobody learns that team skills are opt-in. It only reads what is on disk,
  never runs a sync, and is skipped in CI and node-only mode.
- `.skills.local.example` documents the base-only default and the
  `SKILLS_AUTO_UPDATE=0` / `SKILLS_SKIP_POSTINSTALL=1` opt-outs.

Requires the `base` flag from MetaMask/skills.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the pr-not-ready-for-e2e Skip E2E and block merging. Remove this label once the PR is ready to run the E2E tests. label Aug 25, 2026
@metamask-ci metamask-ci Bot added team-mobile-platform Mobile Platform team INVALID-PR-TEMPLATE PR's body doesn't match template labels Aug 25, 2026

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .skills.local.example Resolved
@cursor
cursor Bot requested review from abretonc7s and tommasini August 25, 2026 16:09
@github-actions github-actions Bot added the risk:medium AI analysis: medium risk label Aug 25, 2026
@andrepimenta

Copy link
Copy Markdown
Member Author

Base skills default not shipped

@metamask/skills stays locked at 0.2.0, which still requires SKILLS_AUTO_UPDATE=1 to sync, so that default never takes effect and the new messaging is wrong.

Confirmed — this is a real bug, not a false positive. Verified locally:

$ node -p "require('./node_modules/@metamask/skills/package.json').version"
0.2.0
$ grep -c 'Auto-update is on by default' node_modules/@metamask/skills/bin/metamask-skills.mjs
0

The auto-update default lives in MetaMask/skills#135, which is unmerged and unpublished. Under 0.2.0 the gate is still isTruthy(getConfigValue(...)), so an unset SKILLS_AUTO_UPDATE skips the sync entirely — nothing installs, and two claims in this PR are false:

  • .skills.local.example: "Skills refresh automatically during yarn install/setup"
  • scripts/setup.mjs: "The base set is installed for everyone"

Marking this PR as draft. It is sequenced behind a skills release and cannot merge standalone:

  1. feat: install a base skill set by default skills#135 merges
  2. @metamask/skills is published with the base flag and auto-update default
  3. This PR adds the dependency bump to that version, then merges

The postinstall guard (|| true) is the one part that is correct and useful today — it stops an unresolvable skills CLI from failing yarn install regardless of version. Everything else in here depends on step 2.

Good catch by Bugbot; I'd have shipped documentation for a state that didn't exist.

@andrepimenta
andrepimenta marked this pull request as draft August 25, 2026 16:18
@andrepimenta andrepimenta changed the title feat: install base agent skills by default feat: install base agent skills by default (blocked on skills release) Aug 25, 2026
@metamask-ci metamask-ci Bot removed the INVALID-PR-TEMPLATE PR's body doesn't match template label Aug 25, 2026

@NicolasMassart NicolasMassart left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One additional non-blocking concern beyond the existing Cursor thread.

Comment thread package.json Resolved
@Qbandev

Qbandev commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

The three changed files are fine as written — but the pairing with MetaMask/skills#135 has a supply-chain gap worth closing before the default flips on. Traced every hop:

yarn install would execute an unpinned script from the tip of skills main.

  1. postinstall() → ensurePublicSkillsCache() clones/reset --hard origin/main the skills repo into .skills-cache/ — no tag, no commit pin (bin/metamask-skills.mjs:307-335).
  2. buildDelegatedEnv() defaults METAMASK_SKILLS_DIR to that cache (:387-390).
  3. delegate('sync') correctly runs the lockfile-pinned package's tools/sync — but tools/sync then prefers and execs $METAMASK_SKILLS_DIR/tools/install, i.e. arbitrary shell from whatever is on main at that instant (tools/sync:405-428).

The npm package path has lockfile pinning, release gating, and Socket/CodeQL review; the cache path has none of that, and once this default flips it reaches every engineer and every drive-by contributor of a public repo on every yarn install. The fix is small and lives in MetaMask/skills — tools/sync should exec PACKAGE_ROOT/tools/install (the same pinned place tools/sync itself came from) and let the cache supply domains/ content only — but it should land before the dependency bump here. Relatedly, the PR body's "Public content, gitignored output, one clone" framing omits the execution step; worth correcting so reviewers accept the actual risk.

+1 to @NicolasMassart on || true, with data.
The skills CLI already returns 0 on every path — including a literal return result === 0 ? 0 : 0 (bin/metamask-skills.mjs:754) — and the cache fetch is wrapped in its own try/catch. So the only things || true will ever swallow are the missing-binary 127 and a genuine future crash (node throw, OOM) — the one thing you'd want in the log. One-token alternative that keeps installs green:

"postinstall": "yarn skills:postinstall || echo 'skills postinstall failed (non-fatal)'"

Smaller items while it's in draft:

  • README.md:277 still documents the old opt-in default (SKILLS_AUTO_UPDATE=1) — after the dep bump that's backwards; ship the fix in the same commit so the two never disagree.
  • scripts/setup.mjs:288-292 counts every directory under .claude/skills/, including personal skills — filtering on the mms- prefix makes the number match the "installed for everyone" sentence. The message also names Claude Code/Cursor/Codex but checks only .claude/skills.

The escape hatches check out well: CI is skipped, SKILLS_AUTO_UPDATE=0 genuinely skips even the network fetch (thanks to #135 moving the cache refresh behind the gate), and all output dirs are gitignored.

Review pass over the base-skill install. The reporting task could not actually
deliver its message, and two failure paths were wrong in opposite directions.

- The empty-state guidance never rendered. rendererOptions sets
  `showSkipMessage: false`, and listr2 draws
  `task.message.skip && showSkipMessage ? task.message.skip : task.title`, so
  the string passed to task.skip() was discarded and the user saw only
  "Report agent skills [SKIPPED]". On a fresh clone that is the most likely
  path, so the one case the task exists to serve produced no output at all.
  Retitling is the only way the `yarn skills` hint reaches the screen.

- readdirSync failures were all reported as "none installed". EACCES after a
  stray `sudo yarn`, or ENOTDIR if a file shadows the directory, mean the state
  is unknown rather than empty; pointing the reader at `yarn skills` sends them
  at a command that fails the same way without naming the cause. Only ENOENT
  now falls through to the empty state.

- That non-ENOENT case is retitled, not thrown. Both task lists run with
  exitOnError and `tasks.run()` is unguarded, so throwing would abort Husky,
  the Expo build links and the whole iOS / Terms-of-Use stage over a skill
  count — breaking the README's own promise that skipping skills does not
  affect the app build. Confirmed against listr2 8.0.2 that a per-task
  `exitOnError: false` does not help: the task fails, the parent fails, and
  run() still rejects.

- The printed message contradicted itself. It said everything beyond the base
  set is opt-in, then offered `yarn skills` as a refresh — which falls back to
  every domain. It now says so plainly and offers `yarn skills --select` for a
  narrower pick. The Codex claim is dropped too: project-scope skills land in
  .claude/skills, .cursor/rules and .agents/skills, while Codex only ever
  receives `scope: user` skills, which install to $HOME and are not counted.

- The count is filtered to `mms-` so hand-authored skills in the same directory
  do not inflate it.

- postinstall failures are no longer silent. `|| true` swallowed them whole;
  with the sync now default-on it can legitimately fail (unknown domain, or an
  ETIMEDOUT from the delegate cap), and Yarn would still report green. The
  message goes to stderr with a retry hint, and the hook stays non-fatal.

- README and .skills.local.example describe the base-set default rather than
  the old "yarn install refreshes the cache" behaviour.

Still requires the `base` flag from MetaMask/skills: that work is unreleased,
and this repo pins ^0.2.0, which cannot resolve the 0.3.0 that will carry it.
Bump the dependency and lockfile before merging.

Verified: eslint, prettier --check and node --check all clean;
`SKILLS_SKIP_POSTINSTALL=1 yarn postinstall` exits 0, confirming the new `>&2`
redirection parses in Yarn 4.14.1's portable shell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@andrepimenta

Copy link
Copy Markdown
Member Author

Thanks — the supply-chain trace was the most valuable thing on either PR, and all four items are now closed. Two land in MetaMask/skills#135 (c30fa37), two here (224c2de).

yarn install executing unpinned script from skills main. Fixed in skills#135, and landing before the dependency bump here as you asked. tools/sync now execs $SCRIPT_DIR/install — the sibling of the pinned tools/sync already running — rather than preferring $METAMASK_SKILLS_DIR/tools/install:

SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
if [[ -x "$SCRIPT_DIR/install" ]]; then
  INSTALL_BIN="$SCRIPT_DIR/install"

The cache still supplies domains/ content via SOURCES; it no longer supplies executable code. Your hop-by-hop breakdown is what made this obvious — step 3 was genuinely invisible from the PR body.

Worth flagging a consequence, since it cuts the other way: with the pinned installer now always winning, pinned code can be older than cached content. Concretely, a consumer on 0.2.0 whose cache has already merged the mandatory → base rename would run an installer that only reads mandatory, match nothing, and install zero base skills while exiting 0. skills#135 now accepts both keys during the transition to cover that window.

PR body framing. Fair, and still outstanding — "Public content, gitignored output, one clone" describes the data flow and skips the execution step entirely. I'll correct it when this comes out of draft with the dependency bump, rather than describing a risk profile that's about to change again.

|| true. Changed — your echo shape, routed to stderr with a retry hint. Full reasoning in the thread on @NicolasMassart's comment, including confirmation of your return result === 0 ? 0 : 0 finding: after subtracting everything the CLI already normalises to zero, the only survivors are the 127 and a genuine crash, which is precisely why they shouldn't be silent.

README.md:277. Fixed in the same commit, as suggested — the README and .skills.local.example now agree on the base-set default.

scripts/setup.mjs counting every directory. Fixed — filtered to the mms- prefix the installer applies. The Claude Code/Cursor/Codex naming is corrected too: project-scope skills land in .claude/skills, .cursor/rules and .agents/skills, while Codex only ever receives scope: user skills, which install to $HOME and are deliberately not counted. The message now names only Claude Code and Cursor.

While in that file, two further bugs in the same task:

  • The empty-state guidance never rendered at all. rendererOptions sets showSkipMessage: false, and listr2 draws task.message.skip && showSkipMessage ? task.message.skip : task.title — so the string passed to task.skip() was discarded and the user saw only Report agent skills [SKIPPED]. On a fresh clone that's the most likely path, so the one case the task exists for produced no output. Now retitles instead.
  • Every readdirSync failure reported as "none installed." EACCES after a stray sudo, or ENOTDIR, mean the state is unknown, not empty — and "run yarn skills" sends the reader at a command that fails identically without naming the cause. Only ENOENT falls through to the empty state now; the rest retitle with the error code. Deliberately not thrown: both task lists run with exitOnError and tasks.run() is unguarded, so throwing from an informational step would abort Husky, the Expo links and the whole iOS/Terms-of-Use stage over a skill count.

On your escape-hatch check — still accurate, with one addition. SKILLS_AUTO_UPDATE=0 skipping the network fetch still holds. skills#135 now also treats an explicitly empty SKILLS_AUTO_UPDATE= as off rather than unset, since under 0.2.0 isTruthy('') was false and engineers wrote exactly that to disable syncing; reading it as absent would have silently re-enabled it for them.

@github-actions github-actions Bot added size-M and removed size-S labels Sep 1, 2026
@andrepimenta
andrepimenta marked this pull request as ready for review September 3, 2026 11:36

@NicolasMassart NicolasMassart left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will be good to merge once skills bumped to @metamask/skills 0.3.0

andrepimenta added a commit to MetaMask/skills that referenced this pull request Sep 3, 2026
## What

Turns on a **base skill set** that installs for every engineer on a
fresh clone, with no flags and nothing to read first. Mobile gets 10
skills; everything else stays opt-in.

The `mandatory` frontmatter flag is renamed to `base` and actually used.
It was already implemented end to end — `tools/install`, `tools/sync`,
the schema, the CLI, even a `★` marker in the interactive picker — but
**no skill had ever set it**, and the default domain filter was
permissive enough that the bypass had nothing to bypass.

## Why

Three things had to be true together for "base skills by default" to
work. Only the flag existed.

| Problem | Fix |
|---|---|
| Empty `SKILLS_DOMAINS` meant *all* domains, so `base` never applied |
`none`/`all` sentinels |
| `SKILLS_AUTO_UPDATE` was off by default, so `postinstall` refreshed
the cache and installed nothing | Defaults on. The cache refresh moved
*behind* the gate, so opted-out engineers no longer pay a network fetch
per install |
| Six base descriptions were too thin to ever trigger | Rewritten.
`coding-guidelines` was literally `"General coding guidelines"` — 25
characters |

That last one is the subtle one: a skill's description is its **entire
trigger surface**. A 25-character description never fires on the work it
governs, while appearing to be covered.

## Automatic vs explicit

`yarn skills` reads as "give me the skills", so having it install 10
instead of 36 would be a surprising regression for a command someone
typed on purpose. The two paths differ:

| Path | Domains | Installed |
|---|---|---|
| `yarn skills`, no config | all | **36** — unchanged from today |
| `yarn install` → `postinstall`, no config | none | **10** |
| `yarn install` **with `SKILLS_DOMAINS=perps`** | perps | **12** = 10
base + 2 perps |
| `yarn skills --domain none` | none | 10 |

`postinstall` exports `SKILLS_DEFAULT_SCOPE=base` and `tools/sync`
branches its **fallback** on that. Everything above the fallback in the
resolution order is untouched — a `--domain` flag, `SKILLS_DOMAINS`, the
picker, and a saved selection all still win.

That last property is the point of doing it this way. Passing `--domain
none` from `postinstall` would have outranked a saved selection, so an
engineer who opted into `perps` would have silently lost it on their
next `yarn install`.

## The base set (10)

`mobile-testing` · `coding-guidelines` · `controller-integration` ·
`ui-development` · `component-scaffold` · `performance` ·
`pr-guidelines` · `create-pr` · `pr-description` · `pr-codeowners`

`pr-description` and `pr-codeowners` are included because `create-pr`
delegates its main output to them with "if available" phrasing — without
them it silently runs in a reduced mode.

Always-on cost: **~1,130 tokens** per session. The set is deliberately
small: when the agent's skill listing overflows its budget, descriptions
get truncated starting with the least-invoked skills, so a bloated base
set makes *every* skill harder to trigger.

## Safety

Enabling auto-update puts git operations on every engineer's install
path, so this also:

- suppresses git credential prompts (`GIT_TERMINAL_PROMPT=0`, empty
`GIT_ASKPASS`) in both `run()` and the delegated environment —
`tools/sync` pulls the **private** Consensys overlay, which could
otherwise block on stdin during `yarn install` with no output
- bounds every spawn: 300s for network git, 10s for local `bash
--version` probes

## Guardrails added

- `base: true` + `maturity: experimental` is now a **lint error** —
contradictory intent; installing unfinished guidance for everyone is the
bad outcome
- warns when a base skill's description is under 120 characters

## Verified

| Check | Result |
|---|---|
| Automatic path (`SKILLS_DEFAULT_SCOPE=base`) | exactly 10, in
`.claude/skills` and `.agents/skills` |
| Explicit `yarn skills` | 36 |
| Automatic + `SKILLS_DOMAINS=perps` | 12 — saved selection respected |
| `SKILLS_AUTO_UPDATE=0` / `CI=1` | silent, exit 0 |
| Full `postinstall` → `sync` → `install` | exit 0 |
| Overlay merge intact | `coding-guidelines` 71-byte source stub → 3,990
bytes installed |
| `node --test` | 65/65 |
| `lint-skill-entry.mjs` | 52 skills, 0 errors |

## Migration note

Existing engineers are **not** dropped to 10. Their current installs
stay, because pruning is off by default and their next `yarn install`
only adds the base set. `yarn skills --prune-stale` once gives a clean
state.

This does reach **external contributors**: the skills package is a
devDependency of public repos, so a drive-by contributor's `yarn
install` now clones this repo into `.skills-cache/` and writes 10
gitignored skill files. Public content, gitignored output, one clone —
flagging it rather than letting it be discovered later.

## Related

- Extension's base set: stacked PR on this branch
- Consumer side: MetaMask/metamask-mobile#35263 (draft — needs a
published release of this)

Known gaps documented separately: no standardized PR-review skill,
platform APIs at 2 of ~11, and five deprecated testing skills that still
compete with `mobile-testing`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
NicolasMassart pushed a commit to MetaMask/skills that referenced this pull request Sep 3, 2026
Turns on a **base skill set** that installs for every engineer on a
fresh clone, with no flags and nothing to read first. Mobile gets 10
skills; everything else stays opt-in.

The `mandatory` frontmatter flag is renamed to `base` and actually used.
It was already implemented end to end — `tools/install`, `tools/sync`,
the schema, the CLI, even a `★` marker in the interactive picker — but
**no skill had ever set it**, and the default domain filter was
permissive enough that the bypass had nothing to bypass.

Three things had to be true together for "base skills by default" to
work. Only the flag existed.

| Problem | Fix |
|---|---|
| Empty `SKILLS_DOMAINS` meant *all* domains, so `base` never applied |
`none`/`all` sentinels |
| `SKILLS_AUTO_UPDATE` was off by default, so `postinstall` refreshed
the cache and installed nothing | Defaults on. The cache refresh moved
*behind* the gate, so opted-out engineers no longer pay a network fetch
per install |
| Six base descriptions were too thin to ever trigger | Rewritten.
`coding-guidelines` was literally `"General coding guidelines"` — 25
characters |

That last one is the subtle one: a skill's description is its **entire
trigger surface**. A 25-character description never fires on the work it
governs, while appearing to be covered.

`yarn skills` reads as "give me the skills", so having it install 10
instead of 36 would be a surprising regression for a command someone
typed on purpose. The two paths differ:

| Path | Domains | Installed |
|---|---|---|
| `yarn skills`, no config | all | **36** — unchanged from today |
| `yarn install` → `postinstall`, no config | none | **10** |
| `yarn install` **with `SKILLS_DOMAINS=perps`** | perps | **12** = 10
base + 2 perps |
| `yarn skills --domain none` | none | 10 |

`postinstall` exports `SKILLS_DEFAULT_SCOPE=base` and `tools/sync`
branches its **fallback** on that. Everything above the fallback in the
resolution order is untouched — a `--domain` flag, `SKILLS_DOMAINS`, the
picker, and a saved selection all still win.

That last property is the point of doing it this way. Passing `--domain
none` from `postinstall` would have outranked a saved selection, so an
engineer who opted into `perps` would have silently lost it on their
next `yarn install`.

`mobile-testing` · `coding-guidelines` · `controller-integration` ·
`ui-development` · `component-scaffold` · `performance` ·
`pr-guidelines` · `create-pr` · `pr-description` · `pr-codeowners`

`pr-description` and `pr-codeowners` are included because `create-pr`
delegates its main output to them with "if available" phrasing — without
them it silently runs in a reduced mode.

Always-on cost: **~1,130 tokens** per session. The set is deliberately
small: when the agent's skill listing overflows its budget, descriptions
get truncated starting with the least-invoked skills, so a bloated base
set makes *every* skill harder to trigger.

Enabling auto-update puts git operations on every engineer's install
path, so this also:

- suppresses git credential prompts (`GIT_TERMINAL_PROMPT=0`, empty
`GIT_ASKPASS`) in both `run()` and the delegated environment —
`tools/sync` pulls the **private** Consensys overlay, which could
otherwise block on stdin during `yarn install` with no output
- bounds every spawn: 300s for network git, 10s for local `bash
--version` probes

- `base: true` + `maturity: experimental` is now a **lint error** —
contradictory intent; installing unfinished guidance for everyone is the
bad outcome
- warns when a base skill's description is under 120 characters

| Check | Result |
|---|---|
| Automatic path (`SKILLS_DEFAULT_SCOPE=base`) | exactly 10, in
`.claude/skills` and `.agents/skills` |
| Explicit `yarn skills` | 36 |
| Automatic + `SKILLS_DOMAINS=perps` | 12 — saved selection respected |
| `SKILLS_AUTO_UPDATE=0` / `CI=1` | silent, exit 0 |
| Full `postinstall` → `sync` → `install` | exit 0 |
| Overlay merge intact | `coding-guidelines` 71-byte source stub → 3,990
bytes installed |
| `node --test` | 65/65 |
| `lint-skill-entry.mjs` | 52 skills, 0 errors |

Existing engineers are **not** dropped to 10. Their current installs
stay, because pruning is off by default and their next `yarn install`
only adds the base set. `yarn skills --prune-stale` once gives a clean
state.

This does reach **external contributors**: the skills package is a
devDependency of public repos, so a drive-by contributor's `yarn
install` now clones this repo into `.skills-cache/` and writes 10
gitignored skill files. Public content, gitignored output, one clone —
flagging it rather than letting it be discovered later.

- Extension's base set: stacked PR on this branch
- Consumer side: MetaMask/metamask-mobile#35263 (draft — needs a
published release of this)

Known gaps documented separately: no standardized PR-review skill,
platform APIs at 2 of ~11, and five deprecated testing skills that still
compete with `mobile-testing`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The base-skills behaviour this PR depends on shipped in 0.3.0. The previous
range `^0.2.0` resolves to `>=0.2.0 <0.3.0`, so it could never pick the new
version up on its own — the manifest had to change.

Verified against the published package rather than a local checkout:
`yarn skills:postinstall` runs clean, 16 skills carry `base: true`, and the
domain and maturity filters skip the rest as expected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@andrepimenta andrepimenta changed the title feat: install base agent skills by default (blocked on skills release) feat: install base agent skills by default Sep 7, 2026
@socket-security

socket-security Bot commented Sep 7, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednpm/​@​metamask/​skills@​0.2.0 ⏵ 0.3.199 +25100100 +194 +2100

View full report

@github-actions github-actions Bot added risk:low AI analysis: low risk and removed risk:medium AI analysis: medium risk labels Sep 7, 2026
Five points from review, all on the reporting and documentation around the
change rather than the change itself.

`.skills.local.example` documented `SKILLS_SKIP_POSTINSTALL=1` as a setting for
that file, but the CLI reads it from `process.env` only —
`shouldSkipPostinstall(process.env)`, where `SKILLS_AUTO_UPDATE` right beside it
goes through `getConfigValue(env, localConfig, ...)`. An engineer opting out of
the new default-on behaviour would have written the line, seen no effect, and
kept getting a sync on every install. Documented as an export, with
`SKILLS_AUTO_UPDATE=0` named as the one this file can express.

`setup.mjs` claimed more than it knew. The count is a directory listing, not a
report of what the sync did: `metamask-skills postinstall` returns 0 on internal
failure, so a sync that errored leaves the previous install in place and it was
being announced as "installed". It also counts directories from an earlier
all-domains `yarn skills` — postinstall does not pass --prune-stale, so those
persist unmanaged. Now says "Found N", with the caveats in a comment, and the
third bullet points at `--prune-stale` instead of telling engineers to run
`yarn skills` as a routine refresh. That bullet was self-defeating: it silently
expanded a base-only install to every domain, which the line above it describes
as the opposite of what happens automatically.

`error.code` is now `error?.code`. A non-object rejection would have thrown a
TypeError from inside the catch — the exact abort of Husky, Expo links and the
iOS/Terms stage that the comment above it rules out.

AGENTS.md and .gitignore still described `yarn skills` as the only way skills
arrive, and .gitignore pointed at a `SKILLS_DOMAINS=` key this PR comments out.
README's prose said skills sync "on demand" two lines above a code block saying
`yarn install` installs them. AGENTS.md is loaded into every agent's context via
CLAUDE.md, so it was the most consequential of the three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added risk:high AI analysis: high risk and removed risk:low AI analysis: low risk labels Sep 8, 2026
The `|| echo` guard was measurably worse than no guard. Tested with yarn 4.14.1
in a scratch workspace, failing postinstall either way:

  bare        YN0009 + build.log path, re-runs the build on every later install
  || echo     YN0007 only, message printed nowhere, never retried again

Yarn surfaces build output only for builds it considers failed, so the guard
both discarded its own message and marked the build successful — meaning Yarn
stopped retrying too. The commit that added it claimed "postinstall failures are
no longer silent"; they were more silent. It cannot be dropped (unguarded,
`yarn install` exits 1, which the README rules out), so it is now `|| true` and
the README says plainly that nothing is printed either way.

The `--prune-stale` bullet expanded what it claimed to trim. tools/sync:415
falls back to DOMAINS=all unless postinstall exports SKILLS_DEFAULT_SCOPE, so
`yarn skills --prune-stale` installs all 39 and prunes nothing — the exact
self-defeating property removed from the `yarn skills` refresh bullet in the
previous commit, reintroduced one line below it. Now `--domain none
--prune-stale`. The empty-state title had the same flaw and now names
`yarn install` for the base set.

The count reads .claude/skills and said "for Claude Code and Cursor". Those
directories are written independently and drift: on this checkout .claude/skills
held 10 while .cursor/rules and .agents/skills held 39, so the message
undercounted Cursor by 29. It now says "in .claude/skills" and speaks only for
what it read.

Docs: .gitignore:215 still described `yarn skills` as how skills arrive — the
previous commit claimed that fix but only touched the .skills.local block above
it. README and .skills.local.example both overstated the refresh: Yarn re-runs
postinstall only when the dependency tree changes or the last build failed, so
`git pull && yarn install` usually refreshes nothing. Also documented that the
empty-SKILLS_DOMAINS-means-all rule is .skills.local-only; an exported empty
value is not an override at all.

Not addressed: the report task still has no unit test. Testing it needs
setup.mjs to become importable (it runs `await tasks.run()` at module scope) and
Jest's script glob is `.test.ts` against CommonJS, so it is a refactor of a
critical setup path rather than an added file. Raising it rather than doing it
quietly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added risk:low AI analysis: low risk and removed risk:high AI analysis: high risk labels Sep 8, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 8048038. Configure here.

Comment thread scripts/setup.mjs Resolved
Testing the previous commit's `--domain none --prune-stale` against an engineer
who has opted into a domain showed it does the wrong thing for them. There is no
single correct command here, because sync's fallback depends on what the reader
already has:

  saved SKILLS_DOMAINS=perps   plain --prune-stale     12 -> 12  (saved wins)
                               + --domain none         12 -> 10  (opt-in dropped)
  nothing saved                plain --prune-stale     10 -> 37  (fallback is all)
                               + --domain none         10 -> 10

So the previous line was right for the default reader and silently stripped the
domain of anyone who had configured one. Recommend the plain form and mention
`--domain none` as the base-set-only variant, rather than hardcoding either.

Also verified while there that a domain opt-in survives the base-only default on
all three paths — saved, env var and flag each resolve to base + perps (12).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added risk:high AI analysis: high risk and removed risk:low AI analysis: low risk labels Sep 8, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.93%. Comparing base (7640d03) to head (7a00ed9).
⚠️ Report is 501 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #35263      +/-   ##
==========================================
+ Coverage   85.35%   85.93%   +0.58%     
==========================================
  Files        6610     6825     +215     
  Lines      179756   191142   +11386     
  Branches    44611    47558    +2947     
==========================================
+ Hits       153430   164260   +10830     
- Misses      16050    16227     +177     
- Partials    10276    10655     +379     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

0.3.0 cannot deliver base skills to anyone with CONSENSYS_SKILLS_DIR set — it
picks the overlay's copy of tools/install because npm strips the executable bit
its guard tested for. 0.3.1 fixes that, so the floor moves rather than staying
at a release that does not work for part of the team.

Verified against the published package with the exec bits exactly as npm
delivered them (tools/install at 0644) and the overlay configured:

  yarn skills:postinstall   10 base skills, all three targets
  yarn skills               39, every domain
  back to base              10

`yarn install` on its own reports 0 here, which is not a regression: Yarn had
already rebuilt the workspace during `yarn up`, and it re-runs postinstall only
when the dependency tree changes or the last build failed. `--check-cache` does
not force it either. That is the behaviour this branch documents in README and
.skills.local.example.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@andrepimenta andrepimenta removed the pr-not-ready-for-e2e Skip E2E and block merging. Remove this label once the PR is ready to run the E2E tests. label Sep 8, 2026
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🔍 Smart E2E Test Selection

  • Selected E2E tags: None (no tests recommended)
  • Selected Performance tags: None (no tests recommended)
  • Risk Level: low
  • AI Confidence: 97%
click to see 🤖 AI reasoning details

E2E Test Selection:
All changes in this PR are limited to developer tooling and documentation:

  1. package.json: Two changes — (a) postinstall script made non-fatal with || true to prevent build failures if skills sync fails; (b) @metamask/skills bumped from ^0.2.0 to ^0.3.1. The @metamask/skills package is a developer AI assistant tooling package, not an app dependency. Neither change affects any app code, test infrastructure, or user-facing functionality.

  2. scripts/setup.mjs: Added reportAgentSkillsTask — a purely informational task that reads .claude/skills/ directory and reports installed agent skills to developers during yarn setup. This is developer experience (DX) tooling only, with no impact on the app build or test execution.

  3. .gitignore: Comment-only changes (no functional changes to ignore patterns).

  4. AGENTS.md: Documentation updates about skills installation workflow.

  5. README.md, .skills.local.example: Developer documentation files.

  6. yarn.lock: Updated to reflect the @metamask/skills version bump.

None of these changes touch app source code, test infrastructure, E2E test files, page objects, selectors, flows, or any user-facing functionality. No E2E tests need to run.

Performance Test Selection:
No app code changes were made. All changes are limited to developer tooling (@metamask/skills package), setup scripts, and documentation. There is no impact on any performance-sensitive flows such as app launch, login, onboarding, asset loading, swaps, or any other measured performance scenario.

View GitHub Actions results

@sonarqubecloud

sonarqubecloud Bot commented Sep 8, 2026

Copy link
Copy Markdown

@andrepimenta
andrepimenta added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit 4d3aa24 Sep 8, 2026
133 of 138 checks passed
@andrepimenta
andrepimenta deleted the feat/base-agent-skills-by-default branch September 8, 2026 14:05
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 8, 2026
@metamask-ci metamask-ci Bot added the release-8.12.0 Issue or pull request that will be included in release 8.12.0 label Sep 8, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

release-8.12.0 Issue or pull request that will be included in release 8.12.0 risk:high AI analysis: high risk size-M team-mobile-platform Mobile Platform team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants