Skip to content

docs(agents): require an eval before a skill change ships - #331

Merged
YuukiFST merged 2 commits into
mainfrom
docs/eval-gate
Nov 17, 2026
Merged

YuukiFST merged 2 commits into
mainfrom
docs/eval-gate

Conversation

@YuukiFST

Copy link
Copy Markdown
Owner

Problem

Nine skills were cut down in one pass. Links and routes still held, but nothing showed that the skills still did the same work. The diets were reverted, and this PR writes down the rule that should have stopped them from reaching a commit.

What changed

AGENTS.md, Hard rules, one new entry: a change to a skill or playbook is pushed only with a before/after eval showing no quality loss (skill-diet/references/eval.md for model-invoked skills, yuu-mode/playbooks/eval.md for user-invoked skills and playbooks). Until then it stays on a local branch. A change no eval covers, such as a description's triggering, stays there too unless the user waives the eval for it. The commit message carries the pass rates per version.

Evidence

Not verified: this is an instruction, and no check can run it. The first change shipped under it is the test-audit PR, whose commit carries its eval results.

Risk: low, docs only. AGENTS.md is a sensitive path, since every session in this repo loads it.

ELI5

Before a skill change goes into the repo, someone has to show that the agent works at least as well with the new text as with the old one.

YuukiFST and others added 2 commits November 17, 2026 15:19
A diet of nine skills cut their size but proved only that links and
routes held, not that the skills still did the same work. The user set
the gate: a skill or playbook change is pushed only with a before/after
eval showing no quality loss, and stays on a local branch until then.
@YuukiFST
YuukiFST merged commit 11c4ba6 into main Nov 17, 2026
24 checks passed
@YuukiFST
YuukiFST deleted the docs/eval-gate branch November 17, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant