Repository navigation
fix(recall): cap the confidence feedback ratio - #1256
Conversation
Signed-off-by: ydflow <314143294+ydflow@users.noreply.github.com>
|
No findings in the specified diff. The ratio cap fixes the reported overflow while preserving the existing base, recency calculation, counters, and promotion thresholds. Regression tests cover excess feedback with recent, absent, and expired recalls; documentation is consistent across both languages. The PR description satisfies the testing requirement with a representative built-CLI before/after verification. The reported unchanged Windows test failure is not a blocking issue for this diff. I did not run, build, or install anything from the PR. |
SaulMoro
left a comment
There was a problem hiding this comment.
This was generated by AI during triage.
This PR caps the upvote-to-recall ratio in computeConfidence at 1 before the 0.4 / 0.3 / 0.3 weights, so manual recall feedback --positive can no longer push confidence above 1 or stand in for recency (fixes #1255). The one-line formula change is correct, and the new tests pin the three cases from the issue. Every reader of the score now gets values in [0, 1]: frontmatter writeback, prune, promote, and search-index hot/cold. The only behavior change is the intended one: a learning that reached the 0.90 promotion bar, or the hot threshold, only through ratio overflow no longer does. The earlier codex-review-on-assign pass reported no findings, and there are no other review threads.
Local checks at head 3ce7e78c, and on its merge into current origin/main 0da2be76 (the PR base fa6a361b is behind; the merge is clean):
npx tsc --noEmit: pass on head and on the merge.npx vitest run src/__tests__/confidence.test.ts src/__tests__/maintenance-promote.test.ts: 14 passed on head. On the merge, addingmaintenance-prune,maintenance-votes-readonly,search-index,votes, andcommands-reference: 7 files, 132 passed.- Red/green: with
origin/main'sconfidence.tsrestored on the merge, the 3 new tests fail (expected 2.2 to be 1,expected 1.9 to be 0.7twice). With the PR's version they pass. npm run lint(exit 0) andnpm run build: pass on the merge.- Real CLI under a throwaway HOME with a local bare git team repo. I ran
teamai recall feedback --positive lesson5 times (Upvoted: lessoneach time), staged the resulting totals (recalled_count: 1,upvoted_count: 5, recentlast_recalled_at) onteamai-reports, then ranteamai recall maintenance --confidence-writeback. The publishedteamai-learnings:learnings/lesson.mdhadconfidence: 2.2with theorigin/mainbuild andconfidence: 1with the merge build. Both exited 0 and printedPublished maintenance changes to the learnings branch.
Findings
- [P1 blocking] None.
- [P3 nit]
docs/guide/knowledge.md:100(anddocs/guide/zh-CN/knowledge.md:100): "cannot exceed that factor's 30% weight or replace the recency factor" refers to weights and factors the guide never defines, so a reader cannot use it. A plainer sentence is enough: "Confidence is an activity score from 0 to 1, not a measure of whether the knowledge is correct. Extra positive feedback cannot push it above 1 or make up for a learning that has not been recalled recently." Both languages are present and say the same thing. - [P3 nit]
skill-data/core/references/troubleshooting.md:359-362: this paragraph sits under "A recalled doc got no upvote" but describes the score formula, which is not that symptom, and it repeats the guide. AGENTS.md asks to change docs rarely and to treatskill-data/like docs. Consider keeping only the first sentence (manual feedback adds an upvote without a recall), which fits the section, or dropping the paragraph.
Not covered
- I did not get the pending feedback deltas onto
teamai-reportsthrough the CLI itself.teamai pullin this fixture left them pending in~/.teamai/user-votes/, so I staged the totals by hand, as the issue's repro did. Vote sync is unchanged by this PR. - No e2e file covers confidence writeback, so I ran none. I did not run the full unit or E2E suites, or any other provider or agent. The change is a pure function, so they are not needed here.
Verdict: Approve.
Signed-off-by: ydflow <314143294+ydflow@users.noreply.github.com> Co-authored-by: ydflow <314143294+ydflow@users.noreply.github.com>
Fixes #1255.
Manual positive feedback can exceed a learning's recall count, making the confidence ratio exceed its 30% weight and the complete score exceed 1. Cap that ratio before weighting, preserving the existing base, recency, feedback counters, and promotion thresholds.
Add regressions for excess feedback with recent, absent, and expired recalls; clarify the score's limits in the bilingual knowledge guide and agent troubleshooting reference. This addresses one existing arithmetic defect related to #408, without adding an evaluation system.
Validation:
npx vitest run src/__tests__/confidence.test.ts(8 passed).npm run build,npx tsc --noEmit,npm run lintpassed.recall feedback --positive lessoninvocations followed byrecall maintenance --confidence-writeback, with the generated votes staged on the fixture reports branch. Before:confidence: 2.2; after:confidence: 1. No remote service or model was used.maintenance-promote.test.ts:137expectingskills/candidate.mdinside a native Windows path. The same test also fails on clean upstreamfa6a361.