[hz89Dev]

Summary

Score labels from sessions and workflow runs are published to teamai-reports, but nothing reads them. A team cannot see how its tools do on real work, or whether a change to its skills helped. The dashboard now has a Scorers page. It shows pass rates one measurement at a time, and only sums labels that measure the same thing.

Scorers page
├── scorer table            # each scorer opens on its newest committed rubric; sparkline per scorer
├── controls                # window 7/30/90/180 days, group by, split at a team commit
├── measurements            # one chip per scorer + rubric + source + run mode + judge, grouped by rubric
├── headline + split cards  # pass rate, passed, failed; before / after / not recorded with the delta in points
├── labels by value         # each stored value with its score and count, so precision and recall can be read (#1472)
├── weekly trend            # UTC weeks; a week with no labels is a gap, never 0%
├── groups                  # tool, model, team revision, agent definition, role, project, tag, fingerprint, skill
├── failing labels          # newest first, 25 at a time up to 1,000; filter by group; resume command or the run
└── friction reminder       # this machine only: sessions the reminder fired on vs not, last 30 days at most

GET /api/scorers?window&scorer&cohort&groupBy&commit&group&failing
  scorerResults (src/dashboard/scorers.ts)     reads labels, run records, archive names, team clone, reminder state
    ScorerResults.of (src/scorers/results.ts)  pure counting, reusable by a JSON digest

teamai contribute now keeps { hinted } for 30 days when it sweeps a session's state file. Without that record, the reminder table would lose every session older than a day.

Since the review

Finding	Change
P1: the friction reminder table was empty in a user scope install	The user scope workspace now counts the sessions that started outside every project workspace on this machine (ownSessionEvents). A project workspace still counts only its own sessions.
P2: every click re-read the whole reports branch	Team reads (published labels, archive names, run records) are kept until the reports checkout or the team clone moves to a new commit. This machine's labels, run files and reminder state are read on every click. The weekly trend buckets labels once, not once per week. Failing labels are paged.
P3: swept records were kept for other hooks' per-session files	Only files that parse as a contribute state are swept into a record, and only records with hinted are read back.

Also new: Labels by value (#1472), on selected.cohort.values, and the phone layout for its table and the reminder table.

Cost per click on a fixture with 90,000 labels and 18,000 run records (server CPU time per request; wall time was not usable on a loaded machine):

	before (48393701)	after (last run at eb8fd443)
first load	4.3-7.9 s, like every click	4.4-5.3 s (5.25 s)
each later click	4.3-7.9 s	0.44-0.85 s (0.49-0.77 s)
response size	0.57-4.4 MB	14-30 KB
resident memory	1.0-1.2 GiB	1.0-1.2 GiB (1.19 GiB), unchanged

New screenshots:

Desktop	Desktop, dark

Mobile (390px)	Mobile	Mobile, zh-CN, user scope

Screenshots from the first round (fixture team, real teamai dashboard):

Desktop	Desktop

Mobile (390px)	Mobile	Mobile	Tablet (820px)

No horizontal page overflow at 390px or 820px. The four-column tables fit the phone width; wider tables scroll inside their panel.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature causing existing behavior to change)
• Documentation only
• Refactor / internal cleanup

Test Plan

• npx tsc --noEmit passes
• npm run lint passes
• npx vitest run: the scorer, dashboard and contribute tests pass (51 in the five files this branch adds to). A full local run on a heavily loaded machine (load average 120-228) had 13-20 failures in timing-dependent workflow and usage tests this branch does not touch, and the set changed between runs. CI at eb8fd443 passes the full suite on Node 20, 22 and 24 on Ubuntu and macOS, plus Windows and the three E2E shards.
• Added/updated tests for the change

New tests:

scorers-results.test.ts
  opens on the newest committed rubric; never sums source, judge model or run mode
  counts a judge model under one measurement whatever its alias; shows the newest alias
  counts a rescored label once; reads pass/fail as stored
  groups: reported vs requested models, pre-#1176 agents, role, skill (none / not recorded)
  keeps a value named like none apart from labels that recorded none
  splits at a team commit, also within each group
  lists a page of failing labels with their total, and every failing label of a group however many newer ones other groups have
  counts a measurement by label value, highest score first; a rescored label once, under its newest value
  reminder table over labelled sessions only; a week without labels is a gap
dashboard-scorers.test.ts (real git team clone with commits A, B, C)
  joins a workflow label to its pass agent; leaves out a benchmark run, also when its run file is local
  splits with the clone's ancestry; refuses a commit the clone lacks (400)
  names the pushed session; no reasoning, transcript, prompt, native session id, isolation or access in the answer
  reminder table from this machine's member sessions; cut to the 30 days decisions are kept
  counts labels a new reports commit adds, and this machine's own labels whenever they change
  still answers from this machine's labels when the team clone is gone
  reads the failing page and group; refuses values the view does not offer
dashboard-workspaces.test.ts
  gives the user workspace the sessions no project owns, and a project only its own
contribute-reminder-record.test.ts
  fired / not fired survive the 24h sweep; a resumed session swept again stays fired; forgotten after 30 days
  keeps no reminder decision for another hook's per-session file, swept or not
dashboard-ui.test.ts
  the page is wired, and every label it shows has a zh-CN translation

Real CLI record at head eb8fd443 (teamai dashboard built from this branch, same fixture team plus a routing scorer; the user scope run uses a --scope user install of the same team):

$ curl -s '.../api/scorers?workspace=<id>&window=30&scorer=teamai:routing:ops/release'
values fit 1 31, idle 1 112, extra 0 6, missed 0 9 | n 158 passed 143

$ curl -s '...&scorer=review-quality'                                 failing 8 of 8
$ curl -s '...&scorer=review-quality&groupBy=skill&group=<review-diff>'  failing 2 of 2 in review-diff
$ curl -s '...&failing=5000'
{"error":"Scorer results take window 7, 30, 90, 180, a grouping the page offers, a commit as hex digits, and up to 1000 failing labels."}

# user scope install, workspace=user
reminders {days: 30, yes 2 sessions 2 labelled 100%, no 16 sessions 14 labelled 93%, not recorded 5 sessions 4 labelled}
# before the fix every row was 0

In the browser, on the 90,000 label fixture: Show more went 25, 50, ... 1,000 of 3,873, then the button gave way to "Pick a group or a shorter window to list the rest." with no error.

First real CLI record (teamai 0.22.0 built from this branch, head 41256ff3). The fixture team has two scorers. It holds 222 session labels across 3 members, 52 workflow labels joined to 26 run records (one of them a benchmark), and one session pushed to the archive:

$ teamai init https://git.example.invalid/acme/team.git --provider git --scope workspace --agent claude --force
$ teamai dashboard --port 3799

$ curl -s 'localhost:3799/api/scorers?workspace=<id>&window=90'
{"scorers":[{"scorer":"diff-quality","cohorts":1,"default":{"n":50,"passed":39,"failed":11,"rate":0.78}},
            {"scorer":"review-quality","cohorts":8,"default":{"n":126,"passed":93,"failed":33,"rate":0.738}}]}
# diff-quality n=50: the benchmark run's 2 labels are left out (52 with them)

$ curl -s '...&window=90&scorer=review-quality&groupBy=skill&commit=6f7d35c'
split   before 69 (65%)  after 57 (84%)  not recorded 0
groups  debug-failure 31 (74%), review-diff 29 (83%), write-tests 29 (76%), changelog 26 (65%), ui-polish 24 (67%), none 15 (73%)
failing 33, pushed: fix-login-redirect --platform codex
reminders {days: 30, yes 2/2, no 16 sessions 14 labelled 93%, not recorded 5}

$ curl -s -w '%{http_code}' '...&commit=deadbeef'
{"error":"Commit deadbeef is not in this machine's team clone. Run teamai pull, or give a commit of the team repo."} 400

An independent check against the same server recomputed every count from the fixture files. It confirmed that none of the 222 session ids in the fixture appears in any of 33 responses. It also confirmed that no response carries isolation, access, reasoning or prompt text.

In the browser:

• A scorer click followed by a window and grouping change, made before the first response, lands on the newest selection.
• A refused commit keeps the typed text, and the last good split stays on screen.

Not covered locally: providers gitlab and github, and agents other than Claude. They are left to CI. The page reads teamai-reports the same way the Workflow Runs page does.

Related Issues

Closes #1182
Closes #1472

Notes for Reviewers

• What counts together. A measurement fixes the scorer, rubric.sourceHash, source, run mode and judge (kind, tool, model; reported and requested-only apart). The judge's team alias is shown but does not split a measurement. The label parser refuses a label without rubric.sourceHash or judge, so the page has no rubric not recorded or judge not recorded cohort.
• Commit split. The split uses git merge-base --is-ancestor in this machine's team clone, at most 8 at a time. A revision the clone does not have counts as not recorded, never as before.
• Benchmarks. Labels of a run whose published record has benchmark are left out, also when the run file is on this machine. Nothing writes benchmark before [feat] Run benchmark suites as workflows #1180, so today this only guards future records.
• Failing labels show the archived session name, which the issue asks for, plus --platform, which teamai session resume requires. They never show the native session id. teamai score --team-session does not exist yet, so the row offers resume only.
• Reminder table. It is local to this machine and covers the window, cut to the 30 days teamai contribute keeps decisions for. A session counts as fired if its live state or its swept record says so.
• Team read cache. Kept per reports checkout and keyed on the commits of the reports checkout and the team clone. Nothing is kept when either has no commit or is gone, or for http teams, whose sync rewrites reports without a commit. A session archive index written without a commit shows its name after the next commit or pull.
• Unpublished labels. This machine's own labels are counted with the published ones, as the run history does with local runs.

[github-actions]

• [P2 non-blocking] Preserve unsubmitted commit text across renders — src/dashboard/scorers-client.ts:104. Change the window or grouping, then type a commit before that request finishes: its response calls render(), rebuilding the input from scorerCommitDraft, which is updated only on submit. The typed SHA is erased or reverted. Update the draft on input and preserve focus when rendering.

The PR description documents sufficient testing, including a built real-CLI run at 41256ff3. Additional provider/agent coverage is explicitly deferred to CI; no testing finding.

Review was read-only; no PR code was run, built, or installed.

[hz89Dev]

Fixed in f72f689.

• The commit field now updates its draft on every input event, so a render keeps whatever has been typed so far, not only the submitted value.
• render() now puts focus and the caret back on a focused field inside the page view, the same way it already restores focus on session and run rows.

Checked in a real teamai dashboard: I changed the window, then typed 765 into the commit field before the response came back. After the response re-rendered the page, the field was a new node that still held 765, kept focus, and had the caret at 3.

[github-actions]

No new findings in the scoped PR diff.

• Resolved: The earlier commit-draft finding is fixed: input events update scorerCommitDraft, and rendering restores field focus and caret.
• Testing: The description includes sufficient testing and a built real-CLI verification record at 41256ff3. The older recorded head is not blocking; additional provider/agent coverage is explicitly deferred to CI.

Review was read-only. No PR code was run, built, or installed.

[SaulMoro]

This was generated by AI during triage.

@hz89Dev Thanks, this is a substantial piece of work. We ran teamai dashboard from the built CLI at f72f689 against a fixture team with session and workflow labels, and worked out every count by hand first:

• Criteria 1–13 of #1182 hold.
• main answers 404 for /api/scorers, and the new tests fail there.
• Bad parameters get 400, and the server listens on 127.0.0.1 only.
• None of the answers carries reasoning, transcript text or the native session id.
• f72f689 fixes the CI review's one finding, the typed commit erased on re-render.

One change before this can be approved:

• [P1 blocking] The friction-reminder table is always empty in a user-scope install. handleScorers passes scopedEvents(workspace.id) (src/dashboard.ts:320). workspaceEvents (src/dashboard/workspaces.ts:107-117) puts each session in a project workspace or under unassigned, never under user. So with teamai init … --scope user, GET /api/scorers?workspace=user&… returns 0 sessions in every reminder row. The same events and reminder state under a workspace-scope install give yes 2, no 3, not recorded 2. The "Unassigned sessions" workspace, which does hold those sessions, answers configured: false. Your record used --scope workspace, which is why it doesn't show. The reminder state is machine-wide, so the user workspace could take its sessions from the events no project workspace owns. Otherwise, omit reminders there and say so in docs/guide/dashboard.md, whose sentence about the table is wrong for user scope today.

Non-blocking:

• [P2 non-blocking] Cost on a large team. Every change on the page re-reads the whole reports branch and returns every failing label. With 90k labels and 18k run records, each click took 2.0–4.6 s, answers reached 4 MB, and the server used about 980 MiB. The Workflow Runs page takes 1.5 s on the same data. Caching the read per branch commit and paging the failing labels would bound both.
• [P3 nit] The 30-day sweep record in src/contribute-check.ts is also written for the other per-session files in ~/.teamai/sessions/, so a stale TodoWrite-hint file is recorded as "reminder fired". The table doesn't read those today, but #1388 is meant to reuse this record.

Overlap: #1379 conflicts only on the last paragraph of both scorers.md pages. #1414's conflict in src/dashboard.ts exists against main too.

CI review: we agree with its finding, which f72f689 fixes.

Verified at f72f689: tsc and lint pass. 516 unit tests and the e2e files pass, also under a symlinked TMPDIR. CI has no failed job. Not run here: a browser (we checked the API and the client code), and the real contribute-check hook reaching the sweep (we called the same function).

@jeff-r2026 This adds a new dashboard page and API, so it's yours to decide.

[github-actions]

No new findings in the scoped PR diff.

• Resolved: The earlier commit-draft issue is fixed: input events update scorerCommitDraft, and rendering restores focus and caret.
• Testing: The description documents sufficient testing, including a built real-CLI verification at 41256ff3. The older recorded head is not blocking; additional provider/agent coverage is explicitly deferred to CI.

Review was read-only. No PR code was run, built, or installed.

[github-actions]

No new findings in the scoped PR diff.

• Resolved: The earlier commit-draft issue is fixed: input events update scorerCommitDraft, and rendering restores focus and caret.
• Testing: The description includes sufficient testing and a built real-CLI verification record at 41256ff3. The older recorded head is not blocking; additional provider/agent coverage is explicitly deferred to CI.

Review was read-only. No PR code was run, built, or installed.

[hz89Dev]

All three are fixed at eb8fd443. The branch is rebased on main.

• [P1] Empty reminder table in a user-scope install. The fix is in 5a180be. The user workspace now takes the sessions that no project workspace owns (ownSessionEvents, src/dashboard/workspaces.ts). A project workspace still counts only its own sessions. I tested it with the real CLI on a --scope user install, with the same events and reminder state as the workspace-scope fixture. Before the fix every row was 0. After it: yes 2 sessions (2 labelled, 100%), no 16 (14 labelled, 93%), not recorded 5 (4 labelled). Both dashboard.md pages now say which sessions each scope covers.
• [P2] Cost on a large team.

  • Team reads are cached per reports checkout. That covers published labels, archive names and run records. The cache key is the commits of the reports checkout and the team clone, and that change is ac09724. This machine's labels, run files and reminder state are still read on every request.

  • The cache is skipped when either checkout has no commit or is missing, and for http teams. That change is eb8fd44.

  • The weekly trend now buckets labels once instead of once per week (47f064c). That was the largest cost in the profile.

  • Failing labels are paged: 25 per request, at most 1,000, with group and failing query parameters (bf21e24, 3d492e7).

  • Measured on a fixture with 90k labels and 18k run records, as server CPU per request:

    	before	after
    each later click	4.3-7.9 s	0.44-0.85 s
    first load	4.3-7.9 s	4.4-5.3 s
    response size	up to 4.4 MB	14-30 KB
    resident memory	1.0-1.2 GiB	1.0-1.2 GiB

    The first load and the memory did not improve.

• [P3] Sweep record written for other hooks' files. The fix is in 69c7d3e. The sweep only writes a record for a file that parses as a contribute state, which means it has contributed. Only records with hinted are read back. A new test covers a stale <sid>-todowrite-hint.json, both swept and not swept.

The PR also now closes #1472 (labels by value, under selected.cohort.values). The PR body has updated screenshots at 390px, 820px and desktop, and the real-CLI record at the new head.

[github-actions]

No new findings in the scoped PR diff.

• Resolved: The earlier commit-draft issue is fixed: input events update scorerCommitDraft, and rendering restores focus and caret.
• Testing: The description documents sufficient testing, including a built real-CLI verification at 41256ff3. The older recorded head is not blocking; additional provider/agent coverage is explicitly deferred to CI.

Review was read-only. No PR code was run, built, or installed.

[SaulMoro]

This was generated by AI during triage.

Re-review at eb8fd44. All three findings from our last review are fixed, and nothing new blocks. Whether the new dashboard page and its read API go in is the maintainer's call.

Earlier findings

Finding	Status
[P1] Empty friction-reminder table in a user-scope install	Fixed. Built CLI, user scope: yes 2/2/1, no 3/2/2, not recorded 2/2/0 (all 0 at f72f689). Workspace scope is unchanged. The "not recorded" labelled count differs between scopes because the local label lives only in the user scope's data home; that is correct.
[P2] Cost on a large team	Fixed for clicks and answer size. Same 90k-label fixture, old and new builds run in turn: repeat clicks 0.7-3.5 s against 6.8-19.5 s, answers 11-25 KB against up to 4 MB. First load and memory (about 900 MiB) are unchanged, as the body says.
[P3] Sweep record written for other hooks' files	Fixed. Through the real contribute-check hook, only the contribute state file gets a record.

New, non-blocking

• [P3 nit] The new cache of the team's labels lives as long as the server, so its memory is not released. The body says so.

The read API now takes group and failing, and its answer changes shape (failing is {labels,total}, new cohort.values). Both the API and the page are new in this PR, so nothing existing breaks. The PR also closes #1472 (labels by value), and the counts add up to the label count in the real-CLI answers.

Verification at eb8fd44

• npm ci, npm run build, npx tsc --noEmit, npm run lint exit 0.
• The PR's unit files (51 tests, also with a symlinked TMPDIR) and their callers (389 tests) pass. The e2e files that reach the changed code pass; the full e2e suite was not rerun.
• Every CI job passed at this head. The body's real-CLI record is at eb8fd44 and includes a user-scope run.

Not run here: the page in a browser, the gitlab and github providers, agents other than Claude.

[SaulMoro]

This was generated by AI during triage.

Approving at eb8fd44. The review is in our comment above: the three earlier findings are fixed and checked with the built CLI, CI is green, and the body has a real-CLI record at this head. The one nit left (the team-label cache is never released while the server runs) is already noted in the body.