Skip to content

chore(harness): /commit-sweep + nightly automation (#461 A2) - #515

Closed
thomasluizon wants to merge 1 commit into
feature/461-b1-docs-toolsfrom
feature/461-b2-commit-sweep
Closed

thomasluizon wants to merge 1 commit into
feature/461-b1-docs-toolsfrom
feature/461-b2-commit-sweep

Conversation

@thomasluizon

Copy link
Copy Markdown
Owner

Part of #461, bundle 2 (A2).

Stacked on #514 (feature/461-b1-docs-tools). The base auto-retargets to main when #514 merges. Review B1 first.

What landed

  • .claude/skills/commit-sweep/SKILL.md — report-only cross-commit, cross-repo regression sweep. Reads the last N commits (default 10) or --since <when> across main of BOTH repos and diff-reviews them together, applying a six-check incident rubric (migration DDL dup → 42P07, dropped RN transitive pins / Expo-pin generic bump, analyzer-silent-locally ORBIT0001/0002, cross-repo contract drift, cross-platform parity + i18n drift, stale-artifact QA hazard). Emits one severity-ranked report to .claude/audits/commit-sweep.md + a commit-sweep.status sidecar (empty|clean|findings). Never remediates. Roots resolve from ORBIT_UI_ROOT/ORBIT_API_ROOT with local-absolute defaults so it runs identically locally and in CI.
  • .github/workflows/commit-sweep.yml — nightly schedule (cron 0 5 * * *) + workflow_dispatch (commits, scope inputs). Two public checkouts (main at workspace root for skill discovery; thomasluizon/orbit-api into ./orbit-api, both fetch-depth: 0, default GITHUB_TOKEN). Runs anthropics/claude-code-action@v1 with CLAUDE_CODE_OAUTH_TOKEN (the same subscription token claude-review.yml uses — no new secret, no metered key); event-branched prompt (schedule → --since "24 hours ago", dispatch → -n <commits> <scope>). Plain run: steps idempotently create the commit-sweep label, branch on the .status sidecar, upsert a single commit-sweep-labeled tracking issue on findings, and always upload the report artifact + a job-summary line.
  • CLAUDE.md — one pointer line for /commit-sweep.

Harness-only

No product code, no cross-platform mirror, no i18n keys — parity/i18n are N/A. The report + status stay under the already-gitignored .claude/audits/, so the committed surface is exactly the skill, the workflow, and the pointer.

Validation

  • Static: workflow YAML parses; every required grep passes (reuses claude-code-action@v1 + CLAUDE_CODE_OAUTH_TOKEN, wires both window modes, checks out the public sibling, surfaces via gh issue; skill declares --since + the .status sidecar + the report-only guardrail; no em/en dash in authored copy). actionlint has no npm package on this box, so YAML parse is the pre-merge gate; a workflow_dispatch run is the post-merge live proof.
  • LLM-free git mechanics: both git log --since="24 hours ago" and the count-mode window run clean (exit 0) on both repos; the check-1 42P07 grep (both DDL forms, full history) finds real migration DDL. The live /commit-sweep acceptance smoke (which spawns model analysis) was deliberately NOT run — arg-parsing and both window modes were confirmed statically from the SKILL.

🤖 Generated with Claude Code

Add a report-only cross-commit, cross-repo regression sweep over a window
of recent `main` commits in BOTH repos, plus the nightly automation that
runs it and surfaces findings.

- .claude/skills/commit-sweep/SKILL.md: reads the last N commits (default
  10) or `--since <when>` across both repo checkouts, applies a six-check
  incident rubric holistically (migration DDL dup -> 42P07, dropped RN
  pins, analyzer-silent-locally ORBIT0001/0002, cross-repo contract drift,
  parity/i18n drift, stale-artifact QA), and writes one ranked report to
  .claude/audits/commit-sweep.md plus a `.status` sidecar. Never remediates.
  Roots resolve from ORBIT_UI_ROOT/ORBIT_API_ROOT with local defaults.
- .github/workflows/commit-sweep.yml: nightly cron + workflow_dispatch;
  checks out both public repos (main at root for skill discovery, orbit-api
  into ./orbit-api); runs claude-code-action@v1 with the subscription OAuth
  token; event-branched prompt (schedule -> --since 24h, dispatch -> -n N);
  plain run: steps upsert a single commit-sweep-labeled tracking issue on
  findings and always upload the report artifact + a job-summary line.
- CLAUDE.md: one pointer line for /commit-sweep.

Harness-only: no product code, no parity/i18n mirror.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
orbit-ui-mobile-web Ignored Ignored Jul 14, 2026 11:00pm

Request Review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: PR #515 — chore(harness): /commit-sweep + nightly automation (#461 A2)

Scope: PR #515 in thomasluizon/orbit-ui-mobile (base feature/461-b1-docs-tools, stacked on #514)
Recommendation: NEEDS WORK

Summary

This PR adds the report-only /commit-sweep skill (.claude/skills/commit-sweep/SKILL.md), its
nightly + on-demand GitHub Actions workflow (.github/workflows/commit-sweep.yml), and a one-line
CLAUDE.md pointer. It is harness-only — no product code, no apps/*/packages/shared files, no
DTOs — so parity, i18n, contract-drift, DESIGN.md, backend-hard-rules, and FEATURES.md are all
genuinely N/A (confirmed by inspecting the diff, not just taking the PR body's word for it). The
one real defect is a self-inflicted mismatch between the two new files: the workflow's manual
workflow_dispatch path builds a command the skill's own documented argument grammar cannot parse.

Findings

Critical

None.

High

[HIGH] Manual workflow_dispatch invocation sends the skill an -n flag its own argument grammar never defines
· dimension: Correctness (rubric #1)
· location: .github/workflows/commit-sweep.yml:56
· issue: The prompt's command line is:
${{ github.event_name == 'schedule' && '/commit-sweep --since "24 hours ago"' || format('/commit-sweep -n {0} {1}', inputs.commits, inputs.scope) }}
— so a manual run (the workflow_dispatch branch, e.g. the default inputs) sends the skill the literal command /commit-sweep -n 10 both. But .claude/skills/commit-sweep/SKILL.md, added in this same commit, defines the argument grammar as:

  • frontmatter argument-hint: <N=10 | --since <when>> <repo-scope: both|ui|api> — no -n anywhere.
  • Phase 1: "Count mode (default, manual). A numeric first token is the commit count N… Scope (optional trailing token)… A non-numeric, non---since first token is treated as the scope with N defaulting to 10."
    Applied literally to -n 10 both, the first token -n is non-numeric and not --since, so per the skill's own rule it gets swallowed as if it were the scope (an invalid value), leaving the real count (10) and scope (both) unaccounted for — not "N=10, scope=both" as the author intended. The commit message for 0ec984eb itself documents the same wrong belief: "event-branched prompt (schedule -> --since 24h, dispatch -> -n N)", confirming this is an authored inconsistency between the two files added together, not a misreading on my part.
    · risk: Every manual /commit-sweep run via workflow_dispatch — the on-demand entry point this PR advertises ("Runs nightly … and on demand as /commit-sweep") — drives the skill with out-of-grammar arguments. Best case, the executing model infers intent anyway despite the malformed flag; worst case it follows Phase 1 literally and mis-resolves N/scope, silently sweeping the wrong window or wrong repo scope with no error surfaced (the skill has "no adversarial skeptic" per its own Golden Rule, so a wrong window ships as a wrong report with no self-check).
    · fix: Drop the -n flag to match the documented grammar: format('/commit-sweep {0} {1}', inputs.commits, inputs.scope).
    · reference: .claude/skills/commit-sweep/SKILL.md Phase 1 ("Resolve roots + parse args") + argument-hint, both added in this same PR.

Medium

None.

Low / Info

None posted (signal gate — Low/Info are not surfaced on a PR review).

Subagents

All five gated subagents are N/A — this diff touches only .claude/skills/, .github/workflows/,
and root CLAUDE.md; no apps/web/**, apps/mobile/**, i18n JSON, packages/shared/src/types/*,
endpoints.ts, or orbit-api file changed.

Agent Verdict
parity-checker N/A — no apps/web/**/apps/mobile/** file changed
i18n-syncer N/A — no user-facing strings or i18n JSON changed
contract-aligner N/A — no shared types/endpoints/DTOs changed
security-reviewer N/A — no orbit-api code changed
design-reviewer N/A — no apps/*/orbit-landing-page UI files changed

Validation

Not run as a separate step — this review executed against the PR diff; the repo's own required CI
checks (lint/build/test) cover this PR. The changed surface (one skill markdown file, one workflow
YAML, one CLAUDE.md line) has no lint/type-check/test target of its own beyond YAML structural
validity, which was checked manually.

Check Result
Lint N/A — no lintable source changed
Type check N/A — no TS/C# changed
Tests N/A — no test-bearing code changed
Build (api) N/A — orbit-api untouched

Deferred — N/A dimensions & files not verdicted

  • Dimensions 2, 3, 4, 6, 7 (dead code, SOLID, comment policy, type safety, console.log) — N/A:
    no TypeScript/C# source changed; comment-policy lint and type-safety rules don't apply to
    Markdown/YAML.
  • Dimensions 8, 9, 10 (DESIGN.md/AI-slop, parity, i18n) — N/A: no apps/* UI files changed.
  • Dimension 11 (contract drift/backward-compat) — N/A: no packages/shared/src/types/*,
    endpoints.ts, or orbit-api DTOs changed.
  • Dimension 13 (backend hard rules) — N/A: orbit-api untouched.
  • Dimension 14 (FEATURES.md parity) — N/A: /commit-sweep is an internal harness dev-tool, not
    a user-facing feature/Astra/MCP tool/plan-gating/platform surface; FEATURES.md was checked
    (grep) and correctly has no entry for it.
  • Dimension 12 (Security) — reviewed, not deferred: the workflow interpolates inputs.commits
    and inputs.scope (controllable only by someone who already has workflow_dispatch trigger
    rights, i.e. existing write access) into the prompt: field of an action with: block, not into
    a run: shell script — no shell-injection vector, at most a prompt-injection surface bounded by
    the workflow's own --allowedTools allowlist. Judged Low/not privilege-escalating; not posted
    per the signal gate.
  • Files: all three changed files (SKILL.md, commit-sweep.yml, CLAUDE.md) were read in
    full and given a verdict. Nothing changed was left unexamined.
  • Adversarial verification note: an independent skeptic subagent was launched against the one
    High finding per the verification protocol. Its background completion could not be reliably
    retrieved in this run due to sandbox notification-delivery limits, so in its place the finding was
    independently re-verified against primary sources — reading SKILL.md's frontmatter and Phase 1
    text directly from the commit object twice, and cross-checking the commit message for 0ec984eb,
    which itself states the same -n N intent, corroborating the mismatch is real and authored, not a
    misreading. Flagging this deviation from the standard adversarial-subagent flow for transparency.

What's good

  • Harness-only scope is honest and accurate — no product code touched, and the PR body's parity/i18n
    N/A claim checks out on inspection rather than just being asserted.
  • The workflow's checkout/env-var wiring (ORBIT_UI_ROOT = workspace root, ORBIT_API_ROOT =
    <workspace>/orbit-api) matches the skill's Phase 1 root-resolution contract exactly.
  • Report-only guardrails are enforced structurally, not just by convention: the --allowedTools
    allowlist has no gh issue/gh pr write access for the sweep step itself, so even a compromised
    prompt can't post to GitHub directly — only the workflow's own trusted run: steps do that.
  • The .status sidecar branching (empty/clean/findings) is a machine-readable outcome the
    run: steps consume directly rather than re-parsing the markdown report, which is a clean way to
    avoid brittle string-matching.
  • The scheduled (--since "24 hours ago") path is unaffected by the flagged bug — nightly runs will
    work as intended; only the manual workflow_dispatch path is broken.

Recommendation

Fix the one High finding before merge: drop the -n flag from the format() call in
.github/workflows/commit-sweep.yml:56 so the manual-dispatch command matches the grammar
SKILL.md actually documents. This is a one-line change with no ripple effect on the rest of the
PR. Everything else in the diff is clean.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: PR #515 — chore(harness): /commit-sweep + nightly automation (#461 A2)

Scope: PR #515 in thomasluizon/orbit-ui-mobile (base feature/461-b1-docs-tools, stacked on #514)
Recommendation: APPROVE

Note on this review

An earlier automated review on this PR (submitted 2026-07-14T23:12:43Z) requested changes based on
classifying the finding below as High. That classification was posted before this session's
mandatory adversarial verification pass finished running. Verification has since completed — two
independent passes (a dedicated skeptic subagent and a separate full independent re-review) both
confirmed the finding is real but concluded the severity should be Medium, not High, and both
recommended approval. This review reflects that completed verification and supersedes the earlier
one.

Summary

This PR adds the report-only /commit-sweep skill (.claude/skills/commit-sweep/SKILL.md), its
nightly + on-demand GitHub Actions workflow (.github/workflows/commit-sweep.yml), and a one-line
CLAUDE.md pointer. It is harness-only — no product code, no apps/*/packages/shared files, no
DTOs — so parity, i18n, contract-drift, DESIGN.md, backend-hard-rules, and FEATURES.md are all
genuinely N/A (confirmed by inspecting the diff, not just taking the PR body's word for it). All
five gated /pr-review subagents (parity-checker, i18n-syncer, contract-aligner, security-reviewer,
design-reviewer) are correctly N/A for the same reason.

Findings

Critical

None.

High

None.

Medium

[MEDIUM] Manual workflow_dispatch invocation sends the skill an -n flag its own argument grammar never defines
· dimension: Correctness
· location: .github/workflows/commit-sweep.yml:56
· issue: The prompt's command line is:
${{ github.event_name == 'schedule' && '/commit-sweep --since "24 hours ago"' || format('/commit-sweep -n {0} {1}', inputs.commits, inputs.scope) }}
— so a manual run (the workflow_dispatch branch) sends the skill the literal command /commit-sweep -n 10 both. But .claude/skills/commit-sweep/SKILL.md, added in the same commit, defines the argument grammar as:

  • frontmatter argument-hint: <N=10 | --since <when>> <repo-scope: both|ui|api> — no -n anywhere.
  • Phase 1: "Count mode (default, manual). A numeric first token is the commit count N… Scope (optional trailing token)… A non-numeric, non---since first token is treated as the scope with N defaulting to 10."
    Applied literally, -n is a non-numeric, non---since first token, so per the skill's own written rule it would be swallowed as an (invalid) scope value rather than parsed as a count flag. The commit message for 0ec984eb documents the same wrong belief ("dispatch -> -n N"), confirming this is an authored inconsistency between the two files added together in this PR.
    · risk (as verified): downgraded from an initial High to Medium after adversarial review. $ARGUMENTS is consumed by an LLM executor reading the full SKILL.md in context, not a rigid CLI parser — "-n 10 both" is a conventional-enough flag idiom that the executor will very likely still infer N=10, scope=both correctly despite the grammar mismatch. Blast radius is also low: the skill is strictly report-only (writes an audit doc, opens/updates one label + issue; no remediation, no destructive action, no product-code writes), so even a misparse can only affect the count/scope of a single manual sweep, not cause a destructive or silent-failure outcome. The scheduled nightly path (--since "24 hours ago") is unaffected and correctly grammared.
    · fix: drop the flag — format('/commit-sweep {0} {1}', inputs.commits, inputs.scope).
    · reference: .claude/skills/commit-sweep/SKILL.md Phase 1 arg grammar (same PR).

Low / Info

None (signal gate — no nits, style, or "consider…" items).

What's good

  • Clean report-only contract: the skill never remediates or posts destructive changes; the workflow's own run: steps own all GitHub side effects (label, issue upsert, artifact, job summary) — consistent with the existing /audit-* convention.
  • The six-check incident rubric is well-evidenced, each check citing a concrete prior-incident memory key, and the migration-duplication grep robustly covers both structured EF builder calls and raw-SQL DDL forms.
  • ORBIT_UI_ROOT/ORBIT_API_ROOT env-var resolution gives identical behavior locally and in CI; the workflow correctly wires both roots (workspace root + ./orbit-api sibling checkout).
  • --allowedTools is tightly scoped (no gh, no write-capable GitHub tool for the sweep step itself) — consistent with the report-only contract even under a compromised prompt.
  • workflow_dispatch inputs only ever flow into the prompt: field for claude-code-action (never into a run: shell script), so there is no shell-injection vector; issue upsert uses --body-file, no untrusted interpolation.

Deferred / N/A (confirmed by gate, not skipped)

Dead/stale code, SOLID/clean-arch, type safety, console.log, comment policy — no TS/C# product code in this diff. DESIGN.md/AI-slop, cross-platform parity, i18n — no apps/* or i18n files touched. Contract drift / backward-compat guard — no packages/shared/types or DTOs touched. Backend hard rules / security-reviewer / contract-aligner — orbit-api is not checked out in this CI job and is untouched by this diff; not verifiable here, flagged rather than guessed. Validation (lint/typecheck/tests/build) — Build / Unit Tests / SonarCloud run as separate required CI checks per this workflow's CI adaptations; Phase 6 skipped accordingly.

@thomasluizon
thomasluizon deleted the branch feature/461-b1-docs-tools July 15, 2026 11:35
thomasluizon added a commit that referenced this pull request Sep 12, 2026
Stage 8 composes the clock grid on the screen, so neither calendar-time-grid.tsx
carries a direct status fill-token reference any more. The hand-written inventory
still counted one each, so it asserted 56 references against 54 observed.

Both files still exist; only their references are gone.

packages/shared: 88 passed.

Ticket #515 owns deriving this inventory instead of listing it. This is the
fourth pull request it has blocked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lvh6rf1XMFyVJ5hXZoMWhA
thomasluizon added a commit that referenced this pull request Sep 12, 2026
* test(theme): derive bad fill inventory

* test(theme): reject unproven fill roles (#515)

* test(theme): require local bad fill roles (#515)

* test(theme): bind Tailwind role proof to utility (#515)
thomasluizon added a commit that referenced this pull request Sep 13, 2026
* feat(calendar): rebuild week time grid (ORB-50)

* fix(calendar): preserve all-day event copy

* fix(calendar): improve future grid accessibility

* test(shared): prune the time-grid rows stage 8 orphaned

Stage 8 composes the clock grid on the screen, so neither calendar-time-grid.tsx
carries a direct status fill-token reference any more. The hand-written inventory
still counted one each, so it asserted 56 references against 54 observed.

Both files still exist; only their references are gone.

packages/shared: 88 passed.

Ticket #515 owns deriving this inventory instead of listing it. This is the
fourth pull request it has blocked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lvh6rf1XMFyVJ5hXZoMWhA

* fix(calendar): align time-grid dates and geometry (ORB-50)

* chore(shared): regenerate the theme inventory snapshot (ORB-50)

Stage 8 rebuilt the week time grid on both platforms, which changes the
derived status fill-token inventory. Regenerated with
npm run update:theme-inventory -w @orbit/shared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfLBsbGNe7BefVxGNBdCY9

* fix(calendar): use account timezone consistently (ORB-50)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant