feat(diff): --diff-base flag scopes analysis to git-changed files (closes #157)

[Wolfvin]

Closes #157

Summary

Implements issue #157 — --diff-base <ref> global flag that restricts any analysis command to only report findings from files changed relative to a git ref (branch/tag/SHA/HEAD~1). Designed for CI PR checks where only NEW findings introduced by the PR should appear in the output.

Empty diff → early exit with clear JSON message. Invalid ref → JSON error + exit 1. Works on all analysis commands (secrets, smell, complexity, dead-code, debug-leak, circular, taint, vuln-scan, check, analyze, missing-refs, side-effect, perf-hint, regex-audit, a11y, css-deep, dataflow, stack-trace, config-drift, ownership, test-map).

What changed

New files (2)

• scripts/diff_scope.py — DiffScope utility class (312 lines)
  • from_ref(workspace, ref) factory: validates ref, computes changed-file set via git_aware.get_changed_files() + get_untracked_files()
  • allows(path) — path-in-allowlist check (handles absolute/relative, both \ and / separators)
  • filter_findings(findings, file_keys) — drops findings from unchanged files; tries multiple key names (file/path/defined_in/file_path); keeps findings with no file key (workspace-level findings)
  • summary() — dict for embedding in command output
  • Cross-platform path normalization (handles \ and / on any OS)
• tests/test_diff_scope.py — 44 tests (443 lines)

Modified files (3)

• scripts/codelens.py (122 lines added):
  • Add --diff-base to global parser + every subparser (matches --db-path pattern) so it works both before and after the subcommand
  • Pre-parse loop recognizes --diff-base and --diff-base=<ref>
  • Build DiffScope after workspace resolution; attach to args
  • Empty diff → early exit with clear JSON message + exit 0 (per DoD)
  • Invalid ref → JSON error + exit 1 (per DoD)
  • Post-filter: after command runs, filter findings/leaks/hints/issues/violations/matches/chains/results by changed-file set
  • Attach diff_scope summary (base_ref, changed_files, findings_before_filter, findings_after_filter) to result so CI/agents can see what was filtered
  • Graph-producing commands (scan, trace, impact, circular) are NOT filtered — their results are structural, filtering would corrupt the graph
• SKILL.md — add "Diff-Scoped Analysis" to v8.2 capability pillars with CI usage example
• SKILL-QUICK.md — add --diff-base row to flags table

Acceptance criteria (from issue body)

• --diff-base <ref> flag accepted by all analysis commands — wired into global parser + every subparser
• git diff --name-only output used as file allowlist — via git_aware.get_changed_files() (existing utility, issue [FEATURE] Git-diff aware incremental re-index (replace file-watcher polling) #14)
• Empty diff → early exit with message No changed files relative to <ref> — verified via test_empty_diff_early_exit
• Invalid ref → clear error, non-zero exit — verified via test_invalid_ref_exits_nonzero (exit code 1)
• --diff-base documented in --help output for each affected command — appears in both global --help and subcommand --help
• SKILL.md updated: CI usage example with --diff-base — added with 3-line bash example
• sync_command_count.py --apply run (count unchanged — this is a flag, not a new command) — verified count = 70

How it works

Architecture

User invokes: codelens secrets . --diff-base origin/main
                              │
                              ▼
1. Pre-parse loop captures --diff-base value
2. After workspace resolution:
   a. DiffScope.from_ref(workspace, "origin/main")
      - Validates ref via `git rev-parse --verify`
      - Computes changed files via `git diff --name-only origin/main`
      - Includes untracked files (newly created, not yet git add-ed)
   b. If empty diff → early exit with JSON {"status":"ok","message":"No changed files..."}
   c. If invalid ref → JSON error + exit 1
3. Command executes normally (full workspace scan)
4. Post-filter: diff_scope.filter_findings() drops findings from unchanged files
5. Result includes "diff_scope" summary with before/after counts

Why post-filter (not pre-filter)?

The issue says "pass this set as an allowlist to the existing file discovery layer". This would require modifying every engine's os.walk() loop to accept and check an allowlist. That's 10+ engine files touched → high collision risk with other open PRs (PR #141 touches check.py, PR #148 touches deadcode_engine.py + secrets_engine.py).

Instead, this PR uses a post-filter layer: commands still scan the full workspace, but findings from unchanged files are removed before output. This:

• Touches only codelens.py (one file) — minimal collision risk
• Saves output tokens (the CI/agent concern) even if it doesn't save processing time
• Exposes DiffScope as a reusable utility that engines can adopt incrementally later

The trade-off is documented in the PR description and the diff_scope.py docstring.

DiffScope is reusable

PR #141 (issue #57, open) implements similar functionality for the check command only (--diff-scan, --staged, --diff-vs flags). This PR's DiffScope is a reusable utility class that PR #141 could adopt to replace its per-command flags. No file collision — PR #141 touches check.py + baseline_diff.py; this PR touches codelens.py + new diff_scope.py.

Tests

• New: 44 passed in tests/test_diff_scope.py
  • TestConstruction (7) — direct construction, path normalization
  • TestAllows (6) — path matching, backslash handling, empty/None
  • TestFilterFindings (11) — all file key variants, workspace-level findings, custom keys, no mutation
  • TestSummary (3) — structure, sorting, base_ref
  • TestFromRefRealGit (8) — real temp git repos: no changes, uncommitted, untracked, HEAD~1, branch name (auto-detects main/master)
  • TestFromRefErrors (4) — invalid ref, empty ref, non-git dir, missing workspace
  • TestCliIntegration (5) — subprocess: --help, invalid ref exit code, diff_scope in JSON, empty diff early-exit, before-subcommand
• Regression check: test_formatters + test_cli + test_command_count + test_command_registry + test_git_aware → 129 passed, 0 failed (no regressions)
• Command count: unchanged at 70 (flag, not new command)

End-to-end verification

$ python3 scripts/codelens.py --help | grep diff-base
  --diff-base REF       Git ref (branch/tag/SHA/HEAD~1) to diff against...

$ python3 scripts/codelens.py secrets tests/fixtures --diff-base nonexistent-ref-xyz
{
  "status": "error",
  "command": "secrets",
  "error": "Invalid git ref 'nonexistent-ref-xyz': fatal: Needed a single revision",
  "error_type": "diff_scope_error",
  ...
}
exit=1

$ python3 scripts/codelens.py secrets tests/fixtures --diff-base HEAD~1
[CodeLens] --diff-base 'HEAD~1': 11 file(s) in scope
[CodeLens] --diff-base: 2 finding(s) from unchanged files filtered out
{
  "status": "ok",
  ...
  "diff_scope": {
    "base_ref": "HEAD~1",
    "changed_files": ["scripts/codelens.py", ...],
    "changed_count": 11,
    "findings_before_filter": 2,
    "findings_after_filter": 0
  }
}

$ python3 scripts/codelens.py secrets /tmp/clean-repo --diff-base HEAD
{
  "status": "ok",
  "command": "secrets",
  "message": "No changed files relative to 'HEAD'",
  "diff_scope": {"base_ref": "HEAD", "changed_files": [], "changed_count": 0, ...},
  "stats": {},
  "findings": []
}
exit=0

Findings (per pre-flight SKILL.md — flag to BOS)

• PR feat(ci): baseline diff + strict mode + diff-scan flags (closes #57 phase 1+2) #141 collision alert: PR feat(ci): baseline diff + strict mode + diff-scan flags (closes #57 phase 1+2) #141 (issue [FEATURE] CI/CD integration suite — baseline scan, diff scan, strict mode, manifest tests, golden snapshots #57, open) implements similar functionality for the check command ONLY via --diff-scan, --staged, --diff-vs flags. This PR's --diff-base is a GLOBAL flag for ALL analysis commands. The two are complementary, not conflicting — no file collision (PR feat(ci): baseline diff + strict mode + diff-scan flags (closes #57 phase 1+2) #141 touches check.py + baseline_diff.py; this PR touches codelens.py + new diff_scope.py). PR feat(ci): baseline diff + strict mode + diff-scan flags (closes #57 phase 1+2) #141 could adopt DiffScope in a future refactor to replace its per-command flags. BOS should decide whether to merge both (they coexist fine) or consolidate.

• Post-filter vs pre-filter trade-off: This PR uses post-filter (commands scan full workspace, findings filtered after). Engine-level pre-filtering (passing allowlist to os.walk()) would save processing time but touches 10+ engine files — high collision risk. The DiffScope utility is exposed so engines can adopt it incrementally. BOS may want to create a follow-up issue for engine-level pre-filtering if CI processing time becomes a concern.

Branch

feat/issue-157-diff-base-flag

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[sonarqubecloud]

Quality Gate failed

Failed conditions
D Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE