fix(audit): align history contracts and isolate model fixtures - #198
Conversation
The output schema accepted descriptive finding IDs that reconciliation rejected. Match the runtime ID contract and describe coverage using snapshot keys. Verdict fixtures could fall through to installed model CLIs. Use controlled utilities, explicit stubs, and invocation sentinels while retaining CLI tests.
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Organization UI Review profile: QUIET Plan: Advanced Run ID: 📒 Files selected for processing (6)
Comment |
TL;DRThe PR author must acknowledge the current diff and description before requesting review. Author review acknowledgmentAwaiting the PR author's acknowledgment. @DorianZheng: read the current diff and description, then check: Apply boxlite-writing. Does How it works explain the diff's mechanism or rationale? Verify design research against sources; evidence gaps block approval. Use the form best suited to the change. Check drafts too, and repeat this review Unacknowledged PRs are converted to draft. After this check passes, click Ready for review when you want reviews. |
TL;DR
Align auditor history IDs with runtime checks and keep verdict tests independent of installed model CLIs.
Design: #180
Fixes #180. Fixes #149.
How it works
The output schema now constrains finding, disposition, and conflict IDs to the runtime contract. Auditor instructions derive reviewed/unread coverage from the current snapshot keys. Verdict fixtures use controlled utilities and explicit model stubs; sentinels detect unintended CLI calls while deliberate fallback cases retain their own fakes.
Documentation: audit history and test isolation.
Verification
bash plugins/boxlite-agent-tooling/scripts/audit-reconciliation.test.sh→ schema/runtime mismatch with all production changes reverted; passed with the complete fix restored.bash plugins/boxlite-agent-tooling/.agents/hooks/audit/run-verdict-audit.test.sh→ passed with installed CLIs available and with a utility-only PATH lacking model CLIs and ripgrep. Before isolation, the cancellation fixture invoked a sentinel model CLI.