Repository navigation
Rework the audit workflow, findings contract, and validators end to end - #12
Merged
Merged
Conversation
Findings contract: findings.json now uses three verdict branches - confirmed, needs_validation, and rejected - with complete traces, bounded local evidence, and severity capped at demonstrated impact. Coverage: reconnaissance seeds a deterministic coverage ledger with canonical collision-checked unit IDs, append-only attempt provenance, explicit per-state field invariants, and prior-run carry rules that revalidate unchanged confirmations and turn deferred, blocked, and out-of-scope prior units into current work. Coverage-critic waves, critic-inclusive budget reservation, and an explicit incomplete-run state replace silent coverage claims. Ten domain companions (AI/LLM, web protocol and auth, client-side, memory safety and binary, cloud and deployment, data isolation and lifecycle, desktop/mobile/local IPC, protocols/RPC/messaging, resource exhaustion, supply chain and release) share one section schema and are copied verbatim into hunter prompts alongside the ordinary attack classes. Safety: target-controlled execution requires an OS-enforced sandbox with no external network, an allowlisted empty environment, resource limits, and scratch-only writes; missing controls retain the lead as needs_validation. Artifact promotion is a single canonical procedure - descriptor-anchored no-follow walks, regular-file and size checks, exclusive creation - stated once in SKILL.md and embedded byte-identical in the hunter and verifier prompts. Audit output defaults outside the target repository. Validators: validate-findings.cjs is hardened (visible-prose vs exact payload separation, platform-unsafe path rejection, canonical Set-based uniqueness, pre-parse structural limits, no-follow input, fatal UTF-8, sanitized capped error output) and a new zero-dependency validate-coverage-ledger.cjs enforces the ledger contract with the same input and output safety. Both ship with test suites (65 tests) and a cross-validator consistency test. Prompting: instruction conflicts resolved, explicit stop conditions and terminal states, exploration depth bounds, schema branches included in subagent prompts with literal return envelopes, and malformed-result handling that preserves verifier independence. Validated end to end with a live budget-limited audit run against an internal repository: both validators passed and the run reported honest partial coverage.
liuchongai1998
approved these changes
Nov 18, 2026
devsart95
pushed a commit
to devsart95/security-audit-skill
that referenced
this pull request
Nov 18, 2026
…w-and-validators Rework the audit workflow, findings contract, and validators end to end
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
One squashed commit covering a full rework of the skill:
findings.jsonuses three verdict branches (confirmed,needs_validation,rejected) with complete traces, bounded local evidence, and severity capped at demonstrated impact.needs_validation. One canonical artifact-promotion procedure, embedded byte-identical in hunter and verifier prompts. Output defaults outside the target repository.validate-findings.cjshardened (prose vs exact-payload separation, unsafe-path rejection, pre-parse structural limits, no-follow input, fatal UTF-8, capped sanitized errors) and new zero-dependencyvalidate-coverage-ledger.cjs, with 65 tests including a cross-validator consistency test.Verification
gpt-5.6-sol: zero provider errors, both validators passed on generated artifacts, and the run correctly self-reported budget-limited partial coverage with named unvalidated fingerprints.