Repository navigation
fix: validate on-disk existence and integrity of local-check artifacts - #26
Open
anvithsg2004 wants to merge 3 commits into
Open
anvithsg2004 wants to merge 3 commits into
anvithsg2004 wants to merge 3 commits into
Conversation
sheduxdev
added a commit
to sheduxdev/security-audit-skill
that referenced
this pull request
Oct 22, 2026
…integrity of local-check artifacts
sheduxdev
added a commit
to sheduxdev/security-audit-skill
that referenced
this pull request
Oct 22, 2026
…f validating only the first file Conflict in validate-coverage-ledger.cjs resolved by rejecting a second positional path inside parseArgs, since PR cloudflare#84/cloudflare#26 own the CLI argument parsing.
athallajovian9-cyber
added a commit
to JovianOps/security-audit-skill
that referenced
this pull request
Oct 26, 2026
athallajovian9-cyber
added a commit
to JovianOps/security-audit-skill
that referenced
this pull request
Oct 26, 2026
coffeegrind123
added a commit
to coffeegrind123/security-audit-skill
that referenced
this pull request
Nov 11, 2026
Upstream main has not moved since 2026-09-14, but 79 pull requests are open. Each was read at the diff level, tested in its own worktree, and checked for conflicts; the best of each duplicate group is ported. Validators: cloudflare#143 archived verifier owner reuse, cloudflare#87 shared artifacts within one attempt, cloudflare#102/cloudflare#103 duplicate JSON keys, cloudflare#111 1,000-field objects, cloudflare#127 ordinary class must match assignment, cloudflare#104 self-excluded ordinary block, cloudflare#141 verbatim quotes checked with --target, cloudflare#45 line upper bound, cloudflare#84 --terminal rejects open units, cloudflare#145 identity-checked open on Windows (POSIX still fails closed), cloudflare#48 retained-artifact check (skills/ part only). Overlapping changes were merged so both behaviours and both test sets survive. Method: cloudflare#125 critic reasons through handoff, cloudflare#132 deployment controls become units, cloudflare#83 carried-record gates, cloudflare#93 abort cleanup, cloudflare#23 streaming tool calls, cloudflare#50 config-dependent reachability stays needs_validation, cloudflare#58 sequential verifier fallback, cloudflare#96 verifier no longer sees the hunter's verdict, severity or fix. Execution safety (cloudflare#76 neutralized git, symlink-safe copies; cloudflare#95 sanitized repo ID and prior-run compatibility; cloudflare#108 untrusted-content prompt block; cloudflare#129 isolation built beside the host) moves into EXECUTION-SAFETY.md so SKILL.md stays under its token budget. Companions: cloudflare#101 cryptography, cloudflare#116 Android, cloudflare#35 edge isolate state. Skipped: duplicates (cloudflare#82 cloudflare#106 cloudflare#91 cloudflare#70 cloudflare#138 cloudflare#131 cloudflare#75 cloudflare#63 cloudflare#97 cloudflare#123 cloudflare#25 cloudflare#26 cloudflare#16 cloudflare#40 cloudflare#42 cloudflare#142), validator-weakening fallbacks (cloudflare#59 cloudflare#113 cloudflare#99), README/LICENSE churn, rebrands, scope creep, and cloudflare#85 cloudflare#90 cloudflare#8 cloudflare#89 cloudflare#32. UPGRADE.md records this sync procedure. Co-authored-by: Rohankollabathula67 <rohankollabathula67@gmail.com> Co-authored-by: lab 1207 <hajarevaibhav527@gmail.com> Co-authored-by: Rudy Celekli <rudy@gradiahq.com> Co-authored-by: Mohit Rawat <rawatmohit17906@gmail.com> Co-authored-by: MAHER AL-AQIL <18458259+maqalaqil@users.noreply.github.com> Co-authored-by: CodeInf <jiuqiaoy@gmail.com> Co-authored-by: pucedoteth <119044801+pucedoteth@users.noreply.github.com> Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com> Co-authored-by: Pizo <pizofreude@proton.me> Co-authored-by: ZhiHe-ma <302089060+ZhiHe-ma@users.noreply.github.com> Co-authored-by: shivam <juyaljii610@gmail.com> Co-authored-by: Dante <duanjl.china@gmail.com> Co-authored-by: Francisco <francisco@theysay.me> Co-authored-by: Roman Dolgov <alter@users.noreply.github.com> Co-authored-by: bodapatisaikrishna <saikrishnabodapati@gmail.com> Co-authored-by: alvessh <117163985+alvessh@users.noreply.github.com> Co-authored-by: itinance <hhuebel@itinance.com> Co-authored-by: ChoYugyeong <cho4639@gmail.com> Co-authored-by: Shaun Liebenberg <shaun@liebies.co.za> Co-authored-by: JAI <jai19kharb@gmail.com> Co-authored-by: ghostzeronz-coder <ghostzeronz-coder@users.noreply.github.com> Co-authored-by: Aman Mohamed Ali <69119173+spidervirus@users.noreply.github.com> Co-authored-by: Sandeep.dhami <sandeep84397@gmail.com> Co-authored-by: jastfan <duohello640@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #21
Summary
Previously,
validate-coverage-ledger.cjsonly validated the lexical format and ownership prefix (agents/<agent-id>/artifacts/...) oflocal_checks[].artifactpaths. It did not verify whether referenced artifacts actually existed on disk in the retained output directory.As a result, units marked as
covered,candidate, orblockedcould pass validation while pointing to phantom or non-existent local check evidence.This PR extends
validate-coverage-ledger.cjsto enforce strict on-disk validation of all local-check artifacts, verifying their existence, file type, symlink status, and directory containment, while adding a comprehensive regression test suite.How We Solved It
On-Disk Artifact Validation (
validateArtifactFile):fs.lstatSync(without following symlinks) to verify the artifact file exists. If missing (ENOENT/ENOTDIR), it produces:$[...].artifact: local check artifact does not existstat.isSymbolicLink():$[...].artifact: local check artifact must not be a symlink!stat.isFile():$[...].artifact: local check artifact must be a regular filefs.realpathSyncof the artifact against the real path of the agent's artifacts directory (baseDir/agents/<agent_id>/artifacts/). If directory-level symlinks or path traversal attempt to escape this subtree, it produces:$[...].artifact: local check artifact must reside inside agent artifacts directoryPipeline Option Plumbing & Historical Checks:
validateChecks(value, location, errors, options = {})to trigger filesystem checks wheneveroptions.baseDiris provided and lexical checks pass.optionsintovalidateAttemptsso that archived history inunit.attempts[].local_checks[]also enforces artifact integrity.collectUnitErrorsandvalidateDocument(ledger, options = {})to thread options throughout the document traversal.CLI Directory Resolution & Backward Compatibility:
run()),baseDirautomatically defaults to the ledger file's parent directory (path.dirname(path.resolve(file))), which naturally aligns with the<output-dir>/coverage-ledger.jsonlayout used throughout the audit workflow.--output-dir <path>(and--base-dir <path>) supporting both space-separated and--output-dir=<path>syntax.--schema-onlyflag to allow purely structural validation when desired.validateDocument(ledger)calls withoutbaseDirschema-only, preserving backward compatibility for existing in-memory tests.Comprehensive Test Suite (
validate-coverage-ledger.test.cjs):Added extensive positive, negative, and edge-case tests:
$[0].local_checks[0].artifact: local check artifact does not exist).must be a regular file).attempts[]are validated on disk.$[N].local_checks[M]).1on missing artifact,0on valid artifact,--output-dircustom resolution, and--schema-only).Verification
All existing and newly added tests pass cleanly: