Skip to content

fix: allow checks in one attempt to share a promoted artifact - #87

Open
lab1207 wants to merge 1 commit into
cloudflare:mainfrom
lab1207:fix/coverage-ledger-same-attempt-artifact
Open

lab1207 wants to merge 1 commit into
cloudflare:mainfrom
lab1207:fix/coverage-ledger-same-attempt-artifact

Conversation

@lab1207

@lab1207 lab1207 commented Oct 21, 2026

Copy link
Copy Markdown

Summary

validateAttempts registers an archived attempt's promoted artifacts inside the per-check loop:

attempt.local_checks.forEach((check, checkIndex) => {
  ...
  if (typeof check.artifact === "string" && priorArtifacts.has(check.artifact)) {
    errors.push(`... artifact from an earlier attempt cannot be reused`);
  }
  if (check.method === "local" && typeof check.artifact === "string") priorArtifacts.add(check.artifact);
});

So when two checks within the same attempt reuse one promoted artifact, the second check is rejected as reuse from an earlier attempt, even though no attempt boundary was crossed. Per HUNTING.md, a single attempt's owner promotes its artifacts once and multiple checks may reference them; only cross-attempt reuse is forbidden.

Fix

Register an attempt's artifacts only after every one of that attempt's checks has been validated:

const attemptArtifacts = [];
attempt.local_checks.forEach((check, checkIndex) => {
  ...
  if (check.method === "local" && typeof check.artifact === "string") attemptArtifacts.push(check.artifact);
});
for (const artifact of attemptArtifacts) priorArtifacts.add(artifact);

Intra-attempt sharing is now allowed; cross-attempt reuse is still rejected. The current-unit local_checks block already does not register artifacts, so no change was needed there.

Test

Added allows checks within one attempt to share a promoted artifact, which asserts both directions:

  • two local checks in one archived attempt sharing agents/hunter-1/artifacts/result.txt → no errors;
  • the same artifact reused across two attempts → still artifact from an earlier attempt cannot be reused.

Regression proof: with the validator change stashed the new test fails (fail 1); with the change restored the whole file passes (32 tests, 25 pass, 0 fail).

Scope

Only the artifact-registration path in validateAttempts and its test file. Nothing else touched.

Environment

node --test skills/security-audit/validate-coverage-ledger.test.cjs run locally on Node v24.22.0 (lightweight validator run only, no repo build gate). GitHub CI remains the judge.

Model Used

MiniMax M3.1 Flash Free via the opencode agent harness (minimax-m3.1-flash-free)

validateAttempts registered each local check's artifact inside the per-check
loop, so a second check in the same attempt that reused the same promoted
artifact was rejected as reuse from an earlier attempt even though no
attempt boundary had been crossed.

Register an attempt's artifacts only after all of its checks have been
validated, so intra-attempt sharing is allowed and cross-attempt reuse is
still rejected. Add a regression test covering both directions.
coffeegrind123 added a commit to coffeegrind123/security-audit-skill that referenced this pull request Nov 11, 2026
Upstream main has not moved since 2026-09-14, but 79 pull requests are
open. Each was read at the diff level, tested in its own worktree, and
checked for conflicts; the best of each duplicate group is ported.

Validators: cloudflare#143 archived verifier owner reuse, cloudflare#87 shared artifacts
within one attempt, cloudflare#102/cloudflare#103 duplicate JSON keys, cloudflare#111 1,000-field
objects, cloudflare#127 ordinary class must match assignment, cloudflare#104 self-excluded
ordinary block, cloudflare#141 verbatim quotes checked with --target, cloudflare#45 line
upper bound, cloudflare#84 --terminal rejects open units, cloudflare#145 identity-checked
open on Windows (POSIX still fails closed), cloudflare#48 retained-artifact
check (skills/ part only). Overlapping changes were merged so both
behaviours and both test sets survive.

Method: cloudflare#125 critic reasons through handoff, cloudflare#132 deployment controls
become units, cloudflare#83 carried-record gates, cloudflare#93 abort cleanup, cloudflare#23
streaming tool calls, cloudflare#50 config-dependent reachability stays
needs_validation, cloudflare#58 sequential verifier fallback, cloudflare#96 verifier no
longer sees the hunter's verdict, severity or fix.

Execution safety (cloudflare#76 neutralized git, symlink-safe copies; cloudflare#95
sanitized repo ID and prior-run compatibility; cloudflare#108 untrusted-content
prompt block; cloudflare#129 isolation built beside the host) moves into
EXECUTION-SAFETY.md so SKILL.md stays under its token budget.

Companions: cloudflare#101 cryptography, cloudflare#116 Android, cloudflare#35 edge isolate state.

Skipped: duplicates (cloudflare#82 cloudflare#106 cloudflare#91 cloudflare#70 cloudflare#138 cloudflare#131 cloudflare#75 cloudflare#63 cloudflare#97 cloudflare#123 cloudflare#25
cloudflare#26 cloudflare#16 cloudflare#40 cloudflare#42 cloudflare#142), validator-weakening fallbacks (cloudflare#59 cloudflare#113 cloudflare#99),
README/LICENSE churn, rebrands, scope creep, and cloudflare#85 cloudflare#90 cloudflare#8 cloudflare#89 cloudflare#32.

UPGRADE.md records this sync procedure.

Co-authored-by: Rohankollabathula67 <rohankollabathula67@gmail.com>
Co-authored-by: lab 1207 <hajarevaibhav527@gmail.com>
Co-authored-by: Rudy Celekli <rudy@gradiahq.com>
Co-authored-by: Mohit Rawat <rawatmohit17906@gmail.com>
Co-authored-by: MAHER AL-AQIL <18458259+maqalaqil@users.noreply.github.com>
Co-authored-by: CodeInf <jiuqiaoy@gmail.com>
Co-authored-by: pucedoteth <119044801+pucedoteth@users.noreply.github.com>
Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
Co-authored-by: Pizo <pizofreude@proton.me>
Co-authored-by: ZhiHe-ma <302089060+ZhiHe-ma@users.noreply.github.com>
Co-authored-by: shivam <juyaljii610@gmail.com>
Co-authored-by: Dante <duanjl.china@gmail.com>
Co-authored-by: Francisco <francisco@theysay.me>
Co-authored-by: Roman Dolgov <alter@users.noreply.github.com>
Co-authored-by: bodapatisaikrishna <saikrishnabodapati@gmail.com>
Co-authored-by: alvessh <117163985+alvessh@users.noreply.github.com>
Co-authored-by: itinance <hhuebel@itinance.com>
Co-authored-by: ChoYugyeong <cho4639@gmail.com>
Co-authored-by: Shaun Liebenberg <shaun@liebies.co.za>
Co-authored-by: JAI <jai19kharb@gmail.com>
Co-authored-by: ghostzeronz-coder <ghostzeronz-coder@users.noreply.github.com>
Co-authored-by: Aman Mohamed Ali <69119173+spidervirus@users.noreply.github.com>
Co-authored-by: Sandeep.dhami <sandeep84397@gmail.com>
Co-authored-by: jastfan <duohello640@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dante-dan added a commit to Dante-dan/security-audit-skill that referenced this pull request Nov 19, 2026
Combine cloudflare#70, cloudflare#87, cloudflare#91 and cloudflare#106
into cloudflare#82 while retaining each original commit. Use one pass to
validate checks and collect artifacts, then register them once
validation of the attempt is complete. Keep the existing negative
reuse checks and add cloudflare#70's covered-attempt fixture.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant