Skip to content

fix(deep-scan): include accepted worker coverage in reports - #1593

Merged
mldangelo-oai merged 39 commits into
mainfrom
mdangelo/codex/deep-source-coverage
Oct 20, 2026
Merged

mldangelo-oai merged 39 commits into
mainfrom
mdangelo/codex/deep-source-coverage

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Retain accepted worker coverage and its original evidence when publishing or recovering Deep Scan results.

Changes

  • Preserve worker ownership, scoped IDs, receipts, and deferred links across ID collisions and selected terminal checkpoints. Parent and worker tasks follow the retained surface through final ID assignment.
  • Restore the parent's surface copy when a surviving task still needs it and a selected worker checkpoint omits it. Keep its worker ownership, and match named or anonymous worker tasks by their verified source identity while preserving parent-only task associations.
  • Reserve receipt spellings for retained selected surfaces and task-required restored surfaces before repairing historical evidence. Include anonymous rows and duplicate legacy IDs without reserving discarded fallback surfaces. Reject receipt references outside the worker artifacts directory.
  • Recover retained receipts from archived attempts, restore missing copies, and preserve available modified copies. Keep unselected partial-checkpoint receipts out of accepted evidence repairs.
  • Prefer the selected parent snapshot, then historical source chronology, when linking or restoring a surface needed by a surviving task; historical bindings cannot overwrite the selected parent association.
  • Return receipt-repair warnings alongside workbench warnings from parent writes.
  • Keep saved coverage projection helpers in the internal workbench package and register them in the plugin file manifest for standalone and installed builds. This also keeps source files within the existing portable-source size limit.
  • Validate retained coverage before checkpoints and preserve valid findings when recovery skips malformed coverage records. Include canceled-worker checkpoint coverage during zero-accepted deadline recovery.
  • Align worker-overflow test diagnostics with the earlier coverage validator while retaining rejection and empty-artifact assertions.

Testing

  • Final Python regression run: 567 passed, six platform skips across 13 CLI, coverage, checkpoint, source-order, publication, and deferred-identity modules.
  • Recovery regressions cover named and anonymous worker-task links, restored worker ownership through collisions, receipt-case conflicts with duplicate legacy IDs, and selected parent evidence taking precedence over older partial snapshots. Fallback-receipt regressions pass in both case orders; controls verify that no-task and existing-worker-replacement cases do not reserve discarded surfaces. Tests verify frozen replay, retained receipts, and unchanged source artifacts.
  • Standard and Diff parent-write regressions fail without warning forwarding and pass with the fix; both the receipt-repair warning and the workbench warning are returned.
  • Four public worker-write/admission cases verify anonymous current receipt precedence, repeated writes, and unchanged archived receipts. These already passed with unchanged TypeScript production code because both public writers normalize surface IDs before reconciliation.
  • All five required portable source checks passed in order, including nine compatibility-checker tests. MCP typechecking, plugin bundle build, formatting, and diff checks passed.
  • Standalone plugin build and installed-plugin integrity tests passed (three tests), including loading the registered internal helper from packaged artifacts.
  • Earlier full local MCP suite: 510 passed, one failed, five skipped. The sole failure is the existing raw-byte filename fixture, which this macOS filesystem rejects with EILSEQ; full local green status is not claimed. After the final recovery fixes and helper extraction, the complete Deep Scan store integration test file passed again.
  • Includes the Windows fixture lifecycle repair from test(ci): wait for Windows sandbox fixture setup #1799, which passed full hosted CI and five repeated Windows configuration runs. Its resulting main commit also passed full CI.

Full hosted CI passed on 4fce8e3bf574f08ec847b0783834589bee92dbe9: 48 jobs succeeded and two conditional jobs were skipped. Codex and Copilot completed current-head reviews, all review discussions are resolved, and maintainer approval is present.

Risk and rollout

Coverage recovery determines which saved review records and receipts appear in a Deep report. Accepted result snapshots remain authoritative; recovery tests verify that original artifacts and parent associations are preserved. Full passing hosted CI and completed reviews remain required before merge.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the bug Something isn't working label Oct 11, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T09:04:24.358658Z 4fce8e3 Manual request
🔒 Security Review ✅ Completed 2026-10-20T09:00:43.935201Z 4fce8e3 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1233936de6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/codex-security/scripts/workbench_saved_results.py
@mldangelo-oai
mldangelo-oai requested a balanced review from Copilot October 11, 2026 17:22

Copy link
Copy Markdown
Collaborator Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-language recovery, receipt reconciliation, and sealing changes warrant final human review.

0 open findings

What changed in this PR

Retains accepted Deep Scan worker coverage through aggregation, resume, stopped-result recovery, and report sealing.

Changes:

  • Aggregates worker-scoped coverage, questions, exclusions, and receipt references.
  • Validates and repairs archived receipt evidence during retries and recovery.
  • Adds extensive coordinator, recovery, identity, and publication regressions.
File Description
plugins/​codex-security/​tests/​workbench_test_support.py Adds worker coverage ID helper.
plugins/​codex-security/​tests/​test_workbench_stopped_worker_coverage.py Tests stopped-scan coverage projection.
plugins/​codex-security/​tests/​test_workbench_standard_deep_results.py Covers reducer coverage fallback.
plugins/​codex-security/​tests/​test_workbench_saved_source_order.py Updates worker-scoped identity assertions.
plugins/​codex-security/​tests/​test_workbench_checkpoint_heads.py Tests scoped checkpoint recovery.
plugins/​codex-security/​tests/​test_saved_deferred_identity.py Expands deferred identity recovery checks.
plugins/​codex-security/​tests/​test_deep_coverage_recovery.py Adds comprehensive coverage and receipt recovery tests.
plugins/​codex-security/​scripts/​workbench_saved_results.py Projects and reconciles worker coverage during recovery.
plugins/​codex-security/​scripts/​finalize_scan_contract.py Detects receipt-path collisions across surfaces.
plugins/​codex-security/​references/​scan-contract.md Documents retained Deep worker coverage.
plugins/​codex-security/​mcp-app/​tests/​test_deep_scan_store_integration.ts Tests end-to-end coverage and receipt repair.
plugins/​codex-security/​mcp-app/​tests/​test_deep_scan_coordinator.ts Tests aggregation, resume, and snapshot behavior.
plugins/​codex-security/​mcp-app/​tests/​test_deep_scan_artifact_validation.ts Tests receipt validation and archived evidence.
plugins/​codex-security/​mcp-app/​src/​deep-scan/​worker-runner.ts Carries validated discovery snapshots.
plugins/​codex-security/​mcp-app/​src/​deep-scan/​coordinator.ts Publishes coverage from reduced discoveries.
plugins/​codex-security/​mcp-app/​src/​deep-scan/​artifact-validation.ts Validates receipts and aggregates coverage.
plugins/​codex-security/​mcp-app/​src/​coverage-publication.ts Adds publication-safe JSON and path validation.
plugins/​codex-security/​mcp-app/​src/​artifact-scan-draft.ts Preserves and repairs archived receipts.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7d92bcad4b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/codex-security/mcp-app/src/artifact-scan-draft.ts
@mldangelo-oai mldangelo-oai changed the title fix: retain worker coverage in Deep Scan reports fix(deep-scan): include accepted worker coverage in reports Oct 12, 2026

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks—the earlier dangling-link reproduction and prior collision/anonymous-receipt probes now pass. This owner-level merge-safety pass found a remaining parent-history regression, detailed inline: recovery can select stale surface metadata and receipts by checkpoint hash instead of retaining the current parent copy.

I independently confirmed the existing legacy duplicate-ID receipt issue. I also reproduced the latest review's updated-task link failure, including an interrupted real worker write: after candidate resolution, the retained task references a missing surface. With a parent ID collision, the related ownership issue instead redirects the updated task to an unrelated surface. The unchanged-task control passes. The parent warning issue is confirmed specifically at the immediate writer API response; valid receipts remain retained. These existing findings are not duplicated inline.

Validation: 105 focused Python cases, 205 Node test entries, 16 writer/admission receipt cases, builds/typechecks, and all five portable checks passed. Additional recovery/finalizer probes reproduce the history and duplicate-receipt regressions against the exact base, plus the updated-task defects with frozen replay and unchanged source bytes. The base itself has a separate raw updated-task link defect, so that comparison is not wholly green. Full local store/deadline integration remains limited by the unchanged trusted-owner guard; full suites and pending hosted jobs are not claimed green.

Comment thread plugins/codex-security/scripts/workbench_saved_results.py
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review 5ae3da9. This update addresses restored worker ownership, updated worker-task surface links, and parent receipt-repair warning forwarding. It also fixes the duplicate-ID receipt reservation issue from Copilot's previous overview: every retained row at the latest selected source order now reserves its receipts before historical repair. The new recovery and warning regressions failed before their fixes and now pass; the broader Python run passed 540 tests with six platform skips.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 5ae3da99c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Anonymous worker tasks can retain dangling surface links when recovery omits a surface still needed by the task.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Anonymous worker recovery mismatches IDs, leaving dangling surface references

plugins/​codex-security/​scripts/​workbench_saved_results.py:2392

A legacy worker task without an id receives an aggregate ID such as first/0, but recovery generates first/saved-…. When a selected checkpoint retains that task but omits its surface, this ID comparison cannot match the recovered task to the parent copy. The restoration loop then leaves surfaceIds: ["first/api"] pointing to a missing surface. Track aggregate tasks through their source and retained output objects, as for surfaces, rather than comparing differently generated IDs. Extend the omitted-surface regression to cover anonymous worker tasks and frozen replay.

Review effort: Balanced

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review 458ed55. It adds the parent-snapshot chronology fix to the previous worker-link, receipt-reservation, and warning-forwarding fixes: selected parent evidence now wins over older matching snapshots during fallback recovery. All four chronology reproductions failed before the fix and now pass; the broader Python run passed 544 tests with six platform skips, and the complete Deep Scan store integration file passed. Please also verify the earlier fixes against this current head.

@mldangelo-oai
mldangelo-oai requested a balanced review from Copilot October 20, 2026 07:28
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Unknown error
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 458ed55522

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-path evidence recovery requires final human review, with unresolved task-binding and receipt-precedence issues.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Reserve restored fallback receipts before historical receipt repair

plugins/​codex-security/​scripts/​workbench_saved_results.py:2431

Task-dependent parent surfaces are restored after _generic_surface_updates reserves receipt spellings and merges historical receipts. If a restored surface references Review.md while another surface's repair retains historical review.md in the same directory, the finalizer discards the restored surface's receipt and changes no_issue_found to needs_follow_up, even when that receipt exists. Include surviving task-dependent fallback surfaces in receipt reservation before historical repair. Add a regression combining parent fallback with a case-colliding historical receipt.

Review effort: Balanced

Comment thread plugins/codex-security/scripts/workbench_saved_results.py Outdated

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks—the earlier fallback chronology, duplicate receipt, updated-task/ownership, and immediate warning reproductions now pass. This review still confirms three issues already reported:

  • Anonymous persisted worker tasks: admission preserves a missing task ID, aggregation assigns first/0, and recovery assigns first/saved-…. When the selected checkpoint omits the surface, the association is lost and publication seals a dangling first/api link. Named-task and typed-writer controls pass. The base retains the original scoped association but also has a separate dangling raw task, so it is not a wholly clean baseline.
  • Selected parent association: with retained partial worker histories, the later historical binding overwrites the selected parent's binding. The sealed task points to old.md; the preceding head and the no-parent-history control keep new.md. This concerns partial checkpoint recovery, not completed discovery admission.
  • Fallback receipt precedence: historical review.md is repaired before the restored current Review.md is reserved. Selected-checkpoint recovery drops the current receipt reference and downgrades no_issue_found; the distinct-name control preserves both. Source files remain unchanged.

Fresh validation: 123 Python tests, three MCP-to-Python recovery cases, eight combined receipt/link controls, targeted publication/frozen-replay probes, builds/typechecks, formatting, and all five portable checks. Another 204 MCP cases, 72 receipt cases, and 16 writer/admission cases were reused with verified unchanged inputs and scope. Full local store/deadline integration remains limited by the unchanged ownership guard; no full typed-writer transaction or all-platform suite pass is claimed. Hosted CI is still pending. Existing findings are not duplicated inline.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review current head 614393613bdb653a19ed3bb7a282beadfceceb53 after the latest recovery fixes. Anonymous persisted worker tasks now retain their source association through ID normalization. Historical partial-worker bindings cannot overwrite the selected parent binding. Receipt repair reserves evidence for genuine surviving fallback surfaces, with controls for discarded surfaces and existing worker replacements. The coverage projection helpers were extracted to satisfy the existing portable-source file-size limit.

The 12-module Python run passed 558 tests with six platform skips. All five required portable checks passed in order, as did MCP typechecking, plugin build, and the complete Deep Scan store integration test file. Full hosted CI is pending on this head.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 614393613b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-language evidence selection and recovery changes warrant final human review and passing hosted CI.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review current head 4fce8e3bf574f08ec847b0783834589bee92dbe9. This follow-up packages the saved coverage helper under the existing internal workbench package and registers it in the plugin file manifest. Installed and standalone plugins can now import it, and the repository's script help contract remains intact. The recovery logic from the previous reviewed head is unchanged.

The 13-module Python run passed 567 tests with six platform skips. All five required portable checks passed in order, plugin build and manifest formatting passed, and the standalone-build/installed-plugin integrity tests passed. Full hosted CI is pending on this head.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 4fce8e3bf5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-language evidence selection and checkpoint recovery affect report integrity and warrant final human review and passing hosted CI.

No open issues 🎉

Review effort: Balanced

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks—no blocking findings in this re-review. Accepted worker coverage, receipt ownership and deferred links survive publication and frozen recovery. The previous association/precedence cases and the helper packaging/help issues are fixed.

Fresh validation passed 146 focused Python tests, three MCP-to-Python cases, nine publication/replay scenarios using the fresh bundle, 39 isolated startup/help checks, builds, types, formatting and all five portable checks. Another 204 unchanged TypeScript cases were reused after verifying input identity. Full local store/deadline integration remains limited by the existing filesystem ownership guard; hosted CI is still running.

@mldangelo-oai
mldangelo-oai merged commit 9cc1a87 into main Oct 20, 2026
111 checks passed
@mldangelo-oai
mldangelo-oai deleted the mdangelo/codex/deep-source-coverage branch October 20, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants