Skip to content

fix(skill): allow readable interactive verification results - #1836

Merged
mldangelo-oai merged 4 commits into
openai:mainfrom
dipeshbabu:fix/verify-fix-readable-results
Oct 23, 2026
Merged

mldangelo-oai merged 4 commits into
openai:mainfrom
dipeshbabu:fix/verify-fix-readable-results

Conversation

@dipeshbabu

@dipeshbabu dipeshbabu commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #1830. Interactive verify-fix conversations now default to readable Markdown with a verdict, source evidence, and proof gaps for each finding. Explicit JSON requests and automated result contracts retain the existing JSON shape.

Changes

  • Make the result format conditional on the caller's request, consistent with other interactive finding workflows.
  • Preserve the verification method, verdict criteria, explicit invocation requirement, and read-only workflow.
  • Keep CLI and hosted verification on their shared prompt that explicitly requests JSON. No commands, flags, settings, or CLI output contracts change.

Testing

Approved head: 265c3b9c610c5d5ef351979ad9ca39cae4970666.

  • Four real native Codex 0.163 runs, without an imposed output schema, reproduced the old skill returning JSON for an ordinary request and confirmed the revised skill returns Markdown. Explicit JSON retained the exact contract; the unrepaired synthetic target still received still_vulnerable.
  • Fresh Codex 0.164 conversations again passed ordinary-request Markdown and explicit-request JSON. They performed runtime application checks, retained correct verdicts, and left source trees byte-for-byte unchanged. The skill matched the published PR in every run.
  • Real CLI verification, rejected-authentication checks, and Standard/bounded two-round Deep scans passed with both verification fixes applied. Both runtimes completed the scan pipelines and passed sealed-artifact/output-consistency checks. The 0.164 runs were compatibility smoke tests on a smaller fixture.
  • The final integration with main 53da0ecc80df21d69a7a264539f1a2352764fe75 passed the required portable checks in order: Ruff lint, Ruff formatting, SDK build:ci, plugin source compatibility, and all nine source-checker tests. Plugin and SDK builds, types/generated-file checks, formatting, and git diff --check passed.
  • All 118 focused verification/diagnostic/hosted tests passed on that 0.164 integration. The earlier full local SDK run on 0.163 passed 6,956 tests with 83 skips and zero failures (seed 2887100861).
  • Published-head source CI passed: 48 successful jobs, two skipped. Post-merge CI also passed on merge e1dd428df151b669297b4642b59133dfce92a2c8: 49 successful jobs, one skipped.

Risk and rollout

Interactive responses change format. Automated JSON output, verdict criteria, evidence requirements, and the read-only boundary remain unchanged. Live QA ran on Linux; hosted CI also covers Windows and macOS. The Deep QA run was bounded to two discovery rounds, not an exhaustive assessment of the fixture.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the bug Something isn't working label Oct 20, 2026

@mldangelo-oai mldangelo-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reproduced the forced JSON response with the old skill, then tested this version through real Codex conversations. Ordinary requests now produce readable Markdown, an explicit JSON request still matches the existing contract, and fixed and unfixed cases both receive the correct verdict. The source trees stayed unchanged throughout.

I also checked the CLI and hosted callers, ran real Standard and bounded Deep scans with both fixes applied, and validated the final composition with current main. This changes the presentation without weakening the verification or evidence requirements. The small, focused change looks good to merge.

@mldangelo-oai
mldangelo-oai merged commit e1dd428 into openai:main Oct 23, 2026
124 of 125 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 23, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verify-fix forces JSON for interactive verification results even when JSON was not requested

3 participants