Skip to content

Python: add public API compatibility checks - #8271

Draft
Eduard van Valkenburg (eavanvalkenburg) wants to merge 2 commits into
microsoft:mainfrom
eavanvalkenburg:python-api-compatibility
Draft

Python: add public API compatibility checks#8271
Eduard van Valkenburg (eavanvalkenburg) wants to merge 2 commits into
microsoft:mainfrom
eavanvalkenburg:python-api-compatibility

Conversation

@eavanvalkenburg

Copy link
Copy Markdown
Member

Motivation & Context

Python packages currently have no automated check that surfaces accidental breaking changes to released public APIs. This adds a non-blocking PR signal comparable to the existing .NET API compatibility validation while respecting Python package and feature lifecycle stages.

Description & Review Guide

  • What are the major changes? Add a trusted pull_request_target workflow that uses Griffe to compare released Python package APIs with the PR base, excludes Lab, prerelease packages, and APIs already marked @experimental, and recognizes the existing breaking change label as an explicit acknowledgement. Pin Griffe with the Python development dependencies and document the workflow.
  • What is the impact of these changes? Python PRs receive GitHub annotations and a job summary for detected public API breakages. The job is non-blocking, does not run for .NET-only changes, and does not import or install code from the PR checkout.
  • What do you want reviewers to focus on? Please review the pull_request_target trust boundary, base-commit lifecycle/decorator filtering, and whether the breaking change label is the right acknowledgement mechanism.

Related Issue

N/A — this draft explores the requested Python API compatibility workflow and is not linked to an existing issue.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d1fa3e1a-596f-446f-b77f-5c84ec1eaa38
Copilot AI balanced review requested due to automatic review settings September 10, 2026 18:53
@agent-framework-automation agent-framework-automation Bot added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python labels Sep 10, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow exposes a pull_request_target code-execution path and mishandles complete module removal.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds advisory Python public API compatibility checks using Griffe.

Changes:

  • Adds a pull_request_target compatibility workflow and checker.
  • Filters unreleased and experimental APIs.
  • Pins Griffe and documents the process.
File summaries
File Description
.github/scripts/check_python_api_compatibility.py Implements API comparison and reporting.
.github/workflows/python-api-compatibility.yml Runs the advisory PR check.
CONTRIBUTING.md Documents compatibility policy.
python/pyproject.toml Pins Griffe for development.
python/uv.lock Locks Griffe dependencies.
Review details
  • Files reviewed: 4/5 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/python-api-compatibility.yml Outdated
Comment thread .github/scripts/check_python_api_compatibility.py
Comment thread .github/scripts/check_python_api_compatibility.py Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 0c5ca4bfd018
Model: gpt-5.6-sol-fast

Overview

The workflow has strong trust-boundary guardrails: it runs pinned actions with read-only checkout credentials, executes the checker and dependency pin from the trusted base, and disables Griffe inspection. However, released-package removal aborts structured reporting, stale branches are compared against the current base as if they removed newer APIs, and the acknowledgement signal can be generated from an author-controlled title. These gaps make the advisory result incomplete or misleading in concrete PR scenarios.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
3 verified findings remained after source verification (3 medium) across 2 files. Details are attached to the affected lines below.

Affected areas: .github/scripts/check_python_api_compatibility.py, .github/workflows/python-api-compatibility.yml

Comment thread .github/scripts/check_python_api_compatibility.py Outdated
Comment thread .github/workflows/python-api-compatibility.yml Outdated
Comment thread .github/workflows/python-api-compatibility.yml Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d1fa3e1a-596f-446f-b77f-5c84ec1eaa38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants