Skip to content

Establish scoped Puzzletron v2 quality checks - #2145

Merged
j-rausch merged 6 commits into
feature/puzzletron_v2from
jrausch/puzzletron-v2-ci-quality-baseline
Aug 11, 2026
Merged

Establish scoped Puzzletron v2 quality checks#2145
j-rausch merged 6 commits into
feature/puzzletron_v2from
jrausch/puzzletron-v2-ci-quality-baseline

Conversation

@j-rausch

@j-rausch j-rausch commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: Bug fix and new tests

Pull requests targeting feature/puzzletron_v2 inherit repository-wide lint and documentation failures, which prevents unrelated changes from establishing a useful quality baseline. This PR introduces a scoped quality ratchet without claiming full branch-wide CI recovery.

This change:

  • Runs pre-commit hooks and mypy against the exact files changed from the pull request base for this target, while preserving strict all-file checks for other targets and non-pull-request events.
  • Disables persisted checkout credentials in both quality-workflow checkout paths because later steps do not perform authenticated Git writes.
  • Fixes inherited Sphinx diagnostics at their sources so documentation continues to fail on every new warning.
  • Pins types-docutils in the development lint dependencies so changed documentation configuration is type-checked deterministically in uv-created nox environments.
  • Replaces optimization-removable assertions for required pruning metadata with early ValueError failures.

The test changes are explicit:

  • Removes one local-KD test that duplicated the existing logical data-parallel and expert-parallel overlay assertion. The equivalent behavior remains covered by the retained test.
  • Updates the distributed path-consensus expectation to derive platform-canonical paths with Path.resolve().
  • Moves the existing checkpoint mode import to module scope and completes the module documentation.
  • Adds four focused cases covering a missing FFN module name and each missing attention metadata field.
  • Does not disable or skip any tests.

The public Puzzletron API and configuration surface are unchanged. Invalid internal pruning metadata now fails earlier with ValueError, and types-docutils remains limited to the development lint extra.

Testing

The Code Quality workflow covers the target-specific changed-file pre-commit and mypy route, including the pinned documentation stubs and checkout hardening. The Documentation workflow covers the warning-free Sphinx build. The dedicated Puzzletron v2 CPU workflow covers the adjusted local-KD module and the new pruning-validation cases.

The broader Example and GPU required checks are not made green by this PR. A separate follow-up will address inherited TRL compatibility failures in the gpt-oss and llm_distill examples, retire or replace stale v1 Puzzletron GPU coverage that still calls the deleted entrypoint API, and re-evaluate the external Nemotron import, DeepSpeed numerical, and dataset-download failures.

Additional Information

This PR is intentionally limited to the quality ratchet and focused Puzzletron validation changes. Full branch-wide Example and GPU CI recovery is follow-up work.

Check pull request changes without inheriting repository-wide lint debt, and compare full documentation builds against the established feature-branch diagnostic baseline.

Signed-off-by: Johannes Rausch <jrausch@nvidia.com>
Fix the documentation warnings directly and scope lint enforcement to files changed against the Puzzletron v2 target. This preserves strict checks without inheriting repository-wide formatting debt.

Signed-off-by: Johannes Rausch <jrausch@nvidia.com>
Signed-off-by: Johannes Rausch <jrausch@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The pull request updates changed-file CI checks, Sphinx autodoc behavior and references, Puzzletron pruning validation, source formatting, and automodel unit-test maintenance.

Changes

Quality and Puzzletron maintenance

Layer / File(s) Summary
Changed-file quality checks
.github/workflows/code_quality.yml, noxfile.py, pyproject.toml
Pull requests use changed-file pre-commit and mypy checks. Other events retain all-file checks. The development lint dependencies include pinned Docutils typing information.
Sphinx autodoc and reference updates
docs/source/conf.py, docs/source/guides/*
Sphinx removes generated Puzzletron dataclass docstrings, mocks additional imports, and uses explicit documentation references.
Pruning parameter validation
modelopt/torch/puzzletron/pruning/dynamic_block_prune.py, tests/unit/torch/puzzletron/test_dynamic_block_prune_validation.py
Pruning specification construction validates required FFN and attention metadata. Tests cover each missing field and its reported error.
Puzzletron source cleanup
modelopt/torch/puzzletron/anymodel/capabilities.py, modelopt/torch/puzzletron/mip/grid_budgeting.py
Puzzletron modules receive license and documentation updates. Existing expressions and constraint construction are reformatted without runtime changes.
Automodel unit-test maintenance
tests/unit/torch/puzzletron/test_automodel_local_kd_config.py
Tests remove a duplicate case, relocate an import, add licensing metadata, and reformat assertions, configurations, literals, and path comparisons.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: kevalmorabia97

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed No banned security patterns were added to modelopt/examples Python changes; the only new dependency, types-docutils, declares Apache-2.0 licensing.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: scoped quality checks for Puzzletron v2.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jrausch/puzzletron-v2-ci-quality-baseline

Comment @coderabbitai help to get the list of available commands.

@j-rausch
j-rausch marked this pull request as ready for review August 11, 2026 12:21
@j-rausch
j-rausch requested review from a team as code owners August 11, 2026 12:21
@j-rausch
j-rausch requested review from kevalmorabia97 and removed request for a team August 11, 2026 12:21
@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-11 16:11 UTC

Signed-off-by: Johannes Rausch <jrausch@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.

Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.

👉 Steps to fix this

Actionable comments posted: 1

🧹 Nitpick comments (1)
modelopt/torch/puzzletron/pruning/dynamic_block_prune.py (1)

134-134: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Replace required-parameter assertions with explicit exceptions.

Python removes these assertions with -O. Missing metadata can then reach FFNRemovalSpec or fail later during attention mask creation. Raise ValueError for missing metadata.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@modelopt/torch/puzzletron/pruning/dynamic_block_prune.py` at line 134,
Replace the assert validating down_proj_name with an explicit ValueError when
the required metadata is missing, ensuring validation remains active under
optimized Python execution before constructing FFNRemovalSpec or creating
attention masks.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/code_quality.yml:
- Around line 22-27: Add persist-credentials: false to the with configuration of
both actions/checkout@v6 steps, preserving fetch-depth: 0 in the pull_request
checkout and leaving the existing event conditions unchanged.

---

Nitpick comments:
In `@modelopt/torch/puzzletron/pruning/dynamic_block_prune.py`:
- Line 134: Replace the assert validating down_proj_name with an explicit
ValueError when the required metadata is missing, ensuring validation remains
active under optimized Python execution before constructing FFNRemovalSpec or
creating attention masks.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f48bf9d5-9645-40e5-834e-2e96eb3022fe

📥 Commits

Reviewing files that changed from the base of the PR and between 11d1cee and d8e7110.

📒 Files selected for processing (10)
  • .github/workflows/code_quality.yml
  • docs/source/conf.py
  • docs/source/guides/_basic_quantization.rst
  • docs/source/guides/_pytorch_quantization.rst
  • docs/source/guides/_quant_cfg.rst
  • modelopt/torch/puzzletron/anymodel/capabilities.py
  • modelopt/torch/puzzletron/mip/grid_budgeting.py
  • modelopt/torch/puzzletron/pruning/dynamic_block_prune.py
  • noxfile.py
  • tests/unit/torch/puzzletron/test_automodel_local_kd_config.py

Comment thread .github/workflows/code_quality.yml
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 62.09%. Comparing base (11d1cee) to head (b0a2054).
⚠️ Report is 3 commits behind head on feature/puzzletron_v2.

Files with missing lines Patch % Lines
modelopt/torch/puzzletron/mip/grid_budgeting.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                    Coverage Diff                    @@
##           feature/puzzletron_v2    #2145      +/-   ##
=========================================================
+ Coverage                  53.65%   62.09%   +8.43%     
=========================================================
  Files                        706      707       +1     
  Lines                      91275    91347      +72     
=========================================================
+ Hits                       48978    56720    +7742     
+ Misses                     42297    34627    -7670     
Flag Coverage Δ
examples 29.74% <0.00%> (?)
gpu 23.49% <0.00%> (?)
puzzletron 31.87% <87.50%> (+<0.01%) ⬆️
regression 8.99% <0.00%> (?)
unit 29.61% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Johannes Rausch <jrausch@nvidia.com>
Signed-off-by: Johannes Rausch <jrausch@nvidia.com>
@j-rausch j-rausch changed the title Restore green Puzzletron quality checks Establish scoped Puzzletron v2 quality checks Aug 11, 2026
@j-rausch
j-rausch merged commit bf8f894 into feature/puzzletron_v2 Aug 11, 2026
68 of 75 checks passed
@j-rausch
j-rausch deleted the jrausch/puzzletron-v2-ci-quality-baseline branch August 11, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant