Skip to content

fix(backend): isolate NumPy parity behind reference API - #1070

Merged
seonghobae merged 40 commits into
mainfrom
fix/reference-backend-boundary-626
Aug 24, 2026
Merged

seonghobae merged 40 commits into
mainfrom
fix/reference-backend-boundary-626

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

What changed

  • Keep production FitConfig and CLI backend choices Rust-owned (rust/auto) and fail closed when Rust is unavailable.
  • Add explicit fast_mlsirm.fit_reference and fit --reference parity surfaces for NumPy.
  • Update the Rust-ownership ADR, doctoring, README, changelog fragment, and regression tests.

Why

Closes the remaining #626 contract gap without silently breaking parity callers: NumPy remains callable only through a named reference path, while ordinary production configuration cannot select it.

Validation

  • python -m pytest -q tests/test_backend.py tests/test_cov_b_backend_config.py tests/test_config.py tests/test_cli.py tests/test_cov_f_fit.py tests/test_full_information_factor_stability.py tests/test_irt_contract.py tests/test_security_hardening.py
  • Full related regression group including tests/test_paper_features.py: 762 passed
  • ruff check on changed production and focused test files
  • Reference-boundary smoke assertion

Summary by CodeRabbit

  • New Features

    • Added an explicit fit_reference API and --reference CLI mode for NumPy parity and research workflows.
    • Production fitting now uses Rust through rust or auto backend options.
    • Added documented Rust device selection for production runs.
  • Bug Fixes

    • Production fitting now fails closed when the Rust backend is unavailable instead of silently falling back to NumPy.
    • NumPy is rejected as a production backend configuration.

Open in Devin Review

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head review request for 833a0801c7f22cf61e6ca5f23fb4a8a3fa95c4f8.

Please review this exact head:
@opencode-agent review
@cwl-noema-review review

This is the scoped #626 contract slice: production configuration and CLI cannot select NumPy; parity remains available only through the explicit fit_reference/--reference path.

@coderabbitai

coderabbitai Bot commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f65f0b8a-84e6-4e8c-a858-f6d2b9cb6a03

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Rust/PyO3 is now the required production numerical backend. NumPy fitting is available only through explicit fit_reference and fit --reference paths. Backend validation, public exports, CLI dispatch, documentation, and tests enforce this boundary.

Changes

Backend ownership boundary

Layer / File(s) Summary
Production backend contract
python/fast_mlsirm/backend.py, python/fast_mlsirm/config.py, python/fast_mlsirm/fit.py, python/fast_mlsirm/objective.py, python/fast_mlsirm/estimators/mmle.py, tests/test_backend.py, tests/test_config.py, tests/test_cov_b_backend_config.py, README.md, docs/PRD.md, docs/adr/..., docs/doctoring/..., docs/buyer_demo_storyboard.md, docs/changelog.d/..., docs/commercial_readiness.md
Production backend validation now accepts only rust and auto. Rust availability remains fail-closed. NumPy resolution requires the explicit reference scope. Documentation describes the same backend boundary.
Reference fitting path
python/fast_mlsirm/_fit_public.py, python/fast_mlsirm/reference.py, python/fast_mlsirm/__init__.py, python/fast_mlsirm/_legacy_init.py, tests/test_reference_backend.py, tests/test_cov_f_fit.py, tests/test_full_information_factor_stability.py, tests/test_paper_features.py
The package exports a public fit wrapper and a named fit_reference API. The reference API forces NumPy within a scoped authorization context and forwards fitting inputs through the existing pipeline.
CLI reference mode
python/fast_mlsirm/cli.py, tests/test_cli.py, README.md
The CLI removes numpy from --backend choices. The fit --reference flag selects NumPy parity fitting and rejects explicit Rust selection.
Boundary validation coverage
tests/test_irt_contract.py, tests/test_marginal_parity.py, tests/test_missing_and_extreme_robustness.py, tests/test_release_acceptance_backend_authority.py, tests/test_scoring_methods.py, tests/test_security_hardening.py
Tests use fit_reference for NumPy cases, retain fit for Rust cases, validate parity tolerances, and verify public configuration rejects NumPy.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: 🟠 High · up to ce188

The change is intended to keep production fitting Rust-owned and fail closed, but the current implementation still allows an in-process caller to bypass that boundary and run NumPy through the normal fitting entry point. This could cause production behavior to violate the documented backend contract, so the public entry point should be corrected before merging.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant fit_reference
  participant fit
  participant BackendResolver
  participant NumPy
  Caller->>fit_reference: submit reference fitting inputs
  fit_reference->>fit: delegate with NumPy scope active
  fit->>BackendResolver: resolve scoped reference backend
  BackendResolver-->>fit: numpy
  fit->>NumPy: execute reference fitting
  NumPy-->>Caller: return FitResult
Loading
sequenceDiagram
  participant CLI
  participant fit_reference
  participant fit
  CLI->>CLI: parse --reference
  alt --reference enabled
    CLI->>fit_reference: execute NumPy reference fitting
  else production mode
    CLI->>fit: execute Rust-owned fitting
  end
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 54.69% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: isolating NumPy parity behind an explicit reference API.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/reference-backend-boundary-626

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head update: a7b23737d737d465fe29aca333a97ef7f087100b adds a fail-closed CLI guard so --reference cannot be combined with an explicit Rust backend. Please review this exact post-fix head; prior 833a080 evidence is superseded. @opencode-agent review @cwl-noema-review review

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head correction: ea9c7598e4d2f4d26b28f7fae0958f6f0860b8a adds the required level-one title and supported ## Changed section to the changelog fragment. tests/test_changelog_fragment_contract.py now passes 8/8. Review and checks must use this exact head; earlier a7b2373 evidence is superseded. @opencode-agent review @cwl-noema-review review

@seonghobae

Copy link
Copy Markdown
Contributor Author

Correction: the exact current post-fix head is ea9c759. The changelog title/section fix and focused 8/8 contract proof are bound to this SHA; please disregard the previously mistyped long SHA in my comment.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head update: e5e8ed7d6eac9a38930b3e5c20e85ace1a901a98 adds the parser regression proving --backend numpy is rejected by the production CLI. Focused reference/CLI tests and Ruff pass. Please review this exact head; prior ea9c759 evidence is superseded. @opencode-agent review @cwl-noema-review review

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head documentation alignment: fba03af6f0d78c6afc04f78edb425bd060e6ef92 updates PRD/ADR/doctoring and code comments so NumPy is described as explicit reference only; stale implicit-fallback language is removed from current contract docs. Focused changelog/backend/config/CLI tests (72 passed, 1 deselected) and Ruff pass. Review this exact head; earlier e5e8ed7 evidence is superseded. @opencode-agent review @cwl-noema-review review

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head semantic guard: 6e76ee8a4e649490a29991bdc92cb73cf586f2f adds a regression that rejects the Rust-only plain unidimensional MMLE path from fit_reference, preventing a misleading NumPy result label. Focused reference/backend/CLI/changelog tests pass 44/44 and Ruff passes. Review this exact head; earlier fba03af evidence is superseded. @opencode-agent review @cwl-noema-review review

@seonghobae

Copy link
Copy Markdown
Contributor Author

Correction: the exact current head is 6e76ee8. The previous comment contained a mistyped long SHA; disregard it. All semantic-guard evidence is attached to this corrected current SHA.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking contract finding on exact head 6e76ee8a1b781664ae692b55633e9c7f1aa6e819: the new public fit() signature exposes _allow_reference_backend: bool = False. Any caller can therefore invoke ordinary fast_mlsirm.fit(..., FitConfig(backend="numpy"), _allow_reference_backend=True) and select the NumPy reference owner through the production API, contradicting this PR's stated invariant that NumPy is reachable only through the explicitly named fit_reference surface. Please remove the bypass flag from the public fit signature and route both public entry points through a private implementation/helper instead. Add a regression that proves ordinary fit() has no reference-enabling keyword (e.g. the bypass call is rejected before data/native work) while fit_reference() retains the intended parity path. The new plain-unidimensional MMLE reference rejection is compatible with that refactor.

@seonghobae
seonghobae marked this pull request as draft August 20, 2026 09:15
auto-merge was automatically disabled August 20, 2026 09:15

Pull request was converted to draft

Copy link
Copy Markdown
Contributor Author

Current exact head 6998c42def40f675c7578b1b026ae091c0ea0b34 now also hardens backend/device semantic-control admission test-first: 328514dcfd5caab7e27d63dbd7a6230a88f2e43f adds hostile str-subclass/arbitrary-provider RED coverage, and 6998c42def40f675c7578b1b026ae091c0ea0b34 requires exact built-in text before strip/lower, preserving built-in whitespace/case normalization. This is validation only; no numerical ownership moved out of Rust.

Keep this PR Draft for one remaining contract blocker discovered on the current source: public fit(..., _allow_reference_backend=True) plus public FitConfig.validate(allow_reference_backend=True) still expose a caller-selectable NumPy path despite the PR's stated boundary that production configuration cannot select NumPy. The reference authority should be internal-only (for example, a private implementation/validation helper called by fit_reference) while the public fit signature and public FitConfig.validate() remain production-only. Do not mark Ready until that bypass is removed and exact-head CI/security/review evidence is reacquired.

Copy link
Copy Markdown
Contributor Author

@opencode-agent Please perform a review-only formal review of exact current PR head e47f9e6b257c6dbabff77702588d60bbb5cc5fea. Re-check changed-file scope, current-head findings, unresolved threads, mergeability, and all required Checks. Do not reuse stale review evidence, mutate the branch, self-approve, or merge.

@seonghobae
seonghobae enabled auto-merge (squash) August 20, 2026 23:29
@seonghobae

Copy link
Copy Markdown
Contributor Author

Manual current-head review completed for e47f9e6b257c6dbabff77702588d60bbb5cc5fea. The production/reference boundary is coherent end to end: public fit resolves Rust or fails closed, fit_reference owns the named NumPy parity scope, CLI --reference is explicit, and buyer/doctoring documents match the contract. Changed backend/reference/CLI/release tests (83) pass locally; current-head required Checks are green. No actionable defect found. Please keep formal current-head review and protected merge gates authoritative.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review please review exact current head e47f9e6b257c6dbabff77702588d60bbb5cc5fea. No formal approval is being inferred from this manual review.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Reviewed exact HEAD e47f9e6 with CodeGraph. The public Rust/auto backend boundary and explicit fit_reference authority are covered by 25 backend/reference/release tests plus 82 configuration/CLI/reference tests; focused tests, compile checks, and diff checks passed. Hosted CI, Rust, GPU smoke, security, CodeQL, OpenCode, Noema, Strix, and coverage checks are green at this HEAD. The remaining repository merge state is formal approval required; no bypass used.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Rechecked current PR head e47f9e6b257c6dbabff77702588d60bbb5cc5fea against major comment 3821906950.

No additional edit is needed: the package public export fast_mlsirm.fit is already python/fast_mlsirm/_fit_public.py::fit. It clears inherited _REFERENCE_BACKEND_ACTIVE, validates the production config, and calls resolve_backend before delegating to the implementation. The internal fast_mlsirm.fit.fit is intentionally retained as the reference-capable implementation used by fit_reference; it is not the package public export.

Exact-head proof: tests/test_reference_backend.py::test_public_fit_rejects_numpy_inside_reference_scope passes, along with the backend and release-authority suites: 25 passed. Ruff, interrogate (100%), and git diff --check also pass. The editable Rust build completed successfully and the production fit pipeline/regression subset passed 20 tests.

)

* test(multigroup): expose population label int64 wrap

* fix(multigroup): reject population label int64 overflow

* docs(changelog): record population label narrowing guard

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 7 new potential issues.

Open in Devin Review

Comment thread python/fast_mlsirm/fit.py
Comment thread tests/test_cli.py
Comment thread python/fast_mlsirm/backend.py
Comment thread python/fast_mlsirm/objective.py
Comment thread python/fast_mlsirm/fit.py
Comment thread python/fast_mlsirm/backend.py
Comment thread python/fast_mlsirm/backend.py
@opencode-agent
opencode-agent Bot disabled auto-merge August 21, 2026 14:13
@opencode-agent opencode-agent Bot added area: api API, protocol, event, or external contract priority: medium Normal-priority or P2 work status: needs-review Open pull request requiring current-head review or checks type: bug Defect or incorrect behavior labels Aug 22, 2026
@coderabbitai

coderabbitai Bot commented Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f355818d-3189-4134-8b26-d8cdd3633531

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

_fit_mmle() gained a required backend positional parameter earlier on
this branch (threaded through into FitResult for exact resolved-backend
reporting), but two direct-call tests weren't updated to pass it,
failing closed CI with "missing 1 required positional argument:
'backend'" rather than exercising their intended Rust-fail-closed
assertion. Both call the Rust-required path, so "rust" is the correct
value (the RuntimeError fires before backend is read either way).

test_cli_plain_mmle_backend.py's CLI test writes --out under tmp_path
without confining the process cwd to it first, so cli.py's
_confine_cli_paths correctly rejected the output path as outside the
current working directory. Added the same monkeypatch.chdir(tmp_path)
pattern already established by test_cli.py's autouse _cli_workspace
fixture.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Open in Devin Review

Comment thread python/fast_mlsirm/_fit_public.py
Comment thread python/fast_mlsirm/backend.py

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 0 new potential issues.

Open in Devin Review

@seonghobae
seonghobae enabled auto-merge (squash) August 24, 2026 01:06
@opencode-agent
opencode-agent Bot disabled auto-merge August 24, 2026 02:19
@seonghobae
seonghobae merged commit a599aae into main Aug 24, 2026
38 checks passed
@seonghobae
seonghobae deleted the fix/reference-backend-boundary-626 branch August 24, 2026 06:00
seonghobae added a commit that referenced this pull request Aug 25, 2026
Pin the baseline to protected main@9c12eab1 (v0.9.0). The open-PR queue
collapsed from 74 to 6 since the previous snapshot; replace the stale PR
table with the current six lanes (#1363, #1345, #1279, #1029, #1005, #998)
including their merge-forward state. Update gap-matrix evidence columns for
landed work: automatic Rust backend (#951/#1070), crossed multiple-membership
(#1014), the Figma design-boundary ADR now at docs/adr/0016 via merged
#1130, and cross-engine conformance (#1082) after #1077 closed as completed.
No production code, formula, estimator, or release surface changed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: api API, protocol, event, or external contract priority: medium Normal-priority or P2 work status: needs-review Open pull request requiring current-head review or checks type: bug Defect or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant