Skip to content

test(fipc): split consumer provenance fields + row-order gate - #2090

Open
seonghobae wants to merge 75 commits into
mainfrom
seonghobae/fmls-e-harness-roworder
Open

seonghobae wants to merge 75 commits into
mainfrom
seonghobae/fmls-e-harness-roworder

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • scripts/s1_fipc_consumer_verify.py + contracts/s1-fipc-consumer-verify-v1.schema.json split consumer provenance into sha (consumer checkout), build_source_sha (explicit, never inferred), and loaded_extension_sha256.
  • Row-order drift is quantified (row_order_max_abs, row_order_rmse) without touching optimizer tolerances.
  • build_receipt() reports row_order_diagnosis: "binding_unavailable" when _core.fit_two_tier_grm_fipc is absent instead of surfacing a buried TypeError as a failed fit.

Identification gate

The row-order verdict is interpretable only when the anchored primary-loading pattern has full column rank (plus one anchor per specific block when specific variances are estimated). Otherwise the receipt fails closed with not_identified. The fixture anchors items 0,1,3,4.

GPU

--device {cpu,gpu,auto} threads #2097's device argument through every FIPC call. The receipt records device_requested, gpu_execution_used and gpu_backend.

Lineage

Reading the receipt

Read row_order_diagnosis and each individual gate together. An unavailable binding or unidentified metric leaves the numerical gates unevaluated and false. A nonconverged row-order refit can instead retain finite discrepancy measurements and independently evaluated anchor/prior/negative-probe gates while row_order and all_pass remain false. Neither equal EAP rows nor a small discrepancy overrides explicit nonconvergence.

Test plan

  • tests/test_s1_fipc_consumer_verify.py: receipt keys/enum conform to the schema; missing binding yields binding_unavailable, all_pass: false, no fipc_fit_error
  • Script run end-to-end on a main-era _core prints a schema-conforming receipt
  • Rank-deficient anchor fails closed; flattened binding outputs reshaped (fake-binding tests)
  • Historical S1 execution on producer 91ea81e8 reported CPU/Metal all_pass: true, row_order_max_abs: 0.0. This predecessor evidence does not certify stopping head 14d4ce9b, current consumer execution, combined-main acceptance, or study-quality recovery.
  • Two-sided non_unit_focal_prior; GPU receipt fails closed on CPU fallback (fake-binding tests)

Current exact-source evidence (October 2, 2026 KST)

  • Consumer head: 97c0f5ebdfc449cb67181f827631f898d15476c3; observed main: a0d7958ead3b7bc474e45709ea8fa87122aa6406. Current consumer changes are limited to the script, receipt schema and contract tests; stopping fix(fipc): reject damping-induced false convergence #2295 remains a separate unmerged producer path at 14d4ce9bd2a5503d79ca3e813badc24a4f01ec92.
  • Consumer fail-closed repairs cover explicit convergence of reference/base/permuted fits, finite and correctly shaped consumed outputs, preservation of anchors and specific priors in both refits, exact documented invalid-map rejection, and all-valid-call GPU/auto metadata aggregation. Auto CPU fallback remains permitted; explicit GPU acceptance requires all valid calls to report actual dispatch. These are bookkeeping/admission fixes, not estimator changes.
  • Complete consumer delta and final incremental fix were independently reviewed locally with hashes matched to committed bytes. Exact consumer-head focused + ordinary two-tier tests: 113 passed with warnings as errors. In-memory mutation checks detected 3/3 guard regressions, with 3/3 unmutated baselines and two otherwise-valid controls passing. Local AI review is not a counted GitHub approval.
  • Hosted CI run 36922952395 is SUCCESS. Python 3.12 and 3.14 each executed 7,506 passing tests, 7 skips and 300 warnings. Both Rust workspace and explicit PyO3 binding test steps passed; package, fuzz and GPU-smoke passed. Actual checkout was synthetic PR merge 704cea8689022b3ddb1f24b7ed057c3e5ee19530, parents main + consumer above, tree equal to the consumer tree. It does not contain stopping fix(fipc): reject damping-induced false convergence #2295 or certify real Metal FIPC execution.
  • Existing source-attested stopping extension digest 5a33b0ebfbaf9a425182292cf103692930562825b3f35eecb9bb31c57b4d84e7 was reused without rebuilding. Actual CPU S1 execution was at consumer 0e8503d1df913b116b440077a0daabfa606aa357, not relabeled to this successor: both focal refits reported prior_update_stalled, nonconverged; max-abs/RMSE row discrepancy were zero, but row-order and all-pass correctly remained false. The one-iteration negative probe was step_limited, nonconverged. No scientific recovery improvement is claimed.
  • Current test(fipc): split consumer provenance fields + row-order gate #2090 Noema run 36922947421, job 110574255752, succeeded through token mint/current-head/sidecar setup but failed during model verdict preparation: final gateway HTTP 400 after 351.3 seconds, request ID de0a094f74894286b74f67ca5959c412. This is distinct from the older token-mint HTTP 500 and is not a model verdict. Exact upstream rejection detail remains absent; offline request/envelope observations do not establish its cause.
  • CodeQL handler run 36923644893 and OpenCode handler run 36923381545 were queued at validation on the latest bounded read. Required wrapper failures reflect missing authenticated terminal receipts, not a demonstrated terminal downstream scan/review failure. Preserve existing handlers; no synthetic status, rerun or approval substitutes for those receipts.
  • Ready for review is not merge/scientific/GPU acceptance. Genuine independent approving review/last-push approval and all applicable required current-head gates remain prerequisites. No protection, credential or deployment bypass is authorized.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W7KjsRY2SXHoAk4QkcwtB3

Summary by CodeRabbit

  • New Features
    • Added full bifactor calibration that estimates factor means and standard deviations while keeping item parameters fixed.
    • Added person scoring for two-tier models, including expected raw scores and anchor-based reference score moments.
    • Added CPU, GPU, and automatic device selection for bifactor calibration, with execution and fallback details in results.
  • Bug Fixes
    • Improved two-tier calibration convergence checks and reporting for backtracking and stalled updates.
    • Tightened input validation for response values, quadrature settings, and scoring outputs.

@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: ContextualWisdomLab/fast-mlsirm/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8b778c9e-7d63-4ff4-b6a4-3c95fcb20f62

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

This pull request adds FIPC verification and fitting APIs, GPU dispatch and reduction support, and two-tier reference scoring. It also adds convergence and numerical regression coverage, documentation, a receipt schema, and offline workflow contract tests.

Changes: FIPC consumer verification

Layer / File(s) Summary
Verification inputs and numerical checks
scripts/s1_fipc_consumer_verify.py
Adds synthetic fit inputs, anchor identification checks, finite-value validation, caller-sized quadrature, and expected-raw calculation.
Receipt gates and schema
scripts/s1_fipc_consumer_verify.py, contracts/s1-fipc-consumer-verify-v1.schema.json, tests/test_s1_fipc_consumer_verify.py
Adds fit and validation gates, provenance and GPU checks, fail-closed receipt generation, CLI handling, and schema and gate tests.

Bifactor FIPC and GPU execution

Layer / File(s) Summary
Full bifactor FIPC estimator
crates/mlsirm-core/src/bifactor_full_fipc.rs, crates/mlsirm-core/tests/bifactor_full_fipc_tests.rs, python/fast_mlsirm/bifactor_grm.py, tests/test_bifactor_fipc_full.py, docs/bifactor-full-fipc.md
Adds fixed-item population fitting for factor means and SDs, convergence and likelihood checks, input validation, and fit metadata.
GPU dispatch and reduced moments
crates/mlsirm-core/src/gpu.rs, crates/mlsirm-core/src/gpu_bifactor.rs, crates/mlsirm-core/src/gpu_specific_tree.wgsl, tests/unit/gpu_bifactor_*, tests/unit/gpu_specific_leaf_partition_tests.rs
Records GPU dispatch and fallback metadata. Adds optional specific-moment output and tiled reductions with GPU regression tests.
Device-aware bindings
crates/mlsirm-core/src/bifactor_grm.rs, crates/fast-mlsirm-py/src/lib.rs, python/fast_mlsirm/bifactor_grm.py, tests/test_bifactor_fipc_device_transport.py
Passes device selection to bifactor FIPC, returns GPU metadata, exposes full FIPC through bindings, and restricts negative response placeholders to masked cells.

Two-tier reference scoring

Layer / File(s) Summary
Reference moments and person scoring
python/fast_mlsirm/two_tier_fipc.py, python/fast_mlsirm/two_tier_grm.py, python/fast_mlsirm/_legacy_init.py, crates/fast-mlsirm-py/src/lib.rs, tests/test_two_tier_fipc.py, tests/test_two_tier_expected_raw.py, tests/test_two_tier_reference_prior_admission.py, docs/changelog.d/two-tier-reference-scoring.md
Adds anchor-only reference moments, person scoring, JSON-serializable payload execution, Rust-backed integration, input validation, and public exports.

Two-tier FIPC convergence and numerical regressions

Layer / File(s) Summary
Convergence, recovery, and fit results
FIPC_CLASS_A_ROOT_CAUSE.md, RUST_ACCEPT_PATH_CAPTURE_3fc6160a.*, docs/fipc-convergence.md, tests/test_two_tier_grm.py, tests/unit/two_tier_*fitter*, tests/unit/two_tier_*projection*
Documents convergence and rollback criteria and records acceptance-path captures. Adds fit and predecessor comparisons for anchors, recovery, convergence, and final person projection.
E-step projections and marginal reuse
tests/unit/two_tier_*marginal*, tests/unit/two_tier_*cache*, tests/unit/two_tier_*likelihood_only_tests.rs, tests/unit/two_tier_*projection_tests.rs, tests/unit/two_tier_gpu_*tests.rs
Adds parity and evaluation-count tests for E-step outputs, marginal calculations, cached evaluations, projections, and GPU paths.
GRM buffers and zero-mass parity
crates/mlsirm-core/src/poly.rs, tests/unit/two_tier_zero_primary_mass_tests.rs, tests/unit/two_tier_item_line_search_*
Adds caller-storage variants for GRM log probabilities and gradients, with regression tests for output parity, allocations, and zero-primary-mass cases.

PyPI inventory workflow contract

Layer / File(s) Summary
Inventory workflow behavior
tests/test_pypi_gap_guard_workflow_contract.py
Adds offline tests for token configuration, event handling, release-list and PyPI failures, canonical version selection, and temporary-file cleanup.

Priority: ⬇️ Low

Estimated code review effort: 5 (Critical) | ~120 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant PythonAPI
  participant PyO3Bindings
  participant BifactorFIPC
  participant GPUEStep
  PythonAPI->>PyO3Bindings: call full bifactor FIPC with device
  PyO3Bindings->>BifactorFIPC: pass fit inputs and device
  BifactorFIPC->>GPUEStep: compute reduced posterior moments
  GPUEStep-->>BifactorFIPC: moments and dispatch metadata
  BifactorFIPC-->>PyO3Bindings: fit result and convergence metadata
  PyO3Bindings-->>PythonAPI: fit result with GPU metadata
Loading
Merge Risk: 🔵 Low · up to 0e84f

The new FIPC fitting, GPU dispatch, and scoring code shows no confirmed functional defects in the supplied evidence. Several documents and one test name still describe outdated status or the old seed and should be corrected for accuracy. The PR is mergeable with this follow-up.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 12b7b

The change is a local diagnostic rather than a new privileged service. It clearly labels provenance as caller-asserted and prevents ordinary fitting failures from producing a passing receipt. Behavior with the separately supplied fitting implementation is not fully established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The inspected new entrypoint operates within the invoking process: it reads the installed extension, runs synthetic fits, and emits a receipt to standard output. The evidence does not establish a remote endpoint, tenant-wide authority, or production consumers of these receipts.

Trust Boundaries and Controls

  • observed — CLI and environment revision labels are validated as lowercase hexadecimal identifiers, while device choices are enumerated. Revision lookup uses a fixed subprocess argument list in the script's repository directory. These labels become diagnostic metadata, not authenticated identity or commands interpolated into a shell.

Resilience and Maintainability Implications

  • inferred — Per-invocation receipt state and deferred output prevent ordinary partial fitting failures from exposing a successful receipt. Repeated or concurrent native-call isolation remains unresolved: the consumer passes model arrays to a separately supplied binding, and fake-binding tests cannot prove native input immutability or absence of shared state.
Pre-merge checks · Passed 4 · Failed 1

Failed Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 56.34% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 355 functions across 48 files. (9 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks (4 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the main changes: separating FIPC consumer provenance fields and adding a row-order gate. It is concise and directly related to the pull request objectives.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 56.34% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 355 functions across 48 files. (9 skipped: 9 unsupported.)

✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exact f069c7814eb6dfcbebf0ae0d7c7ccbb1d5daec3c review: 이 harness는 provenance field 분리 자체는 맞지만, 현재 row-order 진단을 scientific acceptance로 해석하면 안 됩니다. N_PRIMARY=2인데 ANCHOR=[1,1,1,0,0,0]이고 PRIMARY_MAP의 첫 세 anchor row는 모두 첫 번째 primary factor에만 loading을 가집니다. 즉 anchored primary-loading submatrix가 full-column-rank가 아니어서 #2084에서 남아 있는 metric-identification 결함과 같은 방향입니다. 이 상태에서 row permutation 전후 EAP 차이는 optimizer order dependence와 식별되지 않은 parameterization drift를 구분하지 못합니다.

RED: factor 2까지 독립적으로 span하는 full-rank anchor fixture와, 일부러 rank-deficient인 현재 fixture를 나란히 실행해 후자는 identification gate에서 fail closed하도록 하십시오. estimate_specific_vars=true 경로도 specific block별 scale anchor를 별도로 요구해야 합니다. GREEN: identification-valid fixture에서만 row-order max-abs/RMSE를 해석하고, seeded true-parameter recovery의 bias/RMSE/coverage를 함께 남기십시오. synthetic fixture는 diagnostic/unit evidence로만 두고 production/scientific acceptance는 real/right-cleared data evidence와 분리해야 합니다.

Provenance도 build_source_sha가 환경변수/CLI caller label이라 loaded extension digest와 cryptographically bind되지 않습니다. loaded_extension_sha256는 유효한 binary identity지만, build_source_sha_present=true 자체는 build provenance 증명이 아닙니다. immutable release/attestation manifest에서 {source_sha, artifact_sha256} binding이 있을 때만 그 관계를 authority로 승격하십시오.

Verdict: provenance field split PASS candidate / FIPC identification-dependent row-order diagnosis FAIL / source↔binary provenance binding PENDING.

seonghobae and others added 2 commits September 29, 2026 08:22
The S1 consumer receipt called _core.fit_two_tier_grm_fipc (shipped by
#2097, not yet on main) and e_step_n_chunks (#2075, not on main). On a
main build every gate failed through a buried TypeError, and the
wrong-model check hit a NameError on focal_y.

- factor build_receipt() out of main(); report
  row_order_diagnosis="binding_unavailable" when the binding is absent
- drop the e_step_* kwargs (main already runs one chunk)
- keep the wrong-model check inside the fit block
- remove the unused _provenance() duplicate
- add a schema-conformance test for the receipt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7KjsRY2SXHoAk4QkcwtB3
@seonghobae

Copy link
Copy Markdown
Contributor Author

Cross-reference from the S1 rerun at #2097 head 91ea81e8 (#2097 (comment)). Two harness defects in scripts/s1_fipc_consumer_verify.py on this branch will make S1 fail once the binding lands, independent of the producer:

  1. ANCHOR = [1,1,1,0,0,0] anchors only items loading on primary 1. Primary 2 has no fixed item, so its focal mean is not identified. The provenance-matched rerun drifts to about -1.42 (truth -0.35) with 100 monotone accepted steps and max_iter_reached on both CPU and Metal, so the converged-gated checks can never pass. The fixture needs at least one anchored item per primary dimension, e.g. anchor items 0 and 3.
  2. theta_p_eap comes back flattened with length n_persons·n_primary (360 here). Lines 147/151 index it by person permutation and feed it to expected_raw without .reshape(N_PERSONS, N_PRIMARY), which raises ValueError: operands could not be broadcast together with shapes (180,) (360,).

These are consumer-fixture fixes. The producer's accept/backtrack logic is not implicated.

@seonghobae
seonghobae marked this pull request as ready for review September 29, 2026 00:36
…utputs

Review on #2090 found two consumer-fixture defects that make S1 fail once
the #2097 binding lands:

- ANCHOR=[1,1,1,0,0,0] anchored only primary-1 items, so the focal
  primary-2 mean was not identified (the rerun drifted to -1.42 vs truth
  -0.35 and hit max_iter). Under FIPC the focal metric comes only from
  fixed items (Kim, 2006); mu enters through A_anchor @ mu, so the
  anchored primary-loading pattern must be full column rank. Anchor
  items 0,1,3,4 (two per primary, both specific blocks covered) and fail
  closed with row_order_diagnosis="not_identified" otherwise. Specific
  variance estimation additionally requires an anchor per specific block.
- theta_p_eap, a_primary and threshold come back flat from PyO3; reshape
  them before permuting rows and computing expected raw scores.

Also label build_source_sha as caller_asserted and the receipt as a
synthetic_diagnostic, and record single-replicate focal-moment and
free-item recovery errors next to the row-order verdict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7KjsRY2SXHoAk4QkcwtB3
@seonghobae

Copy link
Copy Markdown
Contributor Author

Addressed at 4b0960f9:

  • Identification (review + S1 rerun, defect 1). anchor_identification() requires the anchored primary-loading pattern to be full column rank. Under FIPC the focal metric comes only from fixed items (Kim, 2006), and μ enters them only through A_anchor μ. With estimate_specific_vars=True it also requires an anchored item in every specific block (two-tier structure, Cai, 2010). The fixture now anchors items 0,1,3,4 (rank 2, both specific blocks covered; items 2 and 5 free). A rank-deficient anchor fails closed with row_order_diagnosis: "not_identified" before any fit. Test: test_rank_deficient_anchor_fails_closed_before_fitting uses the old [1,1,1,0,0,0].
  • Flat outputs (defect 2). theta_p_eap, a_primary and threshold are reshaped to (n_persons, n_primary) / (n_items, ·) before the row permutation and expected_raw. Test: test_flattened_binding_outputs_are_reshaped_per_person reproduced ValueError: shapes (180,) (360,) before the fix.
  • Provenance. build_source_sha_authority: "caller_asserted". The receipt does not claim a source↔binary binding; that stays PENDING until an attestation manifest exists.
  • Evidence class. evidence_class: "synthetic_diagnostic". The receipt records single-replicate focal_primary_mean_error, focal_primary_sd_error, and free_item_param_rmse. These are one-replicate errors, not Monte Carlo bias. Coverage is not reported because the binding returns no standard errors.

Still open: the identified fixture has not yet been rerun against a #2097 binary. On main the receipt stays binding_unavailable.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4


  • 🪄 Fix CodeRabbit comments on this PR
🤖 Prompt to fix review comments
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Inline comments:
Review comments at @scripts/s1_fipc_consumer_verify.py:
- Line 203: Update the anchor-preservation check that sets
results["anchor_rows_fixed"] to also compare shaped["a_specific"][ANCHOR] with
the corresponding fixed a_specific values at ANCHOR. Preserve the existing
a_primary and threshold comparisons.
- Around line 59-60: Set the cwd of the subprocess.run call for git rev-parse
HEAD to the repository containing this script, so the receipt records the
consumer checkout’s commit rather than the caller’s repository.
- Line 264: Before serializing receipt with json.dumps, convert non-finite
derived diagnostic metrics such as row_order_max_abs and row_order_rmse to null
while preserving the failing diagnosis; set serialization to reject any
remaining non-finite values.
- Around line 219-220: Validate non-null consumer_sha and build_source_sha
values before constructing the receipt, rejecting any that do not match the
schema’s 40-character lowercase hexadecimal format. Preserve null handling and
add neither value to the receipt unless it passes validation.

After applying the fix, consider running `coderabbit review --agent` for local
review. Visit https://docs.coderabbit.ai/cli?utm_source=ghpr

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: ContextualWisdomLab/fast-mlsirm/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 7802df51-1a7a-45df-a158-bd1c8e37c0b9

📥 Commits

Reviewing files that changed from the base of the PR and between a0d7958 and 4b0960f.

📒 Files selected for processing (3)
  • contracts/s1-fipc-consumer-verify-v1.schema.json
  • scripts/s1_fipc_consumer_verify.py
  • tests/test_s1_fipc_consumer_verify.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Comment thread scripts/s1_fipc_consumer_verify.py
Comment thread scripts/s1_fipc_consumer_verify.py Outdated
Comment thread scripts/s1_fipc_consumer_verify.py
Comment thread scripts/s1_fipc_consumer_verify.py Outdated
- non_unit_focal_prior was one-sided (max sd > 1.01), so a focal SD
  shrinking below 1 (the fixture's truth has 0.8) never counted. Test
  |sd - 1| instead.
- --device {cpu,gpu,auto} threads the #2097 device argument through every
  FIPC call, including the wrong-model check. The receipt records
  device_requested, gpu_execution_used and gpu_backend, and a gpu receipt
  fails when the binding falls back to CPU. device is passed only when
  not cpu, so bindings without the argument still work.

On a #2097 build (91ea81e), both --device cpu and --device gpu (Apple
Metal) return all_pass=true with row_order_max_abs=0.0; CPU and GPU focal
moments agree to about 2e-5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7KjsRY2SXHoAk4QkcwtB3
@seonghobae

Copy link
Copy Markdown
Contributor Author

S1 re-run on a real #2097 binary (source 91ea81e8, local maturin build; build_source_sha is still caller-asserted), consumer bf5db339:

device all_pass gpu_execution_used backend row_order_max_abs focal mean error focal sd error ext sha256[:16]
cpu True False None 0.0 [-0.0969, -0.0436] [-0.2678, 0.1883] 3f61c9c2a4ae34c4
gpu True True Metal 0.0 [-0.0969, -0.0436] [-0.2678, 0.1883] 3f61c9c2a4ae34c4

With the identified fixture (anchors 0,1,3,4), the focal primary-2 mean now lands at -0.39 (truth -0.35), where the old fixture drifted to -1.42. The fit converges, and row order is exactly invariant on both CPU and Metal. non_unit_focal_prior is now two-sided, so the fixture's 0.8 SD counts.

This is synthetic diagnostic evidence only. The SD recovery shows a producer-side concern, reported on #2097.

…fits

Score persons from a fitted orthogonal-primary two-tier GRM on the
reference item metric: keep the fit's Rust EAPs, take each person's expected
raw total from the shared expected_raw_two_tier_grm (#2052), and add the
anchor-only reference expected-score moments (Kim, 2006; Lord, 1980).

- mlsirm-core: two_tier_grm_reference_score_moments integrates
  two_tier_recursion::two_tier_expected_raw_at_q (the single owner of the
  two-tier expected raw total) over the independent-normal reference prior
  on a Gauss-Hermite product grid, in one call; PyO3 binding added.
- Python keeps validation and marshaling only (ADR-0002); correlated fits
  and Phi != I fail closed. The NumPy formulas live in tests as an oracle.
- Payload entry point for JSON workers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SjidEWun6sasVjj6xbMit

Copy link
Copy Markdown
Contributor Author

Exact-head admission correction — bf5db339189dde77a711a6da1fe14ef40adbf793

Ready is review admission only. Fresh audit against base a0d7958ead3b7bc474e45709ea8fa87122aa6406 found:

  • latest terminal workflow blockers: CodeQL PR 36538664419=failure
  • unresolved review threads: 4

This PR is moved to Draft/Proposed until the causal owner repair is present on a successor exact head and re-audited. Queued/pending work is neither an additional blocker nor passing evidence. No Close, force push, destructive rebase, manual rerun, synthetic status/approval, merge, auto-merge, or bypass was performed.

@seonghobae
seonghobae marked this pull request as draft September 30, 2026 05:26
seonghobae and others added 3 commits September 30, 2026 14:28
Record the consumer revision from its own checkout, check anchored specific loadings, reject malformed revision labels before receipt work, and fail closed on nonfinite diagnostic metrics. Add regression tests for all four verified review findings.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7KjsRY2SXHoAk4QkcwtB3
Require an explicitly false convergence flag and one of the two documented budget termination reasons. Pin unknown or missing reasons and malformed convergence flags without changing fit success or recovery thresholds. A real seed1002 child-binding probe reproduced step_limited at one iteration.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7KjsRY2SXHoAk4QkcwtB3
Require literal convergence from both row-order fits and finite item/EAP outputs before certifying the S1 consumer diagnostic. Preserve finite row-order metrics when a refit does not converge. Add RED/GREEN regression coverage without changing estimator, stopping, or row-order tolerances.
@seonghobae

Copy link
Copy Markdown
Contributor Author

Exact-head consumer evidence — 0e8503d1df913b116b440077a0daabfa606aa357

Consumer integrity fixes were normally pushed to this PR. The latest two-file delta was independently reviewed (no security or logic findings); the reviewer independently reran 50 consumer tests. At the committed head, consumer + ordinary two-tier tests passed: 58 passed in 11.42 s with warnings treated as errors. The complete PR delta is undergoing a separate read-only review; this comment is not an approval or merge-admission claim.

Actual CPU S1 execution, without rebuilding or repeating the producer stopping suite:

  • Consumer source: 0e8503d1df913b116b440077a0daabfa606aa357; script SHA-256 95f64f1a081c50788dae3ba45ce5c5849d9e3ac6ee7946e2d98dc3b51f5c7ad8.
  • Producer source: immutable stopping head 14d4ce9bd2a5503d79ca3e813badc24a4f01ec92, tree 03bf54b4b10aeabc9ba1ea7323520328de3732eb.
  • Existing source-built wheel SHA-256 5623ceb1d8a24faf21d19414afebab5c239020fe52f2ccac2b20cdef0ce5cbd3; wheel member and imported _core SHA-256 5a33b0ebfbaf9a425182292cf103692930562825b3f35eecb9bb31c57b4d84e7.
  • Parent independently verified the immutable source manifest, wheel/import binding, and all five execution-artifact hashes. The consumer's own build_source_sha remains null/caller-asserted; the external build receipt, not an inferred label, binds the source to the extension.
  • CPU run completed normally: exit 0, 390.25 s. An earlier execution-tool timeout produced no consumer receipt and is not counted as acceptance.

Both the original and permuted focal fits returned converged=false, termination_reason=prior_update_stalled, 55 iterations, full-candidate displacement 10.0, and finite final likelihood -1394.2222831672939. EAP row discrepancy was exactly zero (max-abs and RMSE), but the corrected consumer reported row_order=false, row_order_diagnosis=fit_error, responses_fit_eap_expected_raw=false, and all_pass=false. Equal EAP rows from nonconverged refits cannot certify row-order invariance.

Anchor preservation, fixed specific priors, the one-iteration step_limited negative probe, and invalid-primary-map rejection passed their diagnostic gates. Focal mean errors were [-0.09721866478993235, -0.057401674799650415]; SD errors [-0.26777996948746086, 0.18834206605100245]; free-item RMSE 0.4665341227176292. These are one low-resolution synthetic replicate, not Monte Carlo bias, stabilized recovery, or scientific/production acceptance. No recovery improvement, GPU execution, or combined-main artifact acceptance is claimed. Historical 91ea81e8 all-pass results do not establish acceptance at the stopping head 14d4ce9b.

No likelihood, estimator, quadrature, damping, iteration budget, or row-order tolerance was changed. Finite/full-step/absolute-likelihood/no-rollback stopping invariants remain with the producer. The original binding-unavailable path on main remains supported honestly.

Noema cause separation:

  • Old-head job 109772569369 failed before model setup: four GitHub installation-token mint attempts, final HTTP 500 with empty body; request ID 99DA:20687D:747CB4:90AB6D:6ABCFDDC. This is separate from other model-gateway HTTP 400 reports.
  • New-head job 110548061751 minted tokens successfully, but explicitly skipped model review because this PR was Draft. Its success is not an independent model review.

Local verifiable artifacts (not public uploads):

  • /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-work.md
  • /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-noema-rca.md
  • /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-source-audit.md
  • /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-cpu-exacthead-20261002-044759/consumer.json (SHA-256 575a97d0afeab7404e471ecf8f3784145194c568232f173a054c76bdee8479bd)
  • /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-cpu-exacthead-20261002-044759/execution.json (SHA-256 a86e51168dccf279516baffc474191419077ac0bcfc7b851c51030368f8871b6)

Normal review/required gates and genuine independent approval remain required. No bypass, synthetic status, self-approval, producer edit, deployment, credential change, or workspace cleanup was performed.

Aggregate explicit GPU execution evidence from every valid focal, permutation, and budget-probe fit. A base-fit dispatch cannot certify a later CPU fallback. Cover every call position plus the complete-dispatch positive case without changing numerical arguments or CPU/auto behavior.
Require converged reference calibration before focal work, validate exact finite primary moments and positive SDs, check anchors and fixed specific priors in both refits, and recognize only the documented invalid-map ValueError. Address full-PR review C2-C5 with RED/GREEN regressions without estimator or tolerance changes.
@seonghobae
seonghobae marked this pull request as ready for review October 1, 2026 20:23
@seonghobae

Copy link
Copy Markdown
Contributor Author

Current consumer successor review admission — 12b7b324ce3ff39cab1c2a03232e83104675719a

The complete three-file PR delta against main a0d7958ead3b7bc474e45709ea8fa87122aa6406 was independently reviewed, not merely the newest commit. C1–C5 false-positive admission defects are fixed: every valid GPU FIPC call must report explicit dispatch; reference calibration must converge before focal work; both refits validate anchors and fixed specific priors; primary moments have exact finite vector shapes and positive SDs; invalid-map rejection recognizes the observed validation ValueError rather than unrelated exceptions.

Independent focused execution: 101 consumer tests passed in 1.42 s, warnings treated as errors. Parent verified all three reviewed file hashes, committed the unchanged reviewed bytes, ran the committed-head consumer + ordinary two-tier suite (109 passed in 9.26 s), and normally pushed. Script SHA-256: aaf09b49bedb6c28fb90fcc2432e1c3f0da65124f6a2282640cddb7c288ded71; test SHA-256: 7c5fb7b2359ff3bcf1557339e5a5b47e64f471b03871e7d14228859db69b84f4; schema SHA-256: 988627eb3dea4e94b1c94cfa29ad75904c73145ed54d043c2289b02064bb277a.

This is normal review admission only. This PR is now Ready so required CI and real model review can run, rather than accepting draft-skipped checks. Local independent AI review is not a counted GitHub approval; required independent/last-push approval and all applicable gates remain required. No merge, bypass, self-approval, synthetic status or protection change is claimed or performed.

The earlier real CPU numerical receipt stays pinned to consumer 0e8503d1 and stopping source 14d4ce9b, not relabeled to this successor. Its truthful all_pass=false is correct negative diagnostic evidence: both focal refits stalled/nonconverged despite zero row-EAP discrepancy. That per-execution conjunction is neither scientific acceptance nor consumer code-review readiness. No estimator, grid, iteration, tolerance, rollback or producer source changed in these consumer repairs; GPU hardware and combined-main acceptance remain separate owner work.

Review artifact: /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-full-successor-review.json; task evidence: /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-work.md. These are local paths, not public uploads.

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


  • 🪄 Fix CodeRabbit comments on this PR
🤖 Prompt to fix review comments
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Inline comments:
Review comments at @scripts/s1_fipc_consumer_verify.py:
- Around line 191-265: Update the GPU receipt aggregation in _fipc_gates to run
for both "gpu" and "auto" devices, requiring fit, perm_fit, and fail to report
GPU execution before setting gpu_execution_used to true; clear gpu_backend when
any call does not dispatch.

After applying the fix, consider running `coderabbit review --agent` for local
review. Visit https://docs.coderabbit.ai/cli?utm_source=ghpr

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: ContextualWisdomLab/fast-mlsirm/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 0758af40-be80-42f9-80de-f060d93cac8c

📥 Commits

Reviewing files that changed from the base of the PR and between 4b0960f and 12b7b32.

📒 Files selected for processing (3)
  • contracts/s1-fipc-consumer-verify-v1.schema.json
  • scripts/s1_fipc_consumer_verify.py
  • tests/test_s1_fipc_consumer_verify.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Comment thread scripts/s1_fipc_consumer_verify.py Outdated
Apply all-valid-call GPU metadata aggregation to auto without rejecting its legitimate CPU fallback. Pin all-GPU and each mixed-device call position with RED/GREEN tests; no numerical or explicit-device acceptance changes.
@seonghobae

Copy link
Copy Markdown
Contributor Author

Successor to the GPU metadata review comment — 97c0f5ebdfc449cb67181f827631f898d15476c3

Resolved: GPU-execution metadata now aggregates the three valid FIPC calls (base, row-permuted refit, one-iteration negative-budget probe) for both device="gpu" and device="auto". auto remains allowed to accept CPU fallback when all other consumer diagnostic gates pass, but reports gpu_execution_used=false and gpu_backend=null if any one of the three calls lacks explicit GPU dispatch. Explicit GPU mode retains its fail-closed all-pass requirement. The CPU reference fit and invalid-map probe are outside the valid GPU-execution aggregate.

The independent dirty-delta review passed on exact script/test hashes, reproduced baseline false-positive and verified the fixed behavior, and ran 105 focused consumer tests with warnings as errors. Parent then committed the reviewed bytes as 97c0f5ebdfc449cb67181f827631f898d15476c3; exact committed-head consumer plus ordinary two-tier tests: 113 passed in 12.30 s; normal push was verified at this PR head.

No actual GPU run, estimator/producer change, numerical acceptance or scientific recovery claim is made. The historical source-attested CPU receipt remains pinned to consumer 0e8503d1 / stopping source 14d4ce9b; do not relabel it to this successor. Existing branch approval and final merge requirements remain in force.

The local working tree preserves .opencode/ untracked. Exact test/review hashes and focused artifact: /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-auto-gpu-review.json; fuller run accounting: /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-work.md.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Exact-head #2090 consumer evidence (97c0f5e)

The complete consumer-only delta from main a0d7958 was independently reviewed; all C1–C5 receipt false-positive findings were repaired. Tests, on the current pushed head, pass locally. Actual hosted Ready checks are listed in the live GitHub check suite; several jobs remain in progress and no merge verdict is claimed.

CPU source-attested diagnostic: Consumer execution previously run at 0e8503d against immutable stopping source14d4ce9b and its source-built extension (see the earlier exact-head S1 evidence comment). Both focal refits were nonconverged; the consumer correctly failed closed. This result is intentionally not relabeled as an execution of97c0f5eb or a study-quality recovery result. The GPU metadata repair here validates receipt bookkeeping and does not claim actual hardware GPU execution.

No producer/estimator/tolerance/quadrature changes, no scientific recovery claim, no fake check or approval, and no protection/deployment bypass. Main still does not include the observed stopping commit. A genuine independent review/last-push approval and required current-head gates remain separate merge prerequisites.

Exact CPU evidence files remain in /Users/seonghobae/orca/reports/hermes-rolling-migration/fleet/harness-cpu-exacthead-20261002-044759/; source identity and review records are in harness-work.md and harness-full-successor-review.json.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

Current-head source review request: e481c1e704a13a19356393889150ace2ae9a0d46. Please review the complete current base-to-head delta, including the main integration and the pinned central PyPI inventory contract, rather than reusing the earlier 12b7b review.

Local evidence is bounded: workspace Cargo check; 7 exact Rust regression cases; 15 installed dev-wheel CPU consumer cases; 21 central-workflow contracts. The release build and original broad Python run timed out and remain NONPASS. No full-suite, physical-GPU, scientific recovery, or protected-merge acceptance is claimed. Preserve finite/full-step/absolute-likelihood/no-rollback and negative-recovery invariants.

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (1)
tests/unit/two_tier_grm_tests.rs (1)

1395-1410: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Fix the comment so it matches the asserted mean.

The comment says E[Y_i] = 1. The assertion checks m.mean == 2.0, which is the sum over two items. Update the comment to say that each item has an expected value of 1, so the total score has a mean of 2.

🤖 Prompt for AI Agents
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Review comment at @tests/unit/two_tier_grm_tests.rs around lines 1395 - 1410:
Update the comment in `reference_moments_hit_symmetric_mean` to clarify that
each item has expected value 1 and the total score has mean 2, matching the
assertion; leave the test behavior unchanged.

  • 🪄 Fix CodeRabbit comments on this PR
🤖 Prompt to fix review comments
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Inline comments:
Review comments at @crates/fast-mlsirm-py/src/lib.rs:
- Around line 2349-2352: Update the y conversion in the wrapper around
core_fit_two_tier_grm_fipc to reject negative responses when obs_vec is None,
matching two_tier_oakes_se. Continue mapping negative values to category 0 only
when the corresponding observed-mask entry is false, and preserve the existing
error for other negative categories.

Review comments at @crates/mlsirm-core/src/gpu_specific_tree.wgsl:
- Around line 31-44: Update the person lookup in specific_chunks to find the
observed person for the tile’s initial rank once before the leaf loop, then
advance to the next observed person only when person_rank changes. Keep the
per-leaf products and summation order unchanged.

Review comments at @python/fast_mlsirm/two_tier_fipc.py:
- Around line 152-153: After the `_vector` calls for `primary_mean` and
`primary_sd`, validate that all primary means are finite and all primary
standard deviations are finite and strictly positive; raise a `ValueError`
identifying the invalid input before calling the Rust core.

Review comments at @scripts/s1_fipc_consumer_verify.py:
- Around line 244-248: Update the --build-source-sha argument default so an
empty FIPC_BUILD_SOURCE_SHA is treated as unset, matching the fallback behavior
of --consumer-sha. Ensure the unset value allows build_receipt to report
build_source_sha as null and build_source_sha_present as false.

Review comments at @tests/test_s1_fipc_consumer_verify.py:
- Around line 512-517: Update the Git command in the fixture setup to disable
commit signing for the `git commit` invocation, so the test does not depend on
the developer’s signing configuration. Keep the change scoped to the command in
the fixture and preserve its existing commit behavior.

---

Nitpick comments:
Review comments at @tests/unit/two_tier_grm_tests.rs:
- Around line 1395-1410: Update the comment in
`reference_moments_hit_symmetric_mean` to clarify that each item has expected
value 1 and the total score has mean 2, matching the assertion; leave the test
behavior unchanged.

After applying the fix, consider running `coderabbit review --agent` for local
review. Visit https://docs.coderabbit.ai/cli?utm_source=ghpr

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: ContextualWisdomLab/fast-mlsirm/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9282866c-2504-4d77-948c-6c6ae62d36da
📥 Commits

Reviewing files that changed from the base of the PR and between 9dcf55b and e481c1e.

⛔ Files ignored due to path filters (1)
  • docs/papers/chalmers-2012-mirt.pdf is excluded by !**/*.pdf
📒 Files selected for processing (56)
  • FIPC_CLASS_A_ROOT_CAUSE.md
  • RUST_ACCEPT_PATH_CAPTURE_3fc6160a.json
  • RUST_ACCEPT_PATH_CAPTURE_3fc6160a.md
  • billing-snapshots/perf_0114_regression_20260920.json
  • contracts/s1-fipc-consumer-verify-v1.schema.json
  • crates/fast-mlsirm-py/src/lib.rs
  • crates/mlsirm-core/src/bifactor_full_fipc.rs
  • crates/mlsirm-core/src/bifactor_grm.rs
  • crates/mlsirm-core/src/gpu.rs
  • crates/mlsirm-core/src/gpu_bifactor.rs
  • crates/mlsirm-core/src/gpu_specific_tree.wgsl
  • crates/mlsirm-core/src/poly.rs
  • crates/mlsirm-core/src/two_tier_grm.rs
  • crates/mlsirm-core/tests/bifactor_fipc_tests.rs
  • crates/mlsirm-core/tests/bifactor_full_fipc_tests.rs
  • docs/bifactor-full-fipc.md
  • docs/changelog.d/two-tier-reference-scoring.md
  • docs/changelog.d/two-tier-single-primary-logprob-reuse.md
  • docs/fipc-convergence.md
  • python/fast_mlsirm/_legacy_init.py
  • python/fast_mlsirm/bifactor_grm.py
  • python/fast_mlsirm/two_tier_fipc.py
  • python/fast_mlsirm/two_tier_grm.py
  • scripts/s1_fipc_consumer_verify.py
  • tests/test_bifactor_fipc_device_transport.py
  • tests/test_bifactor_fipc_full.py
  • tests/test_perf_0114_reproduction_contract.py
  • tests/test_pypi_gap_guard_workflow_contract.py
  • tests/test_s1_fipc_consumer_verify.py
  • tests/test_two_tier_expected_raw.py
  • tests/test_two_tier_fipc.py
  • tests/test_two_tier_grm.py
  • tests/unit/gpu_bifactor_consumed_moment_tests.rs
  • tests/unit/gpu_bifactor_count_tests.rs
  • tests/unit/gpu_specific_leaf_partition_tests.rs
  • tests/unit/two_tier_final_person_legacy_fitter.rs
  • tests/unit/two_tier_final_person_projection_tests.rs
  • tests/unit/two_tier_fixed_block_marginal_legacy.rs
  • tests/unit/two_tier_fixed_block_marginal_tests.rs
  • tests/unit/two_tier_fixed_moment_output_tests.rs
  • tests/unit/two_tier_focal_cell_cache_legacy.rs
  • tests/unit/two_tier_focal_cell_cache_tests.rs
  • tests/unit/two_tier_focal_likelihood_only_tests.rs
  • tests/unit/two_tier_free_count_legacy_fitter.rs
  • tests/unit/two_tier_free_count_projection_tests.rs
  • tests/unit/two_tier_full_block_marginal_tests.rs
  • tests/unit/two_tier_gpu_consumed_moment_tests.rs
  • tests/unit/two_tier_gpu_scalar_moment_tests.rs
  • tests/unit/two_tier_gpu_table_cache_tests.rs
  • tests/unit/two_tier_grm_tests.rs
  • tests/unit/two_tier_item_line_search_legacy.rs
  • tests/unit/two_tier_item_line_search_output_tests.rs
  • tests/unit/two_tier_scalar_block_pattern_tests.rs
  • tests/unit/two_tier_specific_projection_legacy_fitter.rs
  • tests/unit/two_tier_specific_projection_tests.rs
  • tests/unit/two_tier_zero_primary_mass_tests.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Comment thread crates/fast-mlsirm-py/src/lib.rs
Comment thread crates/mlsirm-core/src/gpu_specific_tree.wgsl
Comment thread python/fast_mlsirm/two_tier_fipc.py
Comment thread scripts/s1_fipc_consumer_verify.py
Comment thread tests/test_s1_fipc_consumer_verify.py
@seonghobae

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

Exact current head: 0e84fd560df1fc11f388b192b4a1a01bc480a825, integrating main d72ad5e7f7398adfeb4ab2e1af58a8de33897cdd. Please review the whole current base-to-head source delta and adjudicate the five findings from your e481 review against this successor.

Repairs: empty build-provenance environment is unset; Git test fixture disables signing only on its own commit; reference means/SDs reject nonfinite/nonpositive priors before native acquisition; all five related native wrappers reject negative unmasked responses; specific_chunks now resolves the first observed person once per chunk and advances only at rank change, retaining original products/carry/tail order. Chunk-initialization scan cost remains; no universal complexity or measured speedup claim.

Latest main integration keeps the reviewed generalized exact-predictor cache rather than duplicating PR2070 single-primary tables. Memory behavior differs from streamed multi-primary main; historical timing remains labelled historical. The main denominator repair is retained. Exact synthetic GPU kernel cases passed on Apple M5/Metal: all four mask/group phases bitwise-equal, plus Q121/Q241 and Q241/Q121 physical count/moment gates. 99 workflow contracts and 12 benchmark-command contracts passed. Earlier whole-core build and broad Python timeouts remain NONPASS.

These are bounded source/kernel/consumer checks, not full-suite, release, two-host, fitted convergence, or scientific recovery acceptance. No approval or protected merge is claimed.

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
tests/unit/two_tier_final_person_legacy_fitter.rs (1)

3-7: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Describe the legacy-fitter oracles accurately: they freeze only the outer EM loop. All three predecessor fitters copy the control flow, but they call live production helpers (e_step_fipc*, fixed_fipc_e_step, m_step_item, focal_loglik, direct_fipc_loglik, fipc_max_param_change, fipc_convergence_decision). If a later change alters one of these helpers, the oracle and production change together, and the bitwise parity tests still pass. The comments that say "exact" or "frozen" overstate what the tests check.

  • tests/unit/two_tier_final_person_legacy_fitter.rs#L3-L7: change "Exact bc3c3d3 predecessor fitter" to say that only the loop is frozen and the numerical helpers are shared with production.
  • tests/unit/two_tier_free_count_legacy_fitter.rs#L3-L4: add a doc comment with the same scope statement.
  • tests/unit/two_tier_specific_projection_legacy_fitter.rs#L3-L7: remove "production fit statements retained verbatim," or limit it to the loop. If helper drift must be detected, add one pinned golden snapshot (for example, loglik_trace bits and the decision trace).
🤖 Prompt for AI Agents
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Review comment at @tests/unit/two_tier_final_person_legacy_fitter.rs around
lines 3 - 7:
Update the documentation for the three legacy fitter oracles to state that only
the outer EM loop is frozen and numerical helpers are shared with production. In
tests/unit/two_tier_final_person_legacy_fitter.rs, replace the “Exact”
predecessor claim with that scope; in
tests/unit/two_tier_free_count_legacy_fitter.rs, add the same scope statement;
and in tests/unit/two_tier_specific_projection_legacy_fitter.rs, remove the
claim that production fit statements are retained verbatim or limit it to the
loop. Do not add a golden snapshot unless needed to detect helper drift.

  • 🪄 Fix CodeRabbit comments on this PR
🤖 Prompt to fix review comments
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Inline comments:
Review comments at @docs/fipc-convergence.md:
- Around line 116-119: Update the convergence-contract status statements in the
low-Q and successor sections to reflect the merged
`fipc_fixed_support_is_independent_of_focal_remapping` test and the reported
local Rust run results, using one dated evidence statement while preserving the
disclaimer that local runs do not constitute GPU or recovery acceptance. Also
replace both “seed-1000” references with the current seed, 1039.

Review comments at @FIPC_CLASS_A_ROOT_CAUSE.md:
- Around line 29-34: Revise the root-cause record’s description of joint
backtracking so it says the accepted update must not decrease the remapped
likelihood, and clarifies that generating fixture values are not optimizer
inputs. Add that mean-first recovery from alpha 1.0, followed by scale recovery,
runs before recording a full rollback.

Review comments at @tests/test_two_tier_grm.py:
- Around line 334-343: Rename
test_fipc_seed1000_damping_does_not_report_convergence to identify seed 1039,
matching the fixture’s docstring. Update the seed references in the FIPC
convergence documentation to distinguish the seed-1039 test from any seed-1000
results, and identify which seed produced the recorded Metal evidence.

---

Nitpick comments:
Review comments at @tests/unit/two_tier_final_person_legacy_fitter.rs:
- Around line 3-7: Update the documentation for the three legacy fitter oracles
to state that only the outer EM loop is frozen and numerical helpers are shared
with production. In tests/unit/two_tier_final_person_legacy_fitter.rs, replace
the “Exact” predecessor claim with that scope; in
tests/unit/two_tier_free_count_legacy_fitter.rs, add the same scope statement;
and in tests/unit/two_tier_specific_projection_legacy_fitter.rs, remove the
claim that production fit statements are retained verbatim or limit it to the
loop. Do not add a golden snapshot unless needed to detect helper drift.

After applying the fix, consider running `coderabbit review --agent` for local
review. Visit https://docs.coderabbit.ai/cli?utm_source=ghpr

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: ContextualWisdomLab/fast-mlsirm/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c091f8ce-0ac7-45bd-a527-8b0b41945411
📥 Commits

Reviewing files that changed from the base of the PR and between d72ad5e and 0e84fd5.

⛔ Files ignored due to path filters (1)
  • docs/papers/chalmers-2012-mirt.pdf is excluded by !**/*.pdf
📒 Files selected for processing (57)
  • FIPC_CLASS_A_ROOT_CAUSE.md
  • RUST_ACCEPT_PATH_CAPTURE_3fc6160a.json
  • RUST_ACCEPT_PATH_CAPTURE_3fc6160a.md
  • contracts/s1-fipc-consumer-verify-v1.schema.json
  • crates/fast-mlsirm-py/src/lib.rs
  • crates/mlsirm-core/src/bifactor_full_fipc.rs
  • crates/mlsirm-core/src/bifactor_grm.rs
  • crates/mlsirm-core/src/gpu.rs
  • crates/mlsirm-core/src/gpu_bifactor.rs
  • crates/mlsirm-core/src/gpu_specific_tree.wgsl
  • crates/mlsirm-core/src/poly.rs
  • crates/mlsirm-core/src/two_tier_grm.rs
  • crates/mlsirm-core/tests/bifactor_fipc_tests.rs
  • crates/mlsirm-core/tests/bifactor_full_fipc_tests.rs
  • docs/bifactor-full-fipc.md
  • docs/changelog.d/two-tier-reference-scoring.md
  • docs/changelog.d/two-tier-single-primary-logprob-reuse.md
  • docs/fipc-convergence.md
  • python/fast_mlsirm/_legacy_init.py
  • python/fast_mlsirm/bifactor_grm.py
  • python/fast_mlsirm/two_tier_fipc.py
  • python/fast_mlsirm/two_tier_grm.py
  • scripts/s1_fipc_consumer_verify.py
  • tests/test_bifactor_fipc_device_transport.py
  • tests/test_bifactor_fipc_full.py
  • tests/test_gpu_specific_lookup_contract.py
  • tests/test_pypi_gap_guard_workflow_contract.py
  • tests/test_s1_fipc_consumer_verify.py
  • tests/test_two_tier_expected_raw.py
  • tests/test_two_tier_fipc.py
  • tests/test_two_tier_fipc_native_response_admission.py
  • tests/test_two_tier_grm.py
  • tests/test_two_tier_reference_prior_admission.py
  • tests/unit/gpu_bifactor_consumed_moment_tests.rs
  • tests/unit/gpu_bifactor_count_tests.rs
  • tests/unit/gpu_specific_leaf_partition_tests.rs
  • tests/unit/two_tier_final_person_legacy_fitter.rs
  • tests/unit/two_tier_final_person_projection_tests.rs
  • tests/unit/two_tier_fixed_block_marginal_legacy.rs
  • tests/unit/two_tier_fixed_block_marginal_tests.rs
  • tests/unit/two_tier_fixed_moment_output_tests.rs
  • tests/unit/two_tier_focal_cell_cache_legacy.rs
  • tests/unit/two_tier_focal_cell_cache_tests.rs
  • tests/unit/two_tier_focal_likelihood_only_tests.rs
  • tests/unit/two_tier_free_count_legacy_fitter.rs
  • tests/unit/two_tier_free_count_projection_tests.rs
  • tests/unit/two_tier_full_block_marginal_tests.rs
  • tests/unit/two_tier_gpu_consumed_moment_tests.rs
  • tests/unit/two_tier_gpu_scalar_moment_tests.rs
  • tests/unit/two_tier_gpu_table_cache_tests.rs
  • tests/unit/two_tier_grm_tests.rs
  • tests/unit/two_tier_item_line_search_legacy.rs
  • tests/unit/two_tier_item_line_search_output_tests.rs
  • tests/unit/two_tier_scalar_block_pattern_tests.rs
  • tests/unit/two_tier_specific_projection_legacy_fitter.rs
  • tests/unit/two_tier_specific_projection_tests.rs
  • tests/unit/two_tier_zero_primary_mass_tests.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Comment thread docs/fipc-convergence.md Outdated
Comment thread FIPC_CLASS_A_ROOT_CAUSE.md Outdated
Comment thread tests/test_two_tier_grm.py
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review

Please perform the substantive existing self-hosted Noema review for current head 3fcdfbb44f1a99bfc089f10e83132d08d16f8e57 against base d72ad5e7f7398adfeb4ab2e1af58a8de33897cdd, using the deployed auto route and normal independent App publication. This requests an actual review, not unconditional approval or a required-check waiver.

The e481 review findings were repaired and locally checked; the 0e84 review then identified documentation/seed-label issues. The current successor fixes those scope statements and renames the damping fixture to its actual seed1039. Python test AST is unchanged except the name; three legacy Rust fixture bodies are unchanged except doc comments. The fixed-support CPU diagnostic executed on source-matching retained bytes; older Metal evidence remains explicitly seed1000 and is not relabelled.

Previous synthetic source/native kernel checks remain bounded. Full-consumer/scientific recovery, release-wheel, higher-Q stabilization, and two-host GPU acceptance are still unmet; neither local checks nor prior CodeRabbit COMMENTED reviews are formal approval. Preserve finite/full-step/absolute-likelihood/no-rollback and negative recovery gates.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant