Skip to content

Repair original ML and crime exercises with explicit data and verified demos - #2

Merged
jadenfix merged 5 commits into
mainfrom
codex/leakage-safe-ml-demo
Oct 14, 2026
Merged

jadenfix merged 5 commits into
mainfrom
codex/leakage-safe-ml-demo

Conversation

@jadenfix

@jadenfix jadenfix commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

The original ml1.py referenced undefined data, model_selection.py required an implicit remote dataset, and the crime exercise depended on an absent author-local CSV while subtracting unmatched year rows. These entry points now accept explicit local CSVs or explicitly fictional offline fixtures, with the original source preserved byte-for-byte in archive/.

All ten original regression candidate families use training-only five-fold model selection and one disjoint holdout. Log-model predictions return to original target units. The crime exercise joins differences by borough/week, validates panel pairs and numerical inputs, rejects unidentified designs, and fits all three descriptive OLS specifications; the redundant sixweeks main effect is absorbed by the retained week fixed effects. The separate four-model demo workflow remains unchanged with verified GIF, MP4, poster and capture receipt.

Validation: fresh Python 3.12 installation and dependency check; 21 tests; actual fixture and local-CSV model executions; three OLS regressions on 1,040 matched fictional pairs; unchanged source/media provenance and full GIF/MP4 decoding. Hosted Python 3.10/3.12 execution passed both complete jobs and uploaded model receipts.

Reviewed head: 9391f2e3dec6704f3c2b907345714b3cffb22634; tree: 61540009979c11febbf25c59e9c2b7fdc9235f02. The existing media depicts the unchanged four-model workflow, rather than the newly added ten-model or crime runs. Original Boston/London files remain absent; fixture metrics do not reproduce those studies or establish real-world accuracy, causal effects or deployment performance.

Copy link
Copy Markdown
Owner Author

Exact-head implementation and provenance receipt for 08afbffa058047eeee0604916fd036687afe248e (tree 5b8833224d8f33a452a4dd30694b0591b3ba779f):

  • Base main: 9f7e7ac6365ab8eeb00b416aa1ab9be0f03238ab.
  • Maintained workflow: deterministic fictional 240-row housing-style fixture; 192-row training partition; five-fold out-of-fold selection across four candidate pipelines; 48-row holdout evaluated exactly once after selection.
  • Actual fixture receipt: winner quadratic_ridge; CV RMSE/MAE 1.447/1.169; holdout RMSE/MAE/R² 1.398/1.148/0.967. These are fixture observations only—not real-world accuracy, scale, deployment, latency, or performance claims.
  • Captured source commit 62a613e651dc44e18147c04ee6c20315e2f29d95, tree fb562784c10f1e2704226c8029f4c7e541e0b813; the final verifier proves it is an ancestor and that every recorded source blob is unchanged.
  • Local and clean exact-head clone: 12 tests passed, actual workflow reproduced, strict verifier passed, GIF and MP4 fully decoded, and git status --porcelain remained empty.
  • Hosted run 38431156700 is fully green on Python 3.10 and 3.12: tests, real workflow, and strict media verifier. The earlier runner-only ffmpeg absence was repaired by explicitly installing it; no gate was weakened.
  • GIF: 960×540, 17.0s, SHA-256 3ed3bf1f472ee6d9f14bdb60db917096851875dfe690d38963bec6a04f4219b3, Git blob 46c6c2023c49f501944e15d25d8c143d5641449a.
  • MP4: H.264 960×540, 17.0s/170 frames, SHA-256 119b774b994561f28f8c66aac722e8fb9bbe9aa78d1431432790ca21f5a4e43a, Git blob 527fb299097685bf97567ff615c631f8f437608b.
  • Poster SHA-256 9569967e1ac394fbc399eae32a3b5c1cf621b5e695593769b24fc6c0f21327a5; receipt SHA-256 1e7b3c1bff21c6ec6d0642e34488449439cc7151c4b527ef2430a58020a4023a.

Ready for the designated merger’s independent review. I did not merge.

Copy link
Copy Markdown
Owner Author

Independent review — CLEAR at exact head 08afbffa058047eeee0604916fd036687afe248e (tree 5b8833224d8f33a452a4dd30694b0591b3ba779f). I did not merge.

Evidence independently reproduced:

  • Base remains 9f7e7ac6365ab8eeb00b416aa1ab9be0f03238ab; PR is open and mergeable at the requested head.
  • Captured source 62a613e651dc44e18147c04ee6c20315e2f29d95 is an ancestor; its tree is exactly fb562784c10f1e2704226c8029f4c7e541e0b813.
  • Every pinned source blob matches both the recorded source and exact-head working file: workflow 868290c…, requirements 1a0f9d8…, generator 1369fd9…, verifier e277570…, tests f711ff9….
  • Independent exact-head run: 12/12 tests pass; actual CLI reproduces 192 training / 48 holdout rows, winner quadratic_ridge, CV RMSE 1.447, holdout RMSE 1.398 and R² 0.967; strict verifier passes; checkout remains clean.
  • Leakage boundary inspected in code: split occurs before selection; all preprocessing is inside candidate pipelines; five-fold OOF predictions use only x_train/y_train; the winner is fit only on training rows and holdout prediction happens afterward.
  • Additional adversarial probe changed every holdout feature and target to 9999 after recovering the deterministic split. Winner and all candidate CV metrics remained byte-for-byte equal, while only holdout metrics changed. This directly confirms holdout values cannot affect model selection.
  • Independently checked hosted run 38431156700: Python 3.10 and 3.12 jobs each passed tests, real CLI, full provenance/media verifier. GitGuardian is green. The explicit ffmpeg install preserves rather than bypasses decoding.
  • Independently regenerated from captured source: GIF, MP4, poster, receipt, and capture JSON reproduce the published SHA-256 values exactly.
  • Published GIF fully decodes at 960×540 / 17.0 s / 5 intentional scenes; MP4 fully decodes as H.264 960×540 / 17.0 s / 170 frames. I inspected a 7-frame contact sheet covering every scene; fixture labeling and limitations are visible.
  • Remote blobs match local bytes exactly: capture 8b71c0453bb42f8c585bb61c27007db331067e30, GIF 46c6c2023c49f501944e15d25d8c143d5641449a, MP4 527fb299097685bf97567ff615c631f8f437608b, poster 79fbcb7e038cd6d861ab0618d8c2a073c6d38de6, receipt dd79e2959a45fe8e4e249521f51a91a99bbadc7c.

Scope is truthful: deterministic fictional fixture only. The displayed metrics are observations from seed 8257, not claims about real data, accuracy, scale, deployment, latency, or production performance. No blocking findings.

@jadenfix

Copy link
Copy Markdown
Owner Author

Continuation review fixed the original exercises as well as the existing demo path. ml1.py and model_selection.py now run all ten original candidate families with explicit local CSV or explicitly fictional fixture, training-only model selection and original-unit log predictions. crimeee.py now joins year differences by borough/week, rejects mismatched pairs and invalid/rank-deficient designs, and fits all three descriptive OLS specifications without the redundant week-dummy main effect. Original source is preserved byte-for-byte in archive/.

Exact head: 5e4b29f2202f6b18801128efefe6d11372771c27; tree: 5170a02d8ac3790d9ef6c9c404698be1fdd9ecb9. Fresh Python 3.12 installation: 20 tests pass, actual ten-model fixture/local-CSV execution and 3 OLS regressions / 1,040 matched year pairs pass; dependency consistency passes. Hosted Python 3.10/3.12 run passed both complete jobs including original workflow execution and receipt artifacts.

The existing four-model GIF/MP4 inputs and receipt remain unchanged and their fail-closed/full-decode verifier passes. The GIF describes that four-model workflow; it does not depict the newly repaired ten-model or crime path. The absent original Boston/London files are not reproduced, and fixture coefficients are not evidence of real-world accuracy or causal effects. No merge performed.

@jadenfix jadenfix changed the title Add leakage-safe offline ML workflow and verified demo Repair original ML and crime exercises with explicit data and verified demos Oct 14, 2026
@jadenfix

Copy link
Copy Markdown
Owner Author

Final continuation follow-up rejects an empty crime panel before constructing an OLS design, and verifies invalid logarithm inputs. Frozen reviewed head is now 9391f2e3dec6704f3c2b907345714b3cffb22634, tree 61540009979c11febbf25c59e9c2b7fdc9235f02. Fresh local 21-test suite, original actual model/OLS CLI executions, and unchanged source-pinned full media verification pass. Final push CI and final PR CI both passed Python 3.10 and 3.12 with original workflow executions and uploaded JSON receipts. No further source changes planned in this owner lane; no merge performed.

Copy link
Copy Markdown
Owner Author

Independent current-head review: 9391f2e3dec6704f3c2b907345714b3cffb22634, tree 61540009979c11febbf25c59e9c2b7fdc9235f02. All 21 tests pass locally, and current-head Quality runs 38624356785 / 38624351035 are green. Executed both repaired ten-model entrypoints, three descriptive crime regressions with 1040 keyed pairs, original four-model workflow, and explicit CSV paths using generated synthetic inputs. Missing CSV commands return exit 2 without output. Shuffled crime CSV coefficients match fixture results within floating-point tolerance. Old source files are byte-identical in archive/, and the existing four-model media verifies all four artifacts.

Ready for the scoped offline workflows. Original Boston/London datasets are absent; CSV checks used synthetic fixtures. The existing GIF covers the separate four-model path. Fitted-model checkpoint/reload, deployment, causal identification, and real-world accuracy remain unproven. No owner-branch changes were made by this review.

@jadenfix
jadenfix merged commit 27985e5 into main Oct 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant