Repository navigation
validation: bind Rubin executor inputs to immutable source snapshot digest #524
Description
Finding
At Draft #506 exact head f4a7eb5c018530e57a197ab2da56532a73e41456, Rubin projection activation binds source_snapshot_id and source_snapshot_sha256, but the numeric execution boundary does not yet establish that the admitted RubinLoadingObservation rows came from the bytes committed by that digest.
RubinLoadingObservation carries snapshot_id, mapped factor_score, complete-data indicator draws, and AvailableTime. execute_rubin_loading_uncertainty_run receives snapshot_id plus KnowledgeCutoff, admits rows by snapshot ID/time, computes estimator_payload_sha256, and emits the descriptive artifact. It does not receive or derive source_snapshot_sha256. Separately, decide_rubin_projection_activation accepts expected_snapshot_sha256 from its caller.
That means the current contracts can prove:
- the run consumed one exact admitted numeric estimator payload;
- the activation receipt names one immutable source snapshot digest;
- the logical snapshot ID matches across the relevant boundaries.
They do not yet prove that the numeric rows committed by estimator_payload_sha256 were derived from the immutable source bytes named by source_snapshot_sha256. Content identity for the estimator payload must not be silently promoted into source-lineage attestation.
Required RED
Create a realistic contract in which two source byte sets share the same logical snapshot_id but have different canonical SHA-256 digests. Keep the admitted Rubin numeric payload constant unless the test is specifically exercising derivation lineage. A projection activation must not become eligible merely because the caller supplies the approved source digest after execution when the executor/artifact has no provenance binding to that digest.
The RED should exercise the public execution/activation boundary, not private field mutation or a test-only bypass.
Minimal causal repair boundary
Choose one versioned owner contract that can establish source-byte lineage without copying source data or doing cross-service SQL. Acceptable designs include a bounded immutable source receipt/attestation consumed by analysis_engine, provided it is owner-controlled, digest-bound, cutoff-safe, and cannot be caller-backdated or represented by mutable branch/PR/latest locators.
Do not:
- hash arbitrary caller-supplied labels and call that provenance;
- treat
estimator_payload_sha256as generator/source attestation; - copy reusable static psychometric arithmetic out of
psychometric_core/ fast-mlsirm; - fill
PRODUCTION_APPROVED_PAIRINGSor emit positive projection before independent Validation Evidence promotion; - weaken exact-head review/check/security gates.
Acceptance
- source snapshot digest provenance is established before or as part of numeric admission, not appended only at projection-decision time;
- the completed artifact/receipt lineage can demonstrate which immutable source digest the admitted rows derive from;
- same logical snapshot ID + different source digest fails closed;
- malformed/noncanonical/mutable source authority fails closed;
- rolling cutoff semantics and
AvailableTimeremain unchanged; - exact-head Rust format/lint/test/rustdoc, authored line/branch coverage, repository contracts, PostgreSQL integration, and default-branch security/review gates are reacquired on the surviving vehicle;
- ADR 0034/research contract/TRACEABILITY are currentized without marking Draft evidence Accepted.
Queue-control note
Do not create another sequence of small pushes on Draft #506 while repository runner backlog is unresolved. Batch compatible RED → causal fix → tests/docs into one practical final head update; push immediately only to repair a failing required check, resolve a conflict, or answer an active review. This is runner conservation, not gate skipping.
Activity
seonghobae commentedon Sep 14, 2026
Owner-path decision before implementation: do not repair this by adding a caller-set
source_snapshot_sha256field toRubinLoadingObservation, and do not reuseestimator_payload_sha256. Both remain forgeable/semantically wrong as source-lineage authority. The minimal target is an Evidence-bounded-context source-snapshot receipt established before numeric admission, carrying immutable snapshot ID + canonical content SHA-256 + authoritativeAvailableTimeand a versioned receipt identity.analysis_engineshould retain an opaque execution binding derived from that receipt and emit the source digest/receipt identity in the completed Rubin artifact; projection activation should consume that execution binding rather than a free caller-supplied expected source digest. This keeps source content identity separate from the cutoff-admitted estimator payload and makes same logical snapshot ID + different source bytes observable even when admitted numeric values happen to be identical. The receipt authority must reject mutable PR/branch/latest locators and late/backdated availability.Current #506 remains untouched at
f4a7eb5c018530e57a197ab2da56532a73e41456: #523 is still the foundation prerequisite for runner conservation, and another active writer converted #523 back to Draft after the Ready lifecycle probe. I will not create a new small-push sequence on #506 before that prerequisite lands; RED + causal source/activation repair + tests + ADR/research propagation should be batched into one practical head update.seonghobae commentedon Sep 15, 2026
Foundation owner repair is now split correctly into Evidence instead of mutating Rubin observations or reinterpreting the estimator payload digest.
Opened #527 (
feat(evidence): own immutable source snapshot receipts) from protectedmain. Initial contract commit:a9dfcc503fa1ccd5de8291a36e27457a68c10428; immediate warning-root fix:96f490b7876134a19ce5b02d528607924f67b0f3(removed an unused directsha2import because workspace warnings are denied). The Evidence contract now ownsSourceSnapshotReceiptV1: canonical UUIDv7 receipt identity + immutable logical snapshot ID + canonical source-content SHA-256 + authoritativeAvailableTime, with a canonical receipt digest for opaque downstream binding. Same logical snapshot/different source bytes produces a different binding; mutable aliases/noncanonical digest/time/UUID/duplicate or unknown members/oversized wire fail closed.This does not complete #524. Remaining causal integration after the foundation is exact-head GREEN/reviewed/landed: analysis must consume the Evidence receipt before numeric admission, carry the receipt-derived binding in the Rubin execution artifact, and Projection must consume that execution binding instead of a free caller-supplied source digest.
estimator_payload_sha256remains a separate numeric-payload identity and must not be promoted into source provenance. #506 remains unchanged for now so the foundation prerequisite is reviewed first.seonghobae commentedon Sep 15, 2026
#527 foundation tightened after ADR-0008 owner review. The first receipt constructor still accepted a caller-provided
ContentDigest, which preserved byte identity but failed to force the independent Evidence record identity into the source binding. RED94e8b02ddaa9c93767530e0bdc8dd9083f233f3fadds the equal-content/distinct-SourceArtifactcase. Causal fixfcccabf3b9ccdbb592851f3467c9bd50fef525d4removes that free-digest minting path, constructs only from an already validatedSourceArtifact, and addssource_artifact_idalongsidesource_snapshot_sha256. Equal bytes can therefore retain equal content digests while distinct Evidence identities produce distinct receipt bindings, matching ADR 0008's identity-vs-digest separation.The remaining #524 integration boundary is unchanged: Analysis must consume this Evidence-owned binding before Rubin numeric admission and preserve it in the execution artifact; Projection must compare/consume that execution binding instead of accepting a free caller source digest. The receipt shape/content digest remains non-authoritative for ownership/signature/access/chain-of-custody.
seonghobae commentedon Sep 15, 2026
Foundation update from #527/#528: source-snapshot receipt creation no longer accepts a caller-selected receipt UUID. RED
4c273ef275f00baaa118bec8ff2404722cffa492showed one UUIDv7 could identify multiple independently created receipts; causal fix8c5df71660bb0907610f613d3e499c91eb17c8ddnow mintsEvidenceId::new()insideevidence_corewhile preserving strict UUIDv7 validation on persisted wire input.This strengthens the #524 owner path: downstream Analysis should consume a persisted Evidence-owned receipt/binding whose receipt identity, source-artifact identity, source digest, logical snapshot ID, and availability clock are already fixed. Do not reintroduce a free receipt ID when wiring Analysis/Projection. #527 remains Draft and non-GREEN; no positive projection authority changes.
seonghobae commentedon Sep 15, 2026
Follow-up scientific finding: #529 now explicitly owns the remaining backdating-safe availability gap. #527's current
available_atis only a recorded canonicalAvailableTime; because the public constructor still accepts it freely, it cannot yet satisfy #524's "cannot be caller-backdated" acceptance condition. Treat #529 as a prerequisite before wiring #527's receipt into Analysis admission or Projection authority.seonghobae commentedon Sep 15, 2026
#529 owner repair is now implemented branch-locally in #527 and sharpens #524's admissible integration boundary.
Do not make
RubinLoadingObservationcarry another free timestamp/digest. After #527 lands, Analysis should consume the Evidence-ownedSourceSnapshotReceiptV1(minted fromSourceObservation→SourceAvailability) before numeric admission, retain its canonical binding SHA-256 plus immutable source identity/digest in the execution artifact, and require itsAvailableTimeto satisfy the existingKnowledgeCutoffpolicy. Projection should then compare/consume that execution-bound receipt identity rather than accepting an independent caller-selected source SHA-256 as scientific lineage authority.Important clock distinction from the repaired foundation:
SystemTimeis minted at Evidence observation,AvailableTimeis minted separately when the observed source becomes available to analysis, and the owner contract enforcesavailable_at >= system_observed_at; it does not collapse the clocks. Persisted JSON validation is not self-authentication, so Analysis must consume trusted owner state or a repository/service-authenticated receipt, not arbitrary JSON from a caller.#527 current branch lineage includes canonical-wire RED/fix
5548f124...→29450bd9...and availability-owner RED/fix841c7cade...→032b9d7d...→ receipt binding261a3eaf..., with public contract propagation through965db4cf.... Do not integrate this into #506 until #527 has exact-head GREEN and protected-main landing.seonghobae commentedon Sep 15, 2026
#524 owner-path prerequisite #527가
df3ef9ec58ae555092368f4db72c008d8a6e3a15로 전진했습니다. #530 RED7cdaa414...→ fixdf3ef9ec...에서 source receipt binding이 이제 source-artifact/content뿐 아니라 exact Evidence-ownedsource_observation_id와source_availability_id까지 commit합니다. 두 owner IDs와 clocks는 Evidence 내부에서 생성되며 caller가 주입할 수 없습니다.#527가 protected main에 정상 landing된 뒤 Analysis integration은 owner-issued
SourceSnapshotReceiptV1자체를 numeric admission 전에 소비하고 그 opaque binding을 execution artifact에 보존해야 합니다. free source digest/owner-record ID 또는ValidatedSourceSnapshotReceiptWireV1만으로 scientific authority를 구성하면 안 됩니다.estimator_payload_sha256은 계속 admitted numeric payload identity로 분리합니다.#527 hosted exact-head gates/independent approval은 아직 별도 확인이 필요하므로 #524는 계속 blocked/open 상태가 맞습니다.
seonghobae commentedon Sep 15, 2026
#534 is now an explicit prerequisite to this source-lineage acceptance path. #527's receipt itself separates validated-but-untrusted wire from owner-issued state, but the
SourceArtifactit consumes can still be reconstructed directly from public arbitrary wire with a caller-selected validartifact_id;SourceObservation::observethen copies that ID into the owner observation/availability/receipt chain. The content SHA-256 remains integrity-checked, so this is specifically identity trust provenance, not a digest-forgery finding. Rubin numeric admission should consume the owner-issued receipt only after #534 closes that predecessor trust-state split; do not compensate by accepting free artifact IDs/digests at Analysis or Projection.seonghobae commentedon Sep 15, 2026
Evidence prerequisite #534 has a branch-local RED→fix on #527:
9fdc35e4...→fc21cdc5...with exact follow-up head74a986d808dd595452cad2ec05bba161e9fc97d5.Downstream Analysis must not reintroduce the removed trust leak.
ValidatedSourceArtifactWireis structurally/canonically validated but untrusted and is not admissible toSourceObservation; only owner-issued/restoredSourceArtifactmay enter observation. There is currently no public wire→owner promotion API, so #524 should consume only a landed owner-issuedSourceSnapshotReceiptV1path after protected-main verification, not artifact wire JSON, caller artifact IDs, or free source digests.Keep
estimator_payload_sha256distinct from source-content digest and receipt binding. #527 is still Draft with hosted gates queued, so this is a handoff, not released authority.seonghobae commentedon Sep 15, 2026
Evidence prerequisite update (still Draft / not protected authority): #527 has advanced ordinary-forward to
b9b464dcf82c6020a79f4ddc20b9555f3ef81497. The predecessor chain now prevents caller-selected artifact/document UUIDs from becoming trusted Evidence state:SourceArtifact::id()yields nominalSourceArtifactId;DocumentRecord::from_textrequires that handle; generic source/document JSON remains validated-but-untrusted wire with no owner-state promotion. Raw JSON envelopes are also bounded before deserialization allocation.For #524, do not consume
SourceArtifactId,ValidatedSourceArtifactWire, orValidatedDocumentRecordWireas Analysis provenance inputs. The integration boundary remains the owner-issuedSourceSnapshotReceiptV1created after source observation/availability, consumed before Rubin numeric admission, with its opaque binding preserved into the execution artifact. Projection should then consume that execution-bound lineage rather than a free caller digest/ID. #527 is still Draft and must first achieve exact-head gates + protected-main landing; no positive Rubin projection authority is opened by this branch-local repair.