rfc(pnpr): immutable release records by zkochan · Pull Request #44 · pnpm/rfcs

zkochan · 2026-10-27T16:18:10Z

Summary

Proposes immutable package release records in transactional SQL metadata, with archives and large evidence bundles in the filesystem or object storage. npm-compatible metadata is a generated, cached view.

Defines a JCS/SHA-512 inner record derived only from authority, package identity, interpretation profile, and accepted artifact bytes. All endorsements, provenance, source signatures, and derivation evidence are detached; envelope validators track their changes separately.
Returns complete verification envelopes inline in extended metadata and bounded candidate queries. Cold client-side resolution can obtain candidates and their records together; standalone package-scoped retrieval remains available for auditing.
Separates package-index captures from per-version observations. Comparison keys survive unrelated version, tag, and evidence updates while changed install claims invalidate the relevant result.
Defines explicit upstream disagreement modes, SQL relational keys, portable backend guarantees, and per-version rendered fragments. Embedded SQLite is the local target; PostgreSQL and Turso remain shared-backend candidates subject to conformance and workload tests.

Paths are relative to the configured registry base, and package identity is in the path. The concrete examples preserve current pnpr npm bases, including /npm/~local/.

Preserves existing routing, artifact revision, tar-integrity, and source-signature contracts. File-level transfer remains with #9.

Related drafts

Validation

Reviewed against current pnpr routing and the existing revision, resolver, and storage contracts.
Prose lint, whitespace, RFC structure, local links, and attribution pass.
JSON examples parse; evidence separation, artifact binding, capability fields, candidate profiles, and package-scoped paths agree.
Only RFC text changed. Runtime protocol/database conformance, upstream corpus studies, and performance benchmarks remain implementation requirements.

Draft status

Intentionally remains a draft. The formats and routes are strawmen. Ratification still requires complete interpretation rules, endorsement/evidence linkage and freshness rules, authority/trust configuration, candidate/cursor and schema decisions, backend conformance, migration/backup procedures, and compatibility/performance measurements.

Written by an agent (Codex, GPT-6).

coderabbitai · 2026-10-27T16:18:20Z

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

zkochan · 2026-10-27T16:19:05Z

This remains a draft for discussion. Before ratification, settle the record encoding and manifest field contract, authority identity, signer trust, and upstream verification modes.

Companion drafts: #45, #46.

Written by an agent (Codex, GPT-6).

zkochan · 2026-10-27T16:55:31Z

Updated in 2e8764a. Added the SQL metadata and portable backend contract, generated-view caching, embedded SQLite as the local target, and Turso as a candidate shared backend. Backend replacement now requires verified migration and shared conformance tests.

This remains a draft. Record encoding, the manifest field contract, authority portability, signer trust, upstream verification modes, the SQL schema, backend conformance, and migration/backup procedures need design review before ratification.

Prose lint and document checks pass.

Written by an agent (Codex, GPT-6).

zkochan · 2026-10-27T18:02:23Z

Addressed the supplied review in fa3caac.

Added a JCS/SHA-512 record-envelope example, exact raw-input hashes, versioned interpretation rules, relational keys, and proposed capability/metadata/retrieval shapes.
Defined upstream disagreement handling and verification-download costs. Verification requests fail closed unless an operator explicitly accepts archive-authoritative metadata for the exact package/artifact/profile; that manifest must drive the graph.
Added PostgreSQL alongside Turso as a first-class shared-backend candidate. Neither is assumed conformant without transaction/session, durability, and workload checks.
Added per-version rendered fragments and clarified that full npm metadata responses still cost at least their emitted bytes.

The normalization concern is valid even though pnpr and the new pnpm v12 client are both Rust: protocol fixtures and named profiles must define behavior independently of a crate release. I did not adopt the unverified numerical claim about upstream mismatch prevalence or assert that changing databases is automatic. The draft requires a corpus study and backend conformance evidence.

The companion snapshot update pins the new interpretation and acceptance-policy inputs.

Validation: prose lint, whitespace, template/link checks, JSON parsing, and agreement among the protocol examples pass. Implementation conformance and benchmarks remain requirements, not completed tests.

Draft status is preserved. CodeRabbit skips draft review; no manual automated-review round was started. The supplied review has been addressed within scope. Complete field rules, signature/trust details, schema/protocol adoption, and runtime measurements remain before ratification.

Written by an agent (Codex, GPT-6).

zkochan · 2026-10-27T18:18:50Z

Addressed the new supplied review in 5ed610a.

Complete envelopes are inline in extended metadata and bounded candidate-query responses. Cold resolution can request candidates directly without a preliminary packument and a second record GET. Limits and paging avoid eagerly verifying every historical version. This removes a mandatory extra exchange; it is not a measured latency claim.
Package-index observations retain candidate/tag history. Separate per-version install/artifact observations key manifest comparisons, so unrelated publishes, tag changes, and evidence updates preserve those cache hits.
Removed evidence from the hashed inner record and made its artifact descriptor singular. Source signatures, provenance, derivations, and endorsements are detached. Their changes affect envelope validators and relevant rendered-view cache keys, not record integrity. Required evidence must still be authenticated and complete.
Moved package identity into the path and defined paths relative to the effective registry base. I retained /npm/: current endpoint documentation and the current router mount npm named registries under /npm/~<name>/. The older RFC examples omit that ecosystem prefix; removing it here would not match the implementation.

The companion snapshot revision pins both observation levels.

Prose lint, whitespace, document/link checks, JSON parsing, and consistency checks across envelope/capability/candidate/path examples pass. No runtime conformance or benchmark result is claimed.

Draft status is preserved. CodeRabbit skips draft review; no manual automated-review round was started. The supplied review is addressed within scope. Candidate/cursor details, profile rules, evidence/signature contracts, backend conformance, and runtime measurements remain ratification work.

Written by an agent (Codex, GPT-6).

…port Restore the archive-authoritative binding, the no-fallback rules, and the envelope cache key from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

#44 was split into #47, #48, and #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

zkochan · 2026-10-31T17:50:51Z

Closing as superseded: this draft was split into three that can be reviewed and accepted separately.

rfc(pnpr): publish-time manifest verification #47, publish-time manifest verification: deriving install-critical metadata from the archive for hosted packages. It doesn't depend on the others.
rfc(pnpr): SQL metadata storage #48, SQL metadata storage: insert-only manifests, mutable state (tags, revision selections, deprecations) stored beside them, and generated views cached as per-version fragments. rfc(pnpr): resolution snapshots and retained dependency sets #45 and rfc(pnpr): atomic workspace publication #46 build on it.
rfc(pnpr): verifiable release records for every ecosystem, on transparency logs, in-toto, and TUF #49, release records and upstream verification: content-addressed records, endorsements and evidence, interpretation profiles, upstream verification, and the client protocol.

#54, split packuments, builds on #48 and #49 for what clients download.

Written by an agent (Claude Code, claude-opus-5-5).

🤖 Generated with Claude Code

* rfc(pnpr): SQL metadata storage Split from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): key fragments by full identity and store profiles side by side Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): add evidence, tag history, README split, and read-after-write Add npm tracker evidence for the single-document model: 17.9 MB package documents, a 7-minute deprecate, and ETARGET after publish from separately cached full and abbreviated views. Add insert-only tag history and a separate README relation, and expose the committed package generation so a client can read its own publication. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): clarify blob-before-SQL invariant, selected_revisions, and generation scope --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* rfc(pnpr): publish-time manifest verification Split from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): store profile upgrades beside accepted manifests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): normalize npm publish documents before manifest comparison npm publish uploads the output of @npmcli/package-json prepare(), not the raw package.json, so a strict comparison would reject ordinary uploads (bin refs, the gypfile install-script injection, bundleDependencies true, version cleaning). Define these as profile normalizations, accept the one benign direction (submitted claims a build the archive cannot run), add name and version to the compared fields, and add evidence from the npm trackers plus a 1,674-package survey of registry metadata vs archives. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc: extend the manifest survey to the most-downloaded packages Add the latest version of the 5,000 most-downloaded packages and a random older version of each of the top 2,000. 273 of 8,620 versions differ before normalization; all are explained by publisher-side normalization, which adds empty-vs-absent maps, directories.bin expansion, stripped build metadata, and git shorthand expansion to the profile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): clarify lifecycle script comparison, engines normalization, and link #50 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* docs(pnpr): propose atomic workspace publication * docs(pnpr): define publication through SQL transactions * rfc(pnpr): build on the split SQL metadata storage draft #44 was split into #47, #48, and #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): add evidence, keyless retries, uploads by integrity, and workload batches Add npm tracker evidence for half-published releases and ghost-version conflicts. Name the pnpr gaps the transaction must close: a byte-identical retry returns VersionAlreadyPublished, one 100 MiB body carries every archive, and OIDC workloads can only PUT a single package. Upload archives by integrity before commit, treat content as the idempotency key for keyless clients, let workloads submit transactions, and let staging hold a whole transaction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): record committed publications for clients to read Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): advance the current reference on commit, backfill publications, keep the policy epoch across replacement Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): commit publications with releases, match whole keyless requests, recover outcomes after revocation, drain before export Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): settle transaction scope, ecosystems, keyless batches, and retention Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Split from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…port Restore the archive-authoritative binding, the no-fallback rules, and the envelope cache key from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…rency logs, in-toto, and TUF (#49) * rfc(pnpr): release records and upstream verification Split from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): specify signer validation and separate encoding from transport Restore the archive-authoritative binding, the no-fallback rules, and the envelope cache key from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): extend the fragment key only with the evidence generation Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): add publication evidence and first corpus results to release records The takeovers seen in practice (eslint-scope, axios) used stolen publisher credentials, which a registry-endorsed record does not detect. Add a registry-written publication statement (principal kind, OIDC issuer and claims, staging approver) as detached evidence, note that pnpr records only _npmUser.name for workload publishes today, and answer the corpus-study question with a first 1,674-package survey. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc: extend the manifest survey to the most-downloaded packages Add the latest version of the 5,000 most-downloaded packages and a random older version of each of the top 2,000. 273 of 8,620 versions differ before normalization; all are explained by publisher-side normalization, which adds empty-vs-absent maps, directories.bin expansion, stripped build metadata, and git shorthand expansion to the profile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): sign the release record pnpr already serves Split packuments, revisions, and platform variants merged with a release record served immutable at <name>/<version>+r<revision>. This draft now adds the fields that bind that record (schema, authority, revision, interpretation, input digests) instead of defining a second one, hashes the exact bytes the route serves, and keeps endorsements and evidence at <name>/<version>+r<revision>/evidence. The record covers every variant archive, its integrity is the tar's, publication evidence is the form of the version's publication, and pnpr computes the document's trust rank from it. The extended install-v1 representation, the digest route, and candidate lookup are dropped and listed as alternatives. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): build release record verification on existing standards Replace the draft's own mechanisms with the ones other ecosystems use: a transparency log in Go's checksum-database formats (C2SP signed notes, tiles, witnesses) answers the compromised-registry threat; endorsements and publication statements are in-toto Statements in DSSE envelopes over the exact stored bytes, so JCS is dropped; publisher provenance stays Sigstore evidence; TUF root metadata distributes and rotates a registry's keys; a registry is identified by its log's verifier key and a package by its Package URL. Proxied packages are served archive-derived records, as PEP 658 serves wheel metadata, and verification is on by default and fails closed, with exemptions as Go's GONOSUMDB allows. The settled questions leave the draft; witnessing a private log, a shared public log, re-derived profiles, and the cost of deriving proxied records remain open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): serve records by profile, and call inline entries projections A record re-derived under another interpretation profile is served at <name>/<version>+r<revision>/records/<profile>, and clients identify and cache records by PURL, revision, and profile. An inline package-document entry is a projection of the record beside the version's mutable state; verifying clients resolve from the record route. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): settle the remaining release record questions Each deployment runs its own log; a public registry's TUF root names the witnesses its clients require, and a private one may name none, its clients still detecting rewrites through the checkpoints they keep. pnpr keeps a proxied version's archive as long as its record, and the cost of deriving records is a benchmark, not an open question. A publish policy built on publication evidence is left to its own RFC. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): rewrap the release record benchmarks paragraph Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): release records for every ecosystem pnpr serves Records now have one core (schema, Package URL, revision where it exists, interpretation profile, input digests, artifact facts, derived manifest) and a profile per ecosystem deciding what one record covers, how its metadata is derived, its log entry, and its route: - npm: the record split packuments serves, covering every variant archive. - Cargo: the sparse index entry derived from the .crate's Cargo.toml, so the index Cargo resolves from is bound to the crate it downloads. - Python: one record per distribution file, since files can be added to a release; wheel metadata from .dist-info/METADATA as PEP 658 serves it, and source distributions only when PEP 643 makes their metadata static. - OCI: tag-to-digest bindings in the log, since images are already content-addressed and only tags move. Each configured registry has its own log, entries are looked up by PURL as Go's checksum database looks up modules, publication statements cover every ecosystem, PEP 740 attestations and OCI referrers are kept as provenance, and Cargo and Python surveys precede their verify mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): bind variant inputs and the .tgz, and tighten the log guarantees - Each variant entry carries its own inputs, and npm inputs name the published .tgz, so the logged record binds it to the tar digest and npm provenance is accepted only for that .tgz. - An inclusion proof does not show an identity was logged once: monitors catch duplicates over the witnessed log, and locked digests in the lockfile catch the rest, as with Go's checksum database. - Without witnesses, clients detect rewrites of what they saw but not split views; the text no longer claims otherwise. - A registry's authority is its log name and first TUF root digest, so key rotation does not change it. - Profiles are two path segments, and records are cached by authority as well as identity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): define OCI tag records, log sequencing, lookups, and key rotation - An OCI tag is identified by repository and tag; each setting is a new revision of that record, with the manifest digest as its artifact fact, served at -/records/<repository>/<tag>+r<revision>, and its evidence lists the OCI referrers. - Log entries are sequenced in the SQL metadata store in the acceptance transaction and integrated into tiles after commit; a record is served only once its entry has an inclusion proof. - A lookup returns every entry for a PURL, and the client selects the revision and profile it locked. - Clients verify against the current root only: statements are re-signed before a key is retired, and checkpoints stay valid through consistency. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): identify OCI tags explicitly and use a valid OCI Package URL An OCI Package URL names an image by digest, with the repository in repository_url and no namespace, so a tag record now carries repository and tag as its identity and the image it points at as a valid OCI PURL. Tags are looked up at -/log/lookup/<repository>/<tag>. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): key OCI duplicates by tag, and log tag deletions as tombstones The duplicate rule, record identity, and client cache key now use an OCI tag's repository and tag rather than its digest-bearing PURL. Deleting a tag, or the manifest it points at, writes a tombstone revision with no image, so a lookup finds no current binding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): define OCI tombstones, proxied tag observations, and private record caching A tombstone is a tag record with "purl": null, logged and signed like any other; statements name an OCI tag record as <repository>:<tag>+r<revision>. A proxied repository's log holds what pnpr observed when it resolved a tag upstream. A caller-gated record is cached only for the principal that fetched it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): OCI tag records omit inputs, log only tag changes, and get tag-binding statements Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): note that tombstones have no digest, and scope publication statements to released artifacts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): sign publication statements only for hosted publications A record pnpr derived for a proxied artifact gets no publication statement, since pnpr did not see that publication; its evidence is the upstream's, kept as fetched, and trust policy evaluates it as it does without pnpr. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rfc(pnpr): rewrap the publication statement paragraph Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

#44 was split into #47, #48, and #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs(pnpr): propose immutable release records

0af75cf

This was referenced Oct 27, 2026

rfc(pnpr): resolution snapshots and retained dependency sets #45

Closed

rfc(pnpr): atomic workspace publication #46

Merged

docs(pnpr): specify portable SQL metadata storage

2e8764a

docs(pnpr): specify release records and verification policy

fa3caac

docs(pnpr): refine record delivery and evidence identity

5ed610a

This was referenced Oct 27, 2026

rfc(pnpr): publish-time manifest verification #47

Merged

rfc(pnpr): verifiable release records for every ecosystem, on transparency logs, in-toto, and TUF #49

Merged

rfc(pnpr): SQL metadata storage #48

Merged

zkochan added a commit that referenced this pull request Oct 27, 2026

rfc(pnpr): build on the split storage and records drafts

88cb0ed

#44 was split into #47, #48, and #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

zkochan added a commit that referenced this pull request Oct 27, 2026

rfc(pnpr): build on the split SQL metadata storage draft

e365d37

#44 was split into #47, #48, and #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

zkochan mentioned this pull request Oct 31, 2026

rfc(pnpr): split packuments #54

Merged

zkochan closed this Oct 31, 2026

zkochan added a commit that referenced this pull request Nov 1, 2026

rfc(pnpr): release records and upstream verification

49f6fb7

Split from #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

zkochan added a commit that referenced this pull request Nov 1, 2026

rfc(pnpr): build on the split storage and records drafts

0d0dc94

#44 was split into #47, #48, and #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Navigation Menu

Repository navigation

rfc(pnpr): immutable release records - #44

rfc(pnpr): immutable release records#44
zkochan wants to merge 4 commits into
mainfrom
rfc/pnpr-immutable-release-records

zkochan commented Oct 27, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 27, 2026

Draft PR not reviewed

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 31, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

zkochan commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related drafts

Validation

Draft status

Uh oh!

coderabbitai Bot commented Oct 27, 2026

Draft PR not reviewed

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 27, 2026

Uh oh!

zkochan commented Oct 31, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

zkochan commented Oct 27, 2026 •

edited

Loading