Skip to content

Record a directory package's specifiers in the lockfile and make the frozen check an equality #228

Open

Description

@zkochan

Found by a defect-hotspot pass on 2026-10-29, prompted by pnpm/pnpm#17473. This task takes over the checker half (property (a)) of #220, which keeps the copy-contents half and the shared manifest reader.

Property

A lockfile pnpm just wrote passes pnpm install --frozen-lockfile while the manifests, hooks and settings are unchanged, and fails it exactly when a non-frozen install of the same tree would change the lockfile. This holds for every directory package: an injected workspace project, a file: directory, a publishConfig.directory, a workspace:../dir outside the workspace patterns, under shared and dedicated lockfiles, filtered or not.

Fixes in the loop

21 follow-up fixes blame to pnpm/pnpm#15356 between 2026-09-24 and 2026-10-27: pnpm/pnpm#15435, pnpm/pnpm#15668, pnpm/pnpm#16346, pnpm/pnpm#16404, pnpm/pnpm#16429, pnpm/pnpm#16559, pnpm/pnpm#16815, pnpm/pnpm#17025, pnpm/pnpm#17042, pnpm/pnpm#17047, pnpm/pnpm#17053, pnpm/pnpm#17062, pnpm/pnpm#17065, pnpm/pnpm#17067, pnpm/pnpm#17133, pnpm/pnpm#17136, pnpm/pnpm#17138, pnpm/pnpm#17140, pnpm/pnpm#17141, pnpm/pnpm#17143, pnpm/pnpm#17431. Related: pnpm/pnpm#17029, pnpm/pnpm#17375.

Issues: pnpm/pnpm#13836, pnpm/pnpm#16332, pnpm/pnpm#16428, pnpm/pnpm#16452, pnpm/pnpm#16453, pnpm/pnpm#16457, pnpm/pnpm#16557, pnpm/pnpm#17007, pnpm/pnpm#17008, pnpm/pnpm#17128, pnpm/pnpm#17129, pnpm/pnpm#17130, pnpm/pnpm#17374, pnpm/pnpm#17447, pnpm/pnpm#17473. Six carry the regression label.

pnpm/pnpm#17473 itself is fixed on main by pnpm/pnpm#17431, which v12.16.1 predates. Its regression test is in the PR linked from the issue.

Design flaw

The lockfile records a directory package's outputs, not its inputs.

  • An importer records each dependency's specifier beside its version. Its gate compares strings: pnpm_lockfile::satisfies_package_manifest in pnpm/crates/lockfile/src/freshness/manifest.rs and pnpm11/lockfile/verification/src/satisfiesPackageManifest.ts. Neither appears in the hotspot report.
  • A directory package records its snapshots: children and, since fix(deps-resolver): record every declared peer in the lockfile pnpm#17067, its declared peers. Nothing records the specifiers of the hooked manifest the resolver read.

So the check reconstructs them. pnpm/crates/package-manager/src/install/lockfile_freshness/directory_deps/ (about 1,300 lines without tests) with directory_manifests.rs, and pnpm11/lockfile/verification/src/localDirectoryDepIsUpToDate.ts (about 500 lines), locate the manifest, run the hook chain, derive the children with catalog anchoring and peer shadowing, and reconcile the result against the snapshot. The v12 check imports local_directory_child_specs, peer_shadowed_dependency_names, PeerSet, ChildCatalogs, ManifestTransformHooks, resolve_from_catalog and VersionsOverrider to do it. It is a second resolver, and v11 has a third.

Each fix in the loop is the resolver gaining or exposing an input axis the mirror lacked:

axis the mirror gained fix
peers resolved to workspace links, optional peers, catalog peers pnpm/pnpm#16346, pnpm/pnpm#16429, pnpm/pnpm#16559
which manifest describes the directory pnpm/pnpm#17042, pnpm/pnpm#17053, pnpm/pnpm#17133
the peer shadowing rule, the locked-version rule pnpm/pnpm#17029, pnpm/pnpm#17062
transitive directory deps, aliased children, tarball and git specs inside pnpm/pnpm#17065, pnpm/pnpm#17136, pnpm/pnpm#17141, pnpm/pnpm#17143
the hook chain, then its cost pnpm/pnpm#17047, pnpm/pnpm#17431, pnpm/pnpm#17375

Sharing more resolver pieces with the mirror, the approach of #145, #159 and item 1 of #220, cannot terminate: the mirror's input space is the resolver's. #145 named the alternative as its step 3 ("record an injected package's declared specifiers in its packages: entry; freshness becomes plain equality") and was closed without it.

Proposed change

  1. Writer, both CLIs in one release. Each directory package's packages: entry records the specifiers of the hooked manifest as the resolver read them, grouped as the snapshot groups children, beside the peerDependencies it already records. For the pnpm 12: frozen install rejects packageExtensions on injected local dependencies pnpm#17473 repro that is one line, host: workspace:../host, under peer@file:../peer. Precedent for a new field in lockfile 9.0 without a version bump: packageExtensionsChecksum and pnpmfileChecksum. PackageMetadata has no deny_unknown_fields, so an older v12 ignores the field, and the v11 reader is YAML and ignores it too.
  2. The check becomes equality. Walk the lockfile's directory snapshots from the importers. For each directory, read the current hooked manifest through the one reader the resolver uses (Compute an injected package's copy contents in one place, and read its manifest through one function #220 item 1, minus children and peers) and compare specifiers and dependenciesMeta with the importer comparison. Keep workspace: target satisfaction, shared with the importer link check (Share directory manifest loading and local spec satisfaction across resolver, freshness, and repeat-install #159 item 2). Children derivation, catalog anchoring, shadowing, alias handling, tarball and git comparison and the transitive manifest walk leave the check.
  3. Transition. An entry without the field was written by an older pnpm. A frozen install falls back to today's walk, kept as legacy with no new fixes and deleted at the next lockfile major, which is the temporary duplication PHILOSOPHY.md allows when the consolidation point is named. A non-frozen install re-resolves directory packages anyway (reusable_resolution in lockfile_reuse.rs never reuses them), so it writes the field on the next run. No frozen CI breaks on upgrade.
  4. Repeat-install gate. injected_deps_block_fast_path in optimistic_repeat_install/local_file_deps.rs skips the fast path for every dedicated-lockfile workspace with an injected dependency. With the field it compares instead, which closes item 3 of Share directory manifest loading and local spec satisfaction across resolver, freshness, and repeat-install #159.

This is the principle of #219 (workspace state) and #221 (peer reuse) applied to the lockfile: a shortcut compares recorded inputs and never re-derives outputs.

PRs: writer plus equality check plus fallback, one per CLI; the shared manifest reader; the test axes. About a week per CLI.

Tests that would have caught the loop

pnpm/crates/cli/tests/suite/injected_freshness_matrix.rs and injectedFreshnessMatrix.ts already check the property (a frozen install fails exactly when a non-frozen install changes the lockfile), but their rows are the bugs that already happened. pnpm/pnpm#17473 needed a hook source and a directory location they do not vary.

  • Add axes: hook source (none, packageExtensions, pnpmfile readPackage) and directory location (workspace project, workspace:../dir outside the patterns, file: outside the workspace, publish directory), beside layout and filter.
  • A writer/reader property: for a generated hooked manifest, what the writer records equals what the reader reads. A unit test, no install.
  • A fixture lockfile without the field pins the fallback. A v11/v12 differential pins the written field.

Fixes the change makes redundant


Written by an agent (Claude Code, claude-fable-5-1).

Activity

  1. zkochan commented on Oct 29, 2026

    @zkochan
    MemberAuthor

    First PR: pnpm/pnpm#17477 (writer, equality check with the derived walk as the fallback, both CLIs, and the test axes for the record itself).

    Decisions taken there:

    • The record is specifiers under the package's packages: entry, with dependencies and optionalDependencies maps, so the group a child moved to is a change. A directory package with no children records specifiers: {}, which tells a reader the record exists.
    • The record holds what manifest_child_specs (v12) and getNonDevWantedDependencies with the runtime conversion (v11) return, before peer shadowing and catalog resolution: bundled dependencies are left out, a name in both groups counts as optional, and engines.runtime entries count as dependencies. Writer and reader share that function.
    • With a record, the check still walks the local children (workspace:, link:, file: directories), since their targets drift without a specifier change. A registry child's locked version is the resolver's to judge. A catalog: child is left to the catalog check, which also leaves a catalog entry that names a workspace path unchecked.
    • An entry without the record is checked by the derived walk, and only a resolving install adds the record. A lockfile check does not trigger a resolution for a missing record, since the repeat-install gate, the run gate and the pnpr client share the non-frozen scope and every old lockfile would fail them at once.

    Found on the way: pnpm/pnpm#17476. The repeat-install fast path assumed that under a shared lockfile every injected dependency is a workspace project whose manifest it compares, the same assumption as pnpm/pnpm#17473, so an edit to an injected directory outside the workspace patterns was never installed. The PR fixes it by yielding for an injected directory no project vouches for.

    Left for the next PRs: the shared manifest reader (#220 item 1), the matrix axes for hook source and directory location, and deleting the derived walk once the record is universal.


    Written by an agent (Claude Code, claude-fable-5-1).

  2. zkochan commented on Oct 29, 2026

    @zkochan
    MemberAuthor

    Correction to the third decision above, after the first review round of pnpm/pnpm#17477: with a record, the check now compares the specifiers and then runs the same snapshot checks as an entry without one, in both CLIs. Registry children are therefore still held to the snapshot (a required child must be recorded, an undeclared one is a change), and catalog: children are resolved through the catalogs before the check decides whether to follow them into the workspace. Leaving registry children to the resolver and the catalog gap go with the derived walk's removal, once the record is universal.


    Written by an agent (Claude Code, claude-fable-5-1).

  3. zkochan commented on Oct 29, 2026

    @zkochan
    MemberAuthor

    One more case the record would cover: a directory package injected beneath a registry package by a parent-scoped override or a package extension. The freshness check walks directory dependencies from the importers, so it never visits that package, and a frozen install accepts its stale snapshot in both CLIs. Reported with a repro in pnpm/pnpm#17486. With specifiers on every directory package, the check can iterate packages: entries with a directory resolution instead of walking from the importers.


    Written by an agent (Claude Code, claude-fable-5-1).

  4. zkochan commented on Nov 4, 2026

    @zkochan
    MemberAuthor

    Decision: the recorded-specifiers writer and the equality check wait for pnpm v13.

    The precedent cited under "Proposed change" does not fit. packageExtensionsChecksum and pnpmfileChecksum were new top-level fields that appeared only once a project used the new feature. This change adds data to existing packages: entries. Every lockfile with a file: directory or injected dependency would be rewritten on the first install after an upgrade, and pnpm versions would disagree about an unchanged lockfile. That is a lockfile format change, so it belongs to a major version.

    Until then, #255 removes the copies of resolver rules from the v12 freshness check. The function it extracts also serves as the fallback for lockfiles without the field.


    Written by an agent (Claude Code, claude-opus-5-5).

  5. zkochan commented on Nov 8, 2026

    @zkochan
    MemberAuthor

    Status check against main (2026-11-08).

    The plan in the body (add the field without a format bump) no longer holds: the 2026-11-04 comment moved the writer and the equality check to v13. The open pnpm/pnpm#17477 is for the dropped plan. #255 is closed (pnpm/pnpm#17706, pnpm/pnpm#17720, pnpm/pnpm#17722), and pnpm/pnpm#17493 fixed the directory-beneath-a-registry-package case.

    Written by an agent (Claude Code, claude-opus-5-5).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions