Skip to content

fix(install): reject one owner/repo from two hosts before fetching (#3696) - #3895

Merged
Daniel Meppiel (danielmeppiel) merged 9 commits into
mainfrom
danielmeppiel-fix-3696-host-collision
Nov 19, 2026
Merged

Daniel Meppiel (danielmeppiel) merged 9 commits into
mainfrom
danielmeppiel-fix-3696-host-collision

Conversation

@danielmeppiel

Copy link
Copy Markdown
Collaborator

Description

apm_modules/ is host-blind by contract, so acme/tool (github.com) and gitlab.example.com/acme/tool both map to apm_modules/acme/tool. On main the install either failed with the race-dependent Path already has a replacement in progress error or exited 0 after reusing the other host's copy, which deployed the wrong host's content.

Resolution now claims each package directory through the dependency materialization owner (MaterializationSlots in models/dependency/materialization.py) before it fetches or stages anything, for root and transitive declarations. When two distinct identities (get_unique_key differs) claim one directory, install, update and dry-run fail with one error that names both declarations, their hosts, and the declaring parent:

github.com/test/tool and git.example.test/test/tool (required by test/mid) are different packages that would both install to apm_modules/test/tool. Remove one of them, or set 'alias:' on one so it installs to its own directory.

These cases still work as before: the same identity at different refs (the version-conflict path), local sources that share a _local/... slot (#3401, owns_path), and alias: declarations, which install both packages. The root nested-directory check (#3699) and the new check share one helper in the resolver. The contracts-tooling-dependency-identity architecture guard now also requires this owner path, with mutation cases. Docs: manifest-schema reference and the apm-usage dependencies.md guide.

Not changed: adopt/discovery.py keeps its own collision check. Onboarding compares physical local directories, which is a stricter question than install-time identity.

Issue and approved scope

Issue: Fixes #3696

Human scope-approval comment: #3696 (comment)

Does this PR complete the issue, or what remains? It completes the issue for the "clear error naming both declarations" outcome. A host-qualified layout is out of scope (layout/lockfile design change). Deferred: apm deps list labels both rows of an aliased two-host pair with the non-default host.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

Validation evidence (HEAD merged with main f8ef817930; main had not moved):

  • Reporter scenario with the real CLI: on main, .venv/bin/apm install --target copilot fails with Path already has a replacement in progress: .../apm_modules/acme/tool. With this PR it fails before fetching with the error above and writes nothing.
  • APM_BINARY_PATH=.venv/bin/apm .venv/bin/python -m pytest tests/integration/test_same_repo_two_hosts_lifecycle.py tests/integration/test_nested_package_declaration_lifecycle.py -p no:xdist -q: 5 passed. The new file fails on unfixed src/ (2 failed: the direct case exits 0, and the transitive case shows the old error).
  • .venv/bin/python -m pytest <22 resolver test files> tests/unit/models/test_dependency_materialization.py tests/unit/scripts/test_architecture_runner.py -p no:xdist -q: 803 passed.
  • .venv/bin/python -m pytest tests/integration/test_architecture_dependency_reference.py -k "shared_install or nested_root" -p no:xdist -q: passed. tests/integration/test_architecture_owner_rule_mutations.py -k dependency_identity: 5 passed.
  • uvx ruff@0.15.12 check and format --check on src/ tests/ and the linter scripts: passed.
  • python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 ...: 10.00/10.
  • bash scripts/lint-auth-signals.sh: passed. bash scripts/lint-architecture-boundaries.sh: passed.

Spec conformance (OpenAPM v0.1)

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
    (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
    row).
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
    updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under
    tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated via
    uv run --extra dev python -m tests.spec_conformance.gen_statement
    and committed.
  • N/A -- this PR does not change OpenAPM-observable behaviour.

…3696)

apm_modules is host-blind, so acme/tool from github.com and
gitlab.example.com/acme/tool both map to apm_modules/acme/tool. Resolution
now claims each package directory through the materialization owner
(MaterializationSlots) before fetching or staging it, and fails with one
error naming both declarations, their hosts and the declaring parent,
instead of a race ("Path already has a replacement in progress") or a
silent reuse of the other host's copy. Same-identity refs and shared local
slots (#3401) are unaffected.

apm-spec-waiver: resolver pre-fetch rejection of a host-blind layout collision; no OpenAPM requirement changes

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 19, 2026 05:41
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The main merge resurrected Fixed entries that #3886 rewrote; restore
main's CHANGELOG and add only this PR's line.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Root preflight incorrectly claims aliases from discarded duplicate declarations, causing valid dependency configurations to fail with false destination collisions.

1 open issue
What changed in this PR

This PR addresses #3696 by detecting when distinct dependencies would share one host-blind apm_modules directory, replacing staging failures or incorrect package reuse with a clear collision error.

Changes:

  • Adds directory claims through the materialization owner for root and transitive resolution.
  • Preserves local-source sharing and supports aliases for separate destinations.
  • Adds regression coverage, architecture checks, and dependency documentation.
File Description
tests/​unit/​models/​test_dependency_materialization.py Tests cross-host and alias collisions, plus local sharing.
tests/​integration/​test_same_repo_two_hosts_lifecycle.py Covers install, update, dry-run, transitive errors, and aliases.
tests/​integration/​test_architecture_dependency_reference.py Adds mutations checking materialization-owner enforcement.
src/​apm_cli/​models/​dependency/​materialization.py Adds directory claims and collision diagnostics.
src/​apm_cli/​deps/​apm_resolver.py Checks root and selected transitive destinations before package loading.
scripts/​architecture_linter/​checks/​contracts_test_taxonomy.py Requires resolver use of the shared directory-claim owner.
packages/​apm-guide/​.apm/​skills/​apm-usage/​dependencies.md Documents collisions and alias guidance.
docs/​src/​content/​docs/​reference/​manifest-schema.md Explains host-blind paths and collision handling.
CHANGELOG.md Records the cross-host collision fix.
.apm/​architecture/​owners/​contracts-tooling.json Registers directory-collision enforcement under the existing owner.

Review effort: Balanced

Comment thread src/apm_cli/deps/apm_resolver.py Resolved
…lots

ReplacementIndex.bind kept its own apm_modules slot-occupancy check that
ran only when replace: rules exist, so the #3696 two-host collision gave a
different, replacement-worded error (and no local-slot exemption) depending
on an unrelated manifest field. Drop that half; MaterializationSlots owns
shared-directory rejection and labels the original identity. The bind
check keeps only same-unique-key identity ambiguity. The identity guard now
rejects slot occupancy in replacement.py. Lifecycle tests also cover a
shared alias: and apm update after a parent republish.

Addresses panel follow-ups (python-architect, test-coverage-expert).

apm-spec-waiver: resolver pre-fetch rejection of a host-blind layout collision; no OpenAPM requirement changes

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The root preflight claimed every root declaration, including duplicate
declarations the resolver discards, so `acme/tool` declared with aliases
`primary` and `shared` beside `acme/other` aliased `shared` failed with a
false collision even though `acme/tool` installs to `primary`. Phase A
already claims each level's selected nodes before any fetch, so the root
preflight now only rejects nested declarations (#3699).

Addresses Copilot inline on src/apm_cli/deps/apm_resolver.py:1138.

apm-spec-waiver: resolver pre-fetch rejection of a host-blind layout collision; no OpenAPM requirement changes

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

APM Review Panel: ship_with_followups

Installs and updates now fail before any fetch, with one clear error, when two different packages would land in the same apm_modules directory. This closes the silent cross-host reuse and the parallel race reported in #3696.

panel-mode=lean; personas=python-architect,test-coverage-expert,apm-ceo

cc Daniel Meppiel (@danielmeppiel) Sergio Sisternes (@sergio-sisternes-epam) -- a fresh advisory pass is ready for your review.

This PR turns a silent integrity bug into a clear manifest error. On main, acme/tool from github.com and gitlab.example.com/acme/tool could share apm_modules/acme/tool. A sequential install reused the other host's copy and exited 0; a parallel install failed with a race error. On the fixed CLI the install fails before anything is fetched. The error names both host-qualified declarations and the parent that required each one, and it suggests alias: or removing one. The installed-CLI lifecycle test tests/integration/test_same_repo_two_hosts_lifecycle.py covers the direct and via-parent cases with the lock and modules unchanged; that evidence carries the most weight.

The panel agreed on the design: MaterializationSlots is the right owner, and claiming only the winning nodes is correct. Both recommended findings are folded on 2def842. In 3f0df55, ReplacementIndex.bind no longer runs its own slot check, so MaterializationSlots is the only place that rejects a shared directory, with or without replace: rules. bind now checks only for ambiguous identities on one unique key, and the identity guard (mutation-break verified) stops a slot check from returning to replacement.py. The same commit closes the coverage gaps: the lifecycle suite now covers a shared alias: collision and apm update after a parent republishes. In 2def842, the Copilot inline finding is fixed: root preflight no longer claims directories for discarded duplicate root declarations, and only selected nodes claim before fetching (test_actual_resolver_claims_only_the_selected_alias_destination, mutation-break verified).

One gap is left out on purpose. adopt/discovery.py keeps its own exact-path slot check, which belongs to the onboarding owner record rather than the install-time scope of #3696. For remote dependencies that differ only in letter case, apm discover/init can accept a manifest that install then rejects, but install still fails cleanly and early.

Aligned with: portable by manifest (the same manifest gives the same result regardless of install order or parallelism); secure by default (code from one host can no longer silently stand in for a same-named package from another); multi-host (the error labels each package with its host); pragmatic as npm (one actionable error with a one-line fix).

Panel summary

Persona B R N Takeaway
Python Architect 0 2 0 MaterializationSlots is the right owner and claim semantics hold; replacement bind and adopt discovery kept their own slot checks (bind folded, adopt deferred).
Test Coverage Expert 0 2 0 Installed-CLI lifecycle test traps the direct and via-parent install reject; shared-alias and via-parent update traps added (folded).

B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.

Folded in this run

  • Replacement bind slot check collapsed into MaterializationSlots, with a static guard and mutation case -- 3f0df55
  • Lifecycle coverage for a shared alias: collision and for apm update after a parent republish -- 3f0df55
  • Root preflight claims only selected nodes (Copilot inline, thread resolved) -- 2def842
  • CHANGELOG kept the release captain's text plus one line for this PR -- cceac04

Copilot signals reviewed

  • LEGIT: root preflight claimed aliases from discarded duplicate root declarations (false collision). Folded in 2def842.

Top 2 follow-ups

  1. [Python Architect] Route the adopt/discovery.py slot check (apm discover/init) through MaterializationSlots, or match its case-folded comparison, under the onboarding owner record. -- Onboarding and install should give the same answer. Deferred: this crosses into the onboarding-metadata-only owner, outside [BUG] The same owner/repo from two hosts collides in apm_modules with "Path already has a replacement in progress" #3696's install-time scope.
  2. [Python Architect] Track separately: get_unique_key ignores the port, so github.com:8443/x/y and x/y share an identity key. -- This is a pre-existing identity-key flaw outside [BUG] The same owner/repo from two hosts collides in apm_modules with "Path already has a replacement in progress" #3696 that can hide a real collision.

Recommendation

Both recommended panel findings and the Copilot inline thread are folded on 2def842. Lint is clean, the lifecycle and mutation-verified tests pass, and CI is green. Next step is the requested CODEOWNER review (Sergio Sisternes (@sergio-sisternes-epam)).

PR head stance iterations folds deferrals copilot rounds CI mergeable merge state notes
#3895 2def842 ship_with_followups 1 3 2 2 green MERGEABLE BLOCKED pending required review

Full per-persona findings

Python Architect

  • [recommended] ReplacementIndex.bind keeps a second slot-collision check that runs before MaterializationSlots whenever any replace: rule exists at src/apm_cli/models/dependency/replacement.py:350
    The bind key was not case-folded and had no local-vs-local exemption, and its error mentioned replacement for packages that were not replaced. So the answer to one decision depended on an unrelated manifest field. Folded in 3f0df55.
  • [recommended] adopt/discovery.py slot check uses exact paths, so adopt can call a manifest safe that install now rejects at src/apm_cli/adopt/discovery.py:131
    Remote case-variant collisions are decided in two places. Deferred to the onboarding owner.

Test Coverage Expert

  • [recommended] The new reject for two remote deps sharing one alias: had unit-only coverage.
    Proof (missing at): tests/integration/test_same_repo_two_hosts_lifecycle.py::test_alias_installs_both_packages -- proves: two different packages that set the same alias: fail install with one clear error. Folded in 3f0df55.
  • [recommended] The via-parent collision was tested only for install, not update.
    Proof (missing at): tests/integration/test_same_repo_two_hosts_lifecycle.py::test_transitive_declaration_names_its_parent -- proves: apm update fails clearly when a parent's new release pulls the same owner/repo from another host, leaving the install untouched. Folded in 3f0df55.

This panel is advisory. It does not block merge. Re-apply the
panel-review label after addressing feedback to re-run.


Generated by autopilot-pr-review-worker. This comment is AI-generated and may contain errors.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

Lane L: main merged for landing

  • Pre-merge head: 2def84216fafa640b7eb4e929df38ab8d7a229a8
  • Merged origin/main d547a5bad020f061be0f1b4b7d4bd7dd9b02086e (clean merge; GitHub reported DIRTY only on CHANGELOG.md, which this repo merges with merge=union).
  • New head: 56700557dbe4ac67b6f970383b9223efc96caabf
  • CHANGELOG: slotted, so queued PRs no longer collide at the section head while landing serially.
  • Lint: ruff check + ruff format --check silent at the new head; CI green at the new head.
  • Owner-touch report regenerated at the new head; every recorded functional test was re-run at that exact head and passed (one process at a time, no xdist).
  • Push: `` (plain push, no force).

This comment was generated by an AI agent (Lane L landing desk) on behalf of Daniel Meppiel (@danielmeppiel).

@danielmeppiel
Daniel Meppiel (danielmeppiel) merged commit 178c9ea into main Nov 19, 2026
34 checks passed
@danielmeppiel
Daniel Meppiel (danielmeppiel) deleted the danielmeppiel-fix-3696-host-collision branch November 19, 2026 07:10
Daniel Meppiel (danielmeppiel) added a commit that referenced this pull request Nov 19, 2026
The new threat-model row credited #3895 with general dependency-confusion
protection, but it only catches one install graph declaring the same
owner/repo from two hosts; bare shorthand still resolves to the default
host. The rebound sentence also omitted lock rows without a recorded
origin, local marketplaces and in-repo moves, which the check skips.

Addresses supply-chain-security-expert panel findings and Copilot inline
on docs/src/content/docs/enterprise/security.md:323.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Daniel Meppiel (danielmeppiel) added a commit that referenced this pull request Nov 19, 2026
* docs: reground pages freed by #3918 and leftover 0.37 drift

Refs #3948. Wave 2 of the 0.37 docs sweep (docs-corpus-audit): the
pages #3918 owned (concepts, install-packages, enterprise policy and
security, mcp-as-primitive, reference update/manifest-schema, apm-usage
dependencies/governance) plus reference/cli/deps.md (#3916 refusal).
Pages owned by in-flight #3290 are not touched.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* docs(security): scope dependency-confusion and rebound claims to code

The new threat-model row credited #3895 with general dependency-confusion
protection, but it only catches one install graph declaring the same
owner/repo from two hosts; bare shorthand still resolves to the default
host. The rebound sentence also omitted lock rows without a recorded
origin, local marketplaces and in-repo moves, which the check skips.

Addresses supply-chain-security-expert panel findings and Copilot inline
on docs/src/content/docs/enterprise/security.md:323.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(policy): fix skill CLI samples and allow-list wording

The apm-usage governance CLI samples used header and footer lines the
CLI never prints and a dry-run overflow that cannot follow two lines
(the preview limit is 5); align them with policy-reference. Section 3
still described a repo override in the discovered chain, and the
allowlist rows said 'non-empty', contradicting null vs [] semantics.

Addresses doc-writer and supply-chain-security-expert panel findings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(install): qualify marketplace replay and default_host wording

Locked marketplace entries replay offline except for a package whose
locked MCP or LSP servers have no apm_modules copy; that case reads the
marketplace even under --frozen (drift.py catalog-metadata). Also tie
default_host to the shorthand rather than #ref and link the newer
lockfile troubleshooting section.

Addresses Copilot inline on reference/manifest-schema.md:701 and
doc-writer panel nit on consumer/install-packages.md:64.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(docs): track host-qualified policy wording in security.md

security.md no longer says policy patterns are host-blind or cannot
restrict host identity: dependencies.allow/deny accept host-qualified
patterns (policy-reference). The docs-contract test still pinned the old
wording and failed Build & Test Shard 1. Keep its intent (no
transport-aware policy claim) and assert the scheme-blind sentence and
the host-qualified link instead.

Addresses CI failure in tests/unit/test_tls_docs_scope.py:127.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(security): name the manifest whose default_host applies

Each parsed manifest, transitive ones included, resolves its own bare
owner/repo dependencies against its own default_host, so a root
default_host does not pin the whole graph. Say so in the
dependency-confusion row and point at host-qualified allow patterns as
the control.

Addresses supply-chain-security-expert delta panel finding on
docs/src/content/docs/enterprise/security.md:635.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: danielmeppiel <danielmeppiel@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] The same owner/repo from two hosts collides in apm_modules with "Path already has a replacement in progress"

2 participants