Skip to content

feat(marketplace): declare marketplaces in the user manifest, reconciled by apm install -g - #3610

Merged
Daniel Meppiel (danielmeppiel) merged 40 commits into
mainfrom
danielmeppiel-issue-3006-user-manifest-marketplaces
Nov 13, 2026
Merged

Daniel Meppiel (danielmeppiel) merged 40 commits into
mainfrom
danielmeppiel-issue-3006-user-manifest-marketplaces

Conversation

@danielmeppiel

@danielmeppiel Daniel Meppiel (danielmeppiel) commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

TL;DR

The user manifest (~/.apm/apm.yml) can now declare a marketplaces: list, and apm install -g reconciles it into the marketplace registry. It registers missing entries, updates changed ones and removes only the entries it added, so registrations made with apm marketplace add are never touched. Project and package manifests that declare marketplaces: fail with an error that names the user manifest. No new protocol and no new source grammar: each entry reuses the apm marketplace add SOURCE parser.

Note

Local preflight skipped by maintainer instruction (2026-11-12); CI is the gate.

Problem (WHY)

  • Teams replay the same apm marketplace add commands on every machine (the issue cites 9 registrations per machine) and have no clean way to remove one centrally.
  • The registry (~/.apm/marketplaces.json) does not record who added an entry, so any automated sync could overwrite or delete a developer's manual registration.
  • The approved scope (Atlas decision 16, record issuecomment-6823498952) picks the user-config model (Brewfile tap, .npmrc, Cargo [registries]) over the marketplace-directory protocol proposed in the issue body.

Approach (WHAT)

Concern Decision
Where it lives User manifest only, at the existing effective user root (get_manifest_path(InstallScope.USER)); one resolver call in marketplace/user_manifest.py
Entry shape name + source (any apm marketplace add SOURCE) + optional ref (mirrors --ref); unknown keys and duplicate names rejected
Ownership New origin on each registry entry: manifest or manual (absent = manual, so existing files stay byte-identical)
Reconcile Runs at the start of apm install -g, before dependency resolution, so pkg@name in the same manifest resolves
Conflicts Same name as a manual entry -> warning with the fix, never overwritten; enforced atomically in the registry lock
Scope guard marketplaces: in any other apm.yml (project or dependency) fails parse with an actionable error

Implementation (HOW)

File Change
src/apm_cli/marketplace/registration.py (new) Canonical owner of "SOURCE -> validated registration": alias rule, #ref split, source parse, manifest probe + fetch. Extracted from apm marketplace add, which now calls it (no behaviour change).
src/apm_cli/marketplace/user_manifest.py (new) Owner of the marketplaces: key: shape validation, user-manifest detection, project/package rejection message.
src/apm_cli/marketplace/reconcile.py (new) Plan + apply: register / update / remove / keep / conflict, dry-run notices, CommandLogger output, exit-1 hook for install.
src/apm_cli/marketplace/registry.py register_manifest_marketplace (refuses to replace a manual entry inside the mutation lock) and unregister_manifest_marketplace (never removes a manual entry).
src/apm_cli/marketplace/models.py, errors.py MarketplaceSource.origin (excluded from equality), ManualMarketplaceConflictError. Unknown persisted origins degrade to manual.
src/apm_cli/models/apm_package.py marketplaces joins the known keys; every parse calls validate_marketplaces_declaration.
src/apm_cli/commands/install.py One statement before package validation (the install() function is at the PLR0915 cap).
src/apm_cli/commands/marketplace/__init__.py add uses the shared helper; old private names kept as compatibility aliases.

Diagram

Legend: what one apm install -g does with each declared entry and each manifest-owned registry entry (new behaviour boxed).

flowchart LR
    A["apm install -g"] --> B["read ~/.apm/apm.yml marketplaces"]
    subgraph NEW["NEW: reconcile before dependency resolution"]
        B --> C{"name in registry?"}
        C -->|"no"| R["probe + fetch, register origin=manifest"]
        C -->|"yes, manual"| W["warn conflict, keep manual"]
        C -->|"yes, manifest, same source/ref"| K["keep, no network"]
        C -->|"yes, manifest, changed"| U["probe + fetch, update"]
        B --> D["manifest-origin entries no longer declared"] --> X["remove"]
    end
    R --> P["resolve and install dependencies"]
    W --> P
    K --> P
    U --> P
    X --> P
Loading

Trade-offs

  • No spec change. OpenAPM §4.6.2 requires a warning only for keys the consumer does not recognise; marketplaces is a recognised, implementation-defined user-scope key, like the existing replace, lifecycle and allowExecutables, and its rejection outside the user manifest is an independent validation failure that the spec keeps ("This warning does not waive independent validation"). No req-XXX changes, so the spec ritual is N/A; no Mode B critical path (src/apm_cli/install/, deps/, ...) is touched.
  • Dependencies that declare marketplaces: fail to parse, rather than being silently ignored. This enforces "packages cannot declare marketplaces" literally; such a package already fails for its own author at project scope.
  • A failed fetch stops apm install -g (exit 1, nothing removed, other entries still reconciled) instead of continuing with a partial registry, so pkg@name never resolves against a stale or missing catalog. Unchanged entries need no network, so offline re-installs still work.
  • Rejected: syncing native harness marketplace settings, a directory protocol, project-scope declarations, policy or trust changes (all out of scope in the record).
  • User root: resolved through fix(scope): make APM_HOME the single root for APM-owned user state #3614's canonical resolver in one place (user_manifest_path()), plus the registry's CONFIG_DIR. Root alignment itself is [BUG] Document and align APM_HOME behavior for global user state #2884 / fix(scope): make APM_HOME the single root for APM-owned user state #3614 and is not reimplemented here.

Benefits

  1. One marketplaces: block replaces N per-machine apm marketplace add commands.
  2. Deleting an entry removes it on the next apm install -g; no manual cleanup.
  3. Manual registrations are provably safe: the ownership check runs inside the registry lock.
  4. Re-running with unchanged entries makes zero network calls.

Issue and approved scope

Issue: Fixes #3006

Human scope-approval comment: #3006 (comment) (apm-scope:v1, Area project, review contact Daniel Meppiel (@danielmeppiel); responsible-human confirmation by Daniel Meppiel (@danielmeppiel) on 2026-11-12, W2-6 Tier 1).

Does this PR complete the issue, or what remains? It completes the approved scope.

Important

Depends on #3614 (#2884, APM_HOME single root). This branch is stacked on danielmeppiel-apm-home-user-root-2884 (merged in, base stays main so CI runs), so the diff shows #3614's commits until it lands; merge #3614 first. The user manifest resolves only through get_manifest_path(InstallScope.USER) (= get_apm_home() / "apm.yml") and registrations stay in CONFIG_DIR; no APM_HOME literal or hardcoded ~/.apm is added. test_user_manifest_follows_apm_home covers a custom APM_HOME.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor (shared registration helper extracted from marketplace add)

Testing

  • Tested locally
  • All existing tests pass (targeted suites below; CI is the authority for the full suite)
  • Added tests for new functionality (if applicable)

Scenario evidence

# Scenario (user promise) Principle(s) Test(s) proving it Type
1 Declare a marketplace in ~/.apm/apm.yml, run apm install -g, then apm search QUERY@NAME finds its plugins DevX, Portability by manifest tests/integration/test_user_manifest_marketplaces_install.py::test_install_g_registers_declared_marketplaces_and_search_finds_them integration
2 Removing an entry removes it; my apm marketplace add registrations are never overwritten or removed, and conflicts are reported Governed by policy, DevX tests/integration/test_user_manifest_marketplaces_install.py::test_install_g_keeps_manual_and_removes_only_manifest_entries
tests/unit/marketplace/test_user_manifest_marketplaces.py::TestReconcile
::TestRegistryOwnership
integration + unit
3 A project apm.yml (or a dependency) with marketplaces: fails with an error naming the user manifest Secure by default, DevX tests/integration/test_user_manifest_marketplaces_install.py::test_project_manifest_with_marketplaces_fails_with_user_manifest_hint
tests/unit/marketplace/test_user_manifest_marketplaces.py::TestScopeRejection
integration + unit
4 Unchanged entries are kept without network; changed entries update; --dry-run writes nothing DevX tests/unit/marketplace/test_user_manifest_marketplaces.py::TestReconcile (kept / updated / dry-run) unit
5 Malformed entries and failed fetches fail apm install -g with the entry and file named DevX tests/unit/marketplace/test_user_manifest_marketplaces.py::TestParseDeclaredMarketplaces, ::TestReconcile::test_fetch_failure_is_reported_and_keeps_registry unit

Integration tests use a real local-directory marketplace (real parser, probe, fetch and registry; no mocks, no network).

Validation output
$ .venv/bin/python -m pytest tests/unit/marketplace/test_user_manifest_marketplaces.py tests/integration/test_user_manifest_marketplaces_install.py -q -p no:xdist   # on the merge commit
36 passed in 12.68s

$ .venv/bin/python -m pytest tests/unit/marketplace/test_marketplace_registry.py tests/unit/marketplace/test_marketplace_models.py tests/unit/marketplace/test_marketplace_commands.py tests/unit/marketplace/test_marketplace_commands_surface.py tests/unit/marketplace/test_marketplace_commands_phase3.py tests/unit/marketplace/test_parser.py tests/unit/marketplace/test_catalog_discovery_reuse.py (+ the two files above) -q -p no:xdist
387 passed in 24.66s

$ .venv/bin/python -m pytest tests/unit/test_manifest_extension_keys.py tests/unit/registry/test_compat_matrix.py tests/unit/deps/test_replacement_model.py tests/unit/test_agent_plugins.py tests/unit/test_install_command.py -q -p no:xdist
305 passed, 2 xfailed in 72.51s

Mutation check (each guard broken, target test re-run; original restored):
KILLED registry: manual-entry check in register_manifest_marketplace
KILLED registry: origin filter in unregister_manifest_marketplace
KILLED reconcile: manual-conflict branch
KILLED reconcile: manifest-owned filter for removals
KILLED reconcile: unchanged -> kept branch
KILLED user_manifest: is_user_manifest scope guard
KILLED models: origin serialisation
KILLED reconcile: install hook
ALL KILLED

CI lint mirror (after merging origin/main): ruff check, ruff format --check, YAML guard,
file-length guard (install.py 2082/2100), relative_to guard, pylint R0801 (10.00/10),
lint-auth-signals.sh, lint-architecture-boundaries.sh -> all exit 0

How to test

  • Create a local marketplace dir with a marketplace.json, then add to ~/.apm/apm.yml: marketplaces: [{name: demo, source: /path/to/dir}]
  • apm install -g -> [+] Registered marketplace 'demo' (N plugins); apm search QUERY@demo finds plugins
  • Run again -> Kept marketplace(s) unchanged: demo
  • apm marketplace add /path/to/dir --name mine, declare name: mine too, run apm install -g -> conflict warning, mine untouched; delete the demo entry -> Removed marketplace 'demo'
  • Put marketplaces: in a project apm.yml, run apm install -> exit 1 naming ~/.apm/apm.yml

Spec conformance (OpenAPM v0.1)

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated
  • N/A -- this PR does not change OpenAPM-observable behaviour. marketplaces is an implementation-defined user-scope key (same class as replace / lifecycle); see Trade-offs.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Centralize APM-owned user metadata under APM_HOME while keeping target deployment paths independent. Document and test the contract across package, config, lifecycle, and architecture boundaries.
…led by apm install -g

Add a 'marketplaces:' list (name + an existing 'apm marketplace add'
source form, optional ref) to the user manifest at the effective APM
user root. 'apm install -g' reconciles it before resolving
dependencies: registers missing entries, updates changed ones and
removes only manifest-origin entries. The registry records each
entry's origin; manual registrations are never overwritten or
removed, and name conflicts are reported. Project and package
manifests that declare 'marketplaces:' fail with an error naming the
user manifest.

The source-to-registration step is extracted from 'marketplace add'
into marketplace.registration so both paths share one parser/probe.

Refs #3006

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 12, 2026 13:08
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Reconciliation can delete existing marketplace registrations after a failed fetch or before rejecting a malformed manifest, risking destructive changes when installation cannot proceed.

4 open issues
What changed in this PR

Adds declarative marketplace registration to the user manifest, reconciled by apm install -g. This implements the narrowed user-configuration approach for #3006 without introducing a marketplace-directory protocol.

Changes:

  • Adds user-only marketplaces: validation and reconciliation before dependency resolution.
  • Tracks registration ownership to preserve manual entries and report name conflicts.
  • Extracts shared registration helpers and adds tests, documentation, and usage guidance.
File Description
tests/​unit/​marketplace/​test_user_manifest_marketplaces.py Tests validation, ownership, reconciliation, and previews.
tests/​integration/​test_user_manifest_marketplaces_install.py Exercises global install and search with local marketplaces.
src/​apm_cli/​models/​apm_package.py Recognizes and validates marketplace declarations.
src/​apm_cli/​marketplace/​user_manifest.py Defines declaration parsing and user-scope restrictions.
src/​apm_cli/​marketplace/​registry.py Adds ownership-protected registration and removal.
src/​apm_cli/​marketplace/​registration.py Extracts source parsing and manifest probing helpers.
src/​apm_cli/​marketplace/​reconcile.py Reconciles declarations and reports results to install.
src/​apm_cli/​marketplace/​models.py Persists registration ownership metadata.
src/​apm_cli/​marketplace/​errors.py Adds a manual-registration conflict error.
src/​apm_cli/​commands/​marketplace/​__init__.py Reuses registration helpers in marketplace add.
src/​apm_cli/​commands/​install.py Invokes reconciliation for user-scope installs.
packages/​apm-guide/​.apm/​skills/​apm-usage/​dependencies.md Explains declaratively registered marketplace references.
packages/​apm-guide/​.apm/​skills/​apm-usage/​commands.md Documents reconciliation and preview behavior.
docs/​src/​content/​docs/​reference/​manifest-schema.md Defines the user-only declaration schema.
docs/​src/​content/​docs/​reference/​cli/​marketplace.md Explains declarations and re-registration after removal.
docs/​src/​content/​docs/​consumer/​installing-from-marketplaces.md Adds configuration examples and ownership guidance.
CHANGELOG.md Records the new marketplace declaration feature.

Review effort: Balanced

Comment thread src/apm_cli/marketplace/reconcile.py Resolved
Comment thread src/apm_cli/marketplace/reconcile.py Resolved
Comment thread src/apm_cli/marketplace/reconcile.py Resolved
Comment thread src/apm_cli/marketplace/registration.py Resolved
…2884)

Finish the APM_HOME single-root contract on top of the cherry-picked
change from @WilliamK112 (closed PR #2967):

- Deployment receipts move under get_apm_home(); the deploy-root
  .apm-local identity stays with the deployment root, like target files.
- The architecture guard also pins the deployment ledger consumer.
- Tests: subprocess isolation (two APM_HOME values, every global
  command family) and parity (unset/empty == ~/.apm); conftest drops a
  developer APM_HOME so the suite stays hermetic.
- Docs: env-var contract lists marketplace registrations, receipts,
  lock, precedence, migration and the out-of-contract paths; apm-guide
  and CHANGELOG updated (entry moved to Unreleased).

Co-authored-by: Ching-Wei Kang <164879897+WilliamK112@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…e-user-root-2884

# Conflicts:
#	docs/src/content/docs/reference/cli/config.md
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…-2884' into danielmeppiel-issue-3006-user-manifest-marketplaces
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve conflicts with #3633 (forwarded apm install options) and #3630 (bootstrap author), keeping the APM_HOME wording.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
apm lock -g and apm update -g chdir into the APM home, so a relative APM_HOME such as 'metadata' resolved to metadata/metadata on later lookups and split the manifest from its lockfile, modules and config. get_apm_home() now expands a leading ~ and anchors a relative value once against the working directory captured at import. Addresses Copilot inline on src/apm_cli/core/scope.py:54 (#2884).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Moving the lifecycle lock under APM_HOME let processes with different homes mutate the same project or harness directories at once, so a failed install's rollback could overwrite another operation's manifest. acquire_lifecycle_lock() now takes the OS-user deployment-root lock in ~/.apm-local/ (named by the deployment owner) before the APM_HOME metadata lock and releases both together; unset APM_HOME keeps the ~/.apm layout. Adds a cross-home contention regression and updates home write-set snapshots and the pre-compile lock probe. Addresses Copilot inline on src/apm_cli/install/locking.py:44 (#2884).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The APM_HOME path contract now states that ~ is expanded, a relative value resolves against the start directory, and ~/.apm-local/.apm-lifecycle.lock serializes commands across APM_HOME values. Mirrors the change in apm-guide and the CHANGELOG entry (#2884).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The cherry-pick from closed PR #2967 added a second Fixed heading with an unrelated GitLab entry inside the released 0.31.0 section. Released sections are not rewritten and the entry is outside #2884's scope. Addresses doc-writer panel finding on PR #3614.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
install.md still named ~/.apm/ as the canonical metadata root, and install/update/deps/lock each restated the APM_HOME contract. They now link the env-var reference instead; the env-var page no longer claims APM_HOME is read once. approve/deny --help, config_loader docstrings and lock -g help name the APM user home. Addresses doc-writer, devx-ux and auth panel findings on PR #3614.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds a symlink-rejection regression for the ~/.apm-local deployment-root lock, loads LOCAL_STATE_DIR lazily so importing locking no longer pulls the deployment ledger, names the shared lock in the busy error, fails closed when a relative APM_HOME cannot be anchored, and documents the receipts containment base. Addresses test-coverage, python-architect, performance, devx-ux and supply-chain panel findings on PR #3614.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The user-scope metadata location (APM_HOME) and lifecycle lock placement are CLI configuration, not OpenAPM manifest, lockfile or resolution behaviour, so no req anchor applies (#2884).

apm-spec-waiver: APM_HOME relocates CLI user-state paths only; no OpenAPM manifest, lockfile or resolution behaviour changes

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
get_apm_home() now errors when a relative APM_HOME cannot be anchored because the start directory is gone; the env-var contract and apm-guide state it. Addresses doc-writer delta panel finding on PR #3614.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	docs/src/content/docs/reference/cli/view.md
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
#3608 added a longer view summary and #3643 added a check that root summaries are not truncated at 80 columns; together they fail tests/unit/test_cli_consistency.py on main and on every PR merge ref. Shorten the summary so the CI merge ref for this PR is green.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This reverts commit bed006f. Lane F owns the main hotfix for the #3608 x #3643 root-help truncation; this PR stays out of that change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…laces

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A malformed user manifest root (empty, list, scalar or invalid YAML)
read as "no marketplaces" and removed every manifest-owned
registration before package parsing rejected the file. A failed fetch
for a renamed or moved entry still removed the stale one. Dry runs and
no-op runs created marketplaces.json and config.json.

Reject a non-mapping root before the registry is read, skip removals
while any declared entry failed, and read the registry without
creating files (read_registered_marketplaces).

Addresses Copilot inline on src/apm_cli/marketplace/reconcile.py:116,
:117 and :140.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`apm marketplace add` still split #ref fragments, resolved ref
precedence and classified direct URLs itself, a second authority for
the grammar the user-manifest reconcile parses with
parse_registration_source. Normalize the CLI-only --branch alias at
the command boundary and route add through the shared parser.

Extend the marketplace-source-admission owner to registration.py and
its guard: add and reconcile must call parse_registration_source, and
the command may not re-split fragments or classify direct URLs.
Behavioural, architecture and mutation tests cover the boundary, plus
regressions for the three reconcile safety fixes.

Addresses Copilot inline on src/apm_cli/marketplace/registration.py:89.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…t-marketplaces

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
main already sits at the 2100-line cap, so the reconcile hook pushed
install.py to 2105 and failed the Lint file-length guardrail. Make the
hook a single call (the user-scope check moves into
reconcile_user_marketplaces_or_exit) and collapse a single-name import.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The user manifest path now reads $APM_HOME/apm.yml (default
~/.apm/apm.yml) everywhere this PR documents it, the source column
notes that entries have no --host equivalent, failure and hand-over
wording matches the code, apm install's reference notes the reconcile
step, the CHANGELOG leads with the benefit, the apm-guide row is
shorter and the "Declare instead of add" callout opens the add
section.

Addresses panel follow-ups from doc-writer, devx-ux-expert and
oss-growth-hacker.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Fetch every new or changed entry first, then apply all registrations
and removals in one locked transaction (apply_manifest_marketplaces)
instead of a lock-read-write cycle per entry. Removals use the [-]
symbol, the kept line pluralizes, a single failure is named directly,
and the manual-conflict warning shows the current source. The source
admission guard now also forbids reconcile.py from re-parsing SOURCE.
The integration module declares its component marker and covers
`apm install -g --dry-run` creating no registry files.

Addresses panel follow-ups from performance-expert, cli-logging-expert,
devx-ux-expert, test-coverage-expert, supply-chain-security-expert and
python-architect.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…anifest-marketplaces

Resolve the #3367 conflict in marketplace add by threading --offline and local apm.yml authoring catalogs through the canonical registration owner (probe_registration offline=, local_source_points_to_file, local hint on MarketplaceManifestNotFoundError) instead of a second inline probe in the command.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A relative local source such as ./catalog resolved against the
directory where `apm install -g` ran, so the same user manifest
registered different paths (or failed) depending on the cwd. Reject
it with an actionable error; absolute and ~/ paths are unchanged. The
shared not-found hint now says "use the path to the apm.yml file" so it
reads right for both `marketplace add` and a manifest `source:`.

Addresses delta panel follow-ups from doc-writer and cli-logging-expert.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…laces

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…laces

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…-marketplaces

Thread #3628's explicit catalog path through the canonical registration owner: probe_registration(..., explicit_path=) fetches only that file, and MarketplaceCatalogPathError carries the --path errors. marketplace add keeps --path admission and the replace warning; the source-admission owner row keeps both admit_marketplace_catalog_path and parse_registration_source.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

APM Review Panel: ship_with_followups

Declarative marketplace registration lands the Brewfile-tap model for APM: declare once in the user manifest, reconcile everywhere with apm install -g.

panel-mode=delta; personas=python-architect,test-coverage-expert,doc-writer,devx-ux-expert,supply-chain-security-expert,performance-expert,cli-logging-expert,oss-growth-hacker,apm-ceo

cc Sergio Sisternes (@sergio-sisternes-epam) -- a fresh advisory pass is ready for your review.

All eight panelists converged on ship after a full pass and a delta pass. The full pass at 4b7dc6f raised findings on docs, UX, output, performance, test tiers and guard coverage. Every in-scope finding was folded (see below). The delta pass at 0d7c2d7 raised two late items: relative local paths resolved against the cwd, and the wording of the shared not-found hint. Both were folded in dc856f4 with an actionable error and updated docs. A final architect and supply-chain check covered the #3628 merge resolution at this head and raised nothing. The source-admission dual guardrail (behavioural tests, static require/forbid guard, architecture tests) held under mutation-break. No panelist dissented.

Strategically this PR delivers the approved #3006 scope (Atlas decision 16: the user-config model of Brewfile tap, .npmrc and Cargo [registries]). Registering a marketplace grants discovery only, never install or trust. The origin model (manifest vs manual, unknown degrades to manual) never overwrites a human registration and gives future policy layers a clean seam.

Material delta since the first pass: the 4 Copilot threads are fixed and resolved. Reconcile now probes first and writes once under the registry lock. Docs follow $APM_HOME. The #3367 local apm.yml authoring catalogs and --offline, and #3628's --path catalog selection, are threaded through the canonical registration.py owner (probe_registration(..., offline=, explicit_path=)). Main, including #3614, is merged at this head. Two items are deferred with scope-boundary notes.

Aligned with: Portable by manifest: registrations are declarative and reproducible across machines. Secure by default: discovery is not trust, relative local paths are rejected, and the source-admission guard now covers reconcile. Governed by policy: manifest vs manual ownership is tracked, and project and dependency manifests cannot declare marketplaces. Multi-harness / multi-host: reconcile runs at the APM_HOME user root, independent of any project. Pragmatic as npm: one command, Brewfile-tap mental model.

Growth signal. The CHANGELOG entry is repostable as a launch note: "declare marketplaces once, like Homebrew taps in a Brewfile".

Panel summary

Persona B R N Takeaway
Python Architect 0 0 0 Both earlier nits resolved or deferred; probe-first + single-transaction pattern and the #3367 merge are clean.
Test Coverage Expert 0 0 0 Earlier findings resolved; every Done-when clause and all 4 Copilot folds have regression traps.
Doc Writer 0 1 0 Earlier findings fixed; relative local source caveat raised in delta, folded in dc856f4.
DevX UX Expert 0 0 0 install.md note, conflict source, and apm-guide row all resolved; no new UX issues.
Supply Chain Security Expert 0 0 0 Forbid-scan gap closed; batched transaction, redaction, and #3367 resolution add no new surface.
Performance Expert 0 0 0 N-saves finding resolved by the batched transaction; no new concern.
CLI Logging Expert 0 1 0 Symbol, plural, and single-failure fixes confirmed; hint wording raised in delta, folded in dc856f4.
OSS Growth Hacker 0 0 0 CHANGELOG benefit lead and add-section callout landed; story is ship-ready.

B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.

Top 2 follow-ups

  1. [Python Architect] Promote client._auto_detect_path_with_payload to a public client function. -- marketplace/registration.py calls it across modules. Deferred: it renames a marketplace/client.py API outside this diff and should follow feat(marketplace): select a catalog in a repo with marketplace add --path #3628, which now also calls a private client helper (_fetch_file) from probe_registration.
  2. [Performance Expert] Probe declared marketplaces in parallel. -- Cold runs scale linearly with the number of new entries. Deferred: this adds concurrent network IO and credential prompts to the install pre-phase; it belongs in a transport-parallelism issue.

Recommendation

All in-scope findings are folded across two passes, mutation-break KILLED every new guard, and no panelist dissented. CI is green on this head; it is ready for the maintainer. Track the two deferred items after merge. Neither changes shipped behaviour or regression safety.

Folded in this run

  • (copilot) Malformed or non-mapping user manifest no longer wipes manifest-owned entries -- resolved in e452cac.
  • (copilot) A failed fetch for a renamed entry no longer removes the stale one -- resolved in e452cac.
  • (copilot) Dry runs and no-op runs create no registry files -- resolved in e452cac.
  • (copilot) marketplace add routes SOURCE through parse_registration_source, with a registered guard -- resolved in ae19e3b.
  • (panel) commands/install.py kept at the 2100-line Lint cap -- resolved in 1071e5f.
  • (panel) Docs: $APM_HOME wording, no --host note, failure and hand-over wording, install.md note, CHANGELOG benefit lead, shorter apm-guide row, callout moved -- resolved in 27d94be.
  • (panel) One locked registry write per reconcile, [-] removal symbol, plural and single-failure wording, conflict shows the current source, forbid-scan covers reconcile, component marker plus dry-run integration test -- resolved in b2e15b0.
  • (panel) Relative local sources rejected in the user manifest; neutral not-found hint -- resolved in dc856f4.
  • (merge) feat(marketplace): load live local catalogs directly from apm.yml #3367 --offline and local apm.yml authoring catalogs threaded through registration.py -- resolved in 0d7c2d7.
  • (merge) feat(marketplace): select a catalog in a repo with marketplace add --path #3628 --path explicit catalog threaded into probe_registration(explicit_path=) with MarketplaceCatalogPathError -- resolved in d9b9d89.

Copilot signals reviewed

  • src/apm_cli/marketplace/reconcile.py:116 -- LEGIT: malformed root read as empty and removed entries (resolved in e452cac).
  • src/apm_cli/marketplace/reconcile.py:140 -- LEGIT: removals ran after a failed fetch (resolved in e452cac).
  • src/apm_cli/marketplace/reconcile.py:117 -- LEGIT: dry run created registry files (resolved in e452cac).
  • src/apm_cli/marketplace/registration.py:89 -- LEGIT: second SOURCE-grammar authority in marketplace add (resolved in ae19e3b).

Deferred (out-of-scope follow-ups)

  • (panel) Public client API for manifest auto-detection -- scope boundary: renames marketplace/client.py APIs outside this diff (_auto_detect_path_with_payload, _fetch_file).
  • (panel) Parallel marketplace probes -- scope boundary: concurrent network/credential prompts in the install pre-phase; transport-parallelism theme.

Regression-trap evidence (mutation-break gate)

Each guard below was removed and its test run; every test FAILED, then the guard was restored. Malformed-root check, skip-removals-on-failure, non-creating registry read, add routing through the shared parser (behavioural + architecture), forbid-scan block, reconcile path in forbid-scan, single registry write, conflict current-source clause, relative-local-source rejection.

Lint contract

Silent at this head merged with main: ruff check and ruff format --check on the CI path set, pylint R0801, YAML-io, file-length (install.py 2100/2100), relative_to, lint-auth-signals.sh and lint-architecture-boundaries.sh.

CI

All checks green at d9b9d89 (23 success, 2 neutral, 1 skipped; no pending), including Lint, Build & Test shards, Lifecycle Smoke, Windows gates, Architecture Ratchet/Scaling, Spec conformance, CodeQL and license/cla. 1 CI fix iteration this run (file-length guard on commands/install.py).

Mergeability status

PR head SHA CEO stance iters folds defers Copilot rounds CI mergeable mergeStateStatus notes
#3610 d9b9d89 ship_with_followups 3 10 2 1 green MERGEABLE BLOCKED awaiting required review

Full per-persona findings (delta pass)

Python Architect

No findings.

Test Coverage Expert

No findings.

Doc Writer

  • [recommended] A relative local path in the user manifest resolves against the directory where apm install -g runs at docs/src/content/docs/reference/manifest-schema.md:440
    ./catalog would register a different path, or fail, depending on the cwd. That breaks "list once, works on every machine". Folded in dc856f4: such sources are rejected with an actionable error, and the docs state that local paths must be absolute or start with ~/.

DevX UX Expert

No findings.

Supply Chain Security Expert

No findings.

Performance Expert

No findings.

CLI Logging Expert

  • [recommended] The shared not-found hint said "pass the apm.yml file", a CLI verb that also reaches the reconcile path, at src/apm_cli/marketplace/registration.py:39
    Folded in dc856f4: it now says "use the path to the apm.yml file itself".

OSS Growth Hacker

No findings.

This panel is advisory. It does not block merge. Re-apply the
panel-review label after addressing feedback to re-run.


Generated by autopilot-pr-review-worker. This comment is AI-generated and may contain errors.

@danielmeppiel
Daniel Meppiel (danielmeppiel) merged commit 94b873c into main Nov 13, 2026
26 checks passed
@danielmeppiel
Daniel Meppiel (danielmeppiel) deleted the danielmeppiel-issue-3006-user-manifest-marketplaces branch November 13, 2026 06:51
Daniel Meppiel (danielmeppiel) added a commit that referenced this pull request Nov 13, 2026
Regenerated from the merge with main fe6f089 (includes #3610) so the
grandfathered set is exactly the current violators: 188 entries across
131 functions. Main fixed two (audit _audit_ci_gate statements,
marketplace add) and added two (install/drift
_materialize_install_path args, one extra branch count on an already
listed function).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Global marketplace directories for enterprise-wide discovery

3 participants