Skip to content

feat(marketplace): select a catalog in a repo with marketplace add --path - #3628

Merged
Daniel Meppiel (danielmeppiel) merged 15 commits into
mainfrom
danielmeppiel-issue-3319-multi-marketplace-catalogs
Nov 12, 2026
Merged

Daniel Meppiel (danielmeppiel) merged 15 commits into
mainfrom
danielmeppiel-issue-3319-multi-marketplace-catalogs

Conversation

@danielmeppiel

@danielmeppiel Daniel Meppiel (danielmeppiel) commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

TL;DR

apm marketplace add gains --path, which registers a catalog stored anywhere inside a git-backed marketplace repository (for example catalogs/frontend/marketplace.json). The path is admitted once — contained, no traversal, must name a .json file — saved in the registration, and reused by browse, search, update and apm install. One repository can now serve one catalog per audience; registrations without --path are unchanged.

Note

Adjacent: open PR #3610 (#3006) moves this same probe block into a new marketplace/registration.py owner. This PR keeps its change to one small branch before auto-discovery so either merge order resolves mechanically; whichever lands second should thread explicit_path into probe_registration. The user-manifest marketplaces: form from #3610 does not accept path yet — out of scope here.

Problem (WHY)

Approach (WHAT)

# Fix
1 --path PATH on marketplace add: validated before any fetch, reads exactly that file (no auto-discovery), actionable error if escaping or missing, nothing registered on failure.
2 Catalog-path admission added to the existing marketplace-source-admission owner (source_identity.py) instead of a new authority; MarketplaceSource.__post_init__ routes persisted paths through it, so hand-edited ../ entries are skipped on load.
3 Static guard extended: command and model must call admit_marketplace_catalog_path.
4 Docs: option, discovery order (first match wins), per-audience producer recipe, and the access-control caveat.

Implementation (HOW)

  • src/apm_cli/marketplace/source_identity.py — admit_marketplace_catalog_path(): rejects empty, control chars, backslashes, absolute/drive paths, non-.json, and ./../empty segments (via validate_path_segments, so percent-encoded %2e%2e is caught); drops one leading ./.
  • src/apm_cli/marketplace/models.py — non-empty path is admitted in __post_init__; empty path (hosted URL / local file) unchanged.
  • src/apm_cli/commands/marketplace/__init__.py — --path option, epilog example, docstring states discovery order. Rejected for hosted marketplace.json URLs and direct local files (they already name the file). Local repository directories are accepted because they use the same path fetch.
  • scripts/architecture_linter/checks/marketplace_bundle_and_audit.py, .apm/architecture/owners/marketplace-plugins.json — guard literals and owner decision text extended.
  • Docs — reference/cli/marketplace.md, consumer/installing-from-marketplaces.md, producer/publish-to-a-marketplace.md (new "One catalog per audience"), packages/apm-guide/.apm/skills/apm-usage/commands.md.

Diagrams

Legend: how marketplace add picks the catalog file; the boxed branch is new, the others are unchanged.

flowchart LR
    A["marketplace add SOURCE"] --> B{"--path given?"}
    B -->|no| C["probe marketplace.json, .github/plugin/, .claude-plugin/"]
    B -->|yes| D["admit_marketplace_catalog_path"]
    D -->|"invalid"| X["error, exit 1, nothing registered"]
    D -->|ok| E["_fetch_file at PATH only"]
    E -->|"missing"| X
    E -->|found| F["registry: path saved"]
    C --> F
    F --> G["browse / search / update / install read saved path"]
    subgraph NEW
      D
      E
    end
Loading

Trade-offs

  • Producer recipe uses remote subdir entries, not local ./ sources. Verified with apm pack: a subfolder apm.yml reads ./ entries relative to the subfolder at pack time, while consumers resolve relative sources from the repository root (Claude Code does the same for git sources). Changing install-time resolution was out of scope; the docs state the rule instead.
  • .json suffix required. Catches the common mistake of passing a directory (catalogs/frontend) with a direct hint; Claude Code's field is also a file path.
  • Reload validation skips bad persisted entries (existing _load behaviour for invalid URLs) rather than failing the whole registry.
  • No producer-side multi-catalog schema or package filters — record out of scope; composition of existing per-folder apm.yml covers it.
  • Not access control — documented in three places; private per-audience access still needs separate repositories ([FEATURE] Authenticate private hosted HTTPS marketplace catalogs without vendor-specific coupling #3300 tracks private hosted catalogs).

Benefits

  1. One repository can expose N catalogs, registered under N names, each listing only its own packages (proven end to end).
  2. Escaping (../, absolute, encoded) and missing paths fail before any registry write.
  3. Behaviour without --path is byte-identical: same probe order, same error text, same stored shape.
  4. Persisted paths now have a single admission owner with a static guard.

Issue and approved scope

Issue: Fixes #3319

Human scope-approval comment: #3319 (comment) (apm-scope:v1, approved by Daniel Meppiel (@danielmeppiel); review contact Daniel Meppiel (@danielmeppiel)). Eligibility re-checked from a trusted origin/main copy: state: record-present, authorizes_implementation: false; fresh responsible-human confirmation relayed in the delivery session.

Completes the issue: every Done-when item is covered — installed-CLI test registers two catalogs from one repository under different names and installs from each; path survives marketplace update (all and named) and install; escaping/missing paths fail before registration; no---path behaviour unchanged; docs and apm-usage/commands.md describe option, recipe, discovery order, and access caveat. The issue's Q2 (both .github/plugin/ and .claude-plugin/ present) is answered in the docs: the first in probe order wins.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

Local preflight skipped by maintainer instruction (2026-11-12); CI is the gate. Local runs were targeted only; the full unit suite was not run locally.

Scenario evidence

# Scenario (user promise) Principle(s) Test(s) proving it Type
1 Register two catalogs from one repo by path, install from each, refresh keeps the path DevX, Vendor-neutral tests/integration/test_marketplace_catalog_path_lifecycle.py::test_two_catalogs_from_one_repository_register_install_and_refresh e2e (installed CLI, generic git host)
2 A catalog only exposes its own packages DevX same test (browse + backend-skill@acme-frontend exits 1); test_marketplace_catalog_path.py::TestMarketplaceAddPath::test_refresh_reads_the_recorded_path e2e / unit
3 ../, absolute, encoded, backslash, directory paths rejected before fetch or write Secure by default TestAdmitMarketplaceCatalogPath::test_rejects_escaping_or_malformed_paths, TestMarketplaceAddPath::test_invalid_path_fails_before_any_fetch_or_registration unit
4 Hand-edited escaping registry entry is not loaded Secure by default TestPersistedRegistrationAdmission::test_registry_skips_hand_edited_escaping_entry unit
5 Missing path → actionable error, nothing registered DevX test_missing_path_fails_with_actionable_error + lifecycle rejection sequence unit / e2e
6 No --path → unchanged discovery and error DevX test_without_path_subfolder_catalog_is_not_discovered, existing test_catalog_discovery_reuse.py unit
7 Admission stays single-owner Governed by policy tests/integration/test_architecture_marketplace_source_admission.py::test_marketplace_source_admission_guard_rejects_unrouted_catalog_path integration
Validation output
$ pytest tests/unit/marketplace/test_marketplace_catalog_path.py -q -p no:xdist
32 passed in 10.50s

$ APM_E2E_TESTS=1 APM_BINARY_PATH=.venv/bin/apm pytest tests/integration/test_marketplace_catalog_path_lifecycle.py -q -p no:xdist
1 passed in 222.95s

$ pytest -q -p no:xdist test_catalog_discovery_reuse.py test_client_*.py test_marketplace_client.py \
    test_marketplace_commands*.py test_marketplace_models.py test_marketplace_registry.py test_registry_integration.py
408 passed in 5.94s

$ pytest tests/integration/test_architecture_marketplace_source_admission.py -q -p no:xdist
3 passed

# Mutation-break (unit file): each mutant must fail tests
M1 CLI ignores --path (falls back to auto-discovery)   -> 4 failed
M2 model skips path admission on reload                  -> 2 failed
M3 owner skips traversal/segment validation              -> 8 failed

# CI lint mirror after merging origin/main
ruff check / ruff format --check (CI paths)              -> ok
yaml-io guard, 2100-line guard, relative_to guard        -> ok
pylint R0801                                             -> 10.00/10
scripts/lint-auth-signals.sh                             -> clean
scripts/lint-architecture-boundaries.sh                  -> exit 0

# Recipe check: subfolder apm.yml with remote subdir entry, `apm pack`
-> emits {"source": "git-subdir", "url": "microsoft/apm", "path": "packages/apm-guide", "ref": "v0.35.1", "sha": ...}

How to test

  • In a repo with catalogs/a/marketplace.json and catalogs/b/marketplace.json: apm marketplace add <repo> --path catalogs/a/marketplace.json --name a (repeat for b); apm marketplace list shows both paths.
  • apm marketplace browse a lists only catalog A's plugins; apm install <plugin>@a works.
  • apm marketplace add <repo> --path ../x.json and --path catalogs/missing/marketplace.json both exit 1 with no registry change.
  • apm marketplace add <repo> without --path behaves as before.

Spec conformance (OpenAPM v0.1)

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
    (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
    row).
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
    updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under
    tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated via
    uv run --extra dev python -m tests.spec_conformance.gen_statement
    and committed.
  • N/A -- this PR does not change OpenAPM-observable behaviour. (Consumer marketplace registration is not specified by OpenAPM v0.1; no src/apm_cli/marketplace/ path is in critical_paths.txt.)

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Register several catalogs from one repository: --path selects a
repository-relative marketplace JSON (contained, no traversal), skips
auto-discovery, and is recorded in the registration so browse, search,
update and install read the same catalog. Mirrors Claude Code's path
field on github/git marketplace sources. Admission is owned by
marketplace/source_identity.py and also guards persisted reloads.

Docs: CLI reference, consumer guide, per-audience producer recipe,
discovery order, and the access-control caveat.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 12, 2026 14:22
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unescaped path characters can alter GitHub/GHES requests and override the selected ref, while a non-string persisted path can abort loading the entire registry.

2 open issues
What changed in this PR

Adds apm marketplace add --path so one repository can expose separate catalogs for different audiences, using the existing marketplace registration and fetch flows.

Changes:

  • Adds explicit catalog selection while preserving default discovery when --path is omitted.
  • Validates and persists catalog paths for reuse by browsing, searching, refreshing, and installation.
  • Adds regression coverage and documents per-audience catalogs, discovery order, and access-control limitations.
File Description
tests/​unit/​marketplace/​test_marketplace_catalog_path.py Tests path admission, persistence, CLI errors, and refresh.
tests/​integration/​test_marketplace_catalog_path_lifecycle.py Exercises two catalogs through registration, refresh, and installation.
tests/​integration/​test_architecture_marketplace_source_admission.py Tests enforcement of shared path admission.
src/​apm_cli/​marketplace/​source_identity.py Adds catalog-path validation and normalization.
src/​apm_cli/​marketplace/​models.py Validates persisted catalog paths during construction.
src/​apm_cli/​commands/​marketplace/​__init__.py Adds --path, explicit fetching, and diagnostics.
scripts/​architecture_linter/​checks/​marketplace_bundle_and_audit.py Requires command and model routing through path admission.
packages/​apm-guide/​.apm/​skills/​apm-usage/​commands.md Documents the option in CLI usage guidance.
docs/​src/​content/​docs/​reference/​cli/​marketplace.md Explains path selection, persistence, and discovery order.
docs/​src/​content/​docs/​producer/​publish-to-a-marketplace.md Adds a per-audience catalog publishing recipe.
docs/​src/​content/​docs/​consumer/​installing-from-marketplaces.md Shows registration of multiple catalogs from one repository.
CHANGELOG.md Records the new catalog-selection feature.
.apm/​architecture/​owners/​marketplace-plugins.json Extends source-admission ownership to catalog paths.

Review effort: Balanced

@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

APM Review Panel: ship_with_followups

Marketplace --path flag adds single-catalog-fetch precision to multi-audience Git repositories with thorough admission and test coverage; panel converged on ship-with-followups.

panel-mode=full; personas=python-architect,devx-ux-expert,supply-chain-security-expert,doc-writer,test-coverage-expert,performance-expert,apm-ceo

cc Sergio Sisternes (@sergio-sisternes-epam) -- a fresh advisory pass is ready for your review.

The panel converged cleanly. Six personas reviewed the full surface -- architecture, security, tests, docs, UX, and performance -- and none found a blocking defect. test-coverage-expert confirmed every Done-when scenario maps to a test at unit or integration tier, and performance-expert noted that --path turns a 3-RTT auto-discovery probe into a single targeted fetch. The two Copilot inline items (non-string ValueError and quote(file_path, safe='/') in _github_contents_url) are already queued for fold by the merge worker and are the right fixes; no panelist challenged their sufficiency.

python-architect and supply-chain-security-expert independently converged on the same defense-in-depth ask: reject URL metacharacters '?' and '#' at the admission gate. Single-owner discipline means the admission gate -- not each downstream URL builder -- owns the full path-validity contract. These characters have zero legitimate use in a repository-relative JSON file path, and any future host builder that interpolates without encoding would reintroduce the injection vector.

devx-ux-expert raised a real UX gap: two --path adds to the same repo with colliding derived names silently overwrite the first registration. The doc-writer's duplication finding is minor hygiene -- the consumer guide restates discovery rules that live canonically in the CLI reference -- but is worth a quick trim to keep the one-source-of-truth rule intact.

Dissent. python-architect rated '?'/'#' rejection as recommended; supply-chain-security-expert rated it nit (because the planned quote() fold already neutralizes the GitHub builder). I side with python-architect: the admission gate is the single owner and should reject characters that are never legitimate, regardless of downstream encoding.

Aligned with: Secure by default: Admission gate rejects traversal, backslashes, control chars, and percent-encoded bypasses before any registration; planned folds add non-string rejection and URL encoding. Governed by policy: Static architecture linter enforces single-owner discipline: command and model must call admit_marketplace_catalog_path. Portable by manifest: Path persists in the marketplace registration and mirrors Claude Code's native path field. Pragmatic as npm: --path follows standard CLI convention; the discovery-order default means zero config for the common case.

Panel summary

Persona B R N Takeaway
Python Architect 0 1 1 Catalog-path admission correctly extends single-owner gate with dual guardrail; harden gate to also reject URL-significant chars.
DevX UX Expert 0 1 1 Solid flag design, help, errors; recommend guarding silent overwrite when --path reuses the same derived name.
Supply Chain Security Expert 0 0 1 Admission is thorough; two planned folds (non-string ValueError, quote in _github_contents_url) are necessary and sufficient. One defense-in-depth nit.
Doc Writer 0 1 2 Docs match the code and cover the full approved scope (option, recipe, discovery order, access caveat, apm-usage guide), all ASCII. Consumer guide repeats reference text; trim it.
Test Coverage Expert 0 0 0 All five Done-when scenarios map to tests at unit+lifecycle tiers; admission has 13 rejection vectors; path persists through refresh/update/install; ship.
Performance Expert 0 0 1 --path eliminates up to 2 wasted discovery probes (3 RTT -> 1 RTT on add); admission guard in post_init adds ~2us/source on registry load -- noise. Ship.

B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.

Top 5 follow-ups

  1. [Python Architect] Reject '?' and '#' in admit_marketplace_catalog_path as defense-in-depth alongside the Copilot quote() fold. -- Both python-architect and supply-chain-security-expert converged on this. The admission gate should own the full path-validity contract for all downstream consumers.
  2. [DevX UX Expert] Warn (do not silently overwrite) when a --path registration resolves to a name that already exists with a different url or path. -- The primary --path use case is multiple catalogs from one repo; a silent replace is found only on a failed install. A one-line warning with a --name hint removes the footgun.
  3. [Doc Writer] Trim consumer guide duplication: replace restated discovery-order and path-rules text with a link to the CLI reference. -- One source of truth prevents drift on future changes.
  4. [Doc Writer] Error message for missing catalog should name the user's --path value, not a generic 'No marketplace.json found'. -- --path accepts any .json file name; the current message is misleading for non-default names.
  5. [DevX UX Expert] Move the Claude Code interop note out of the --path flag table into the prose section below. -- Tool-specific jargon in the flag table confuses non-Claude users.

Architecture

classDiagram
    direction LR
    class MarketplaceSource {
      <<ValueObject>>
      +name str
      +url str
      +ref str
      +path str
      +__post_init__()
    }
    class SourceAdmission {
      <<CanonicalOwner>>
      +parse_marketplace_source(src, host_flag) Identity
      +admit_marketplace_catalog_path(path) str
    }
    class MarketplaceClient {
      <<IOBoundary>>
      +_fetch_file(source, file_path) dict
      +_auto_detect_path_with_payload(source) tuple
    }
    class MarketplaceRegistry {
      <<IOBoundary>>
      +add_marketplace(source) None
      +_load() list
    }
    class ArchLinter {
      <<StaticGuard>>
      +_check_marketplace_source_admission()
    }
    class PathSecurity {
      <<Pure>>
      +validate_path_segments(path_str) None
    }
    MarketplaceSource ..> SourceAdmission : __post_init__ delegates
    SourceAdmission ..> PathSecurity : catalog path uses
    MarketplaceClient ..> MarketplaceSource : reads
    MarketplaceRegistry *-- MarketplaceSource : persists
    ArchLinter ..> SourceAdmission : guards single-owner
    class MarketplaceSource:::touched
    class SourceAdmission:::touched
    class ArchLinter:::touched
    classDef touched fill:#fff3b0,stroke:#d47600
Loading
flowchart TD
    CLI["apm marketplace add SOURCE --path P"] --> PARSE["parse_marketplace_source"]
    PARSE --> PATH_Q{"--path given?"}
    PATH_Q -- no --> AUTO["_auto_detect_path_with_payload: root, .github/plugin/, .claude-plugin/"]
    PATH_Q -- yes --> DIRECT{"hosted URL or local file SOURCE?"}
    DIRECT -- yes --> ERR1["exit 1: --path not supported"]
    DIRECT -- no --> ADMIT["admit_marketplace_catalog_path"]
    ADMIT -- ValueError --> ERR2["exit 1: Invalid --path with hint"]
    ADMIT -- ok --> FETCH["_fetch_file at PATH only"]
    FETCH -- None --> ERR3["exit 1: no catalog at path"]
    FETCH -- found --> BUILD["MarketplaceSource path re-admitted in __post_init__"]
    AUTO --> BUILD
    BUILD --> SAVE["registry.add_marketplace"]
    SAVE --> USE["browse / search / update / install read saved path"]
Loading

Recommendation

Ship after the merge worker folds the two planned Copilot items (non-string ValueError, quote() in _github_contents_url) and the converged '?'/'#' admission rejection. The silent-overwrite warning (devx-ux-expert) and docs trim (doc-writer) are valuable but do not need to gate this PR. Test coverage is comprehensive, architecture is clean, and the feature closes the accepted issue scope.


Full per-persona findings

Python Architect

  • [recommended] admit_marketplace_catalog_path should reject ? and # at the gate as defense-in-depth at src/apm_cli/marketplace/source_identity.py:109
    A path like catalogs/marketplace.json?ref=evil#frag.json passes all current admission checks. Downstream fetchers build URLs differently: _github_contents_url interpolates file_path unencoded (Copilot fold adds quote -- correct), GitLab already quotes. Per single-owner discipline the admission gate should own the full path-validity contract for ALL consumers rather than relying on each to quote. These characters are never legitimate in a repository-relative catalog path.
    Suggested: After the backslash check, raise ValueError when '?' or '#' is present; add a parametrize case to test_rejects_escaping_or_malformed_paths.
  • [nit] Design patterns: well-applied ValueObject and single canonical owner; no further abstraction needed
    MarketplaceSource is a frozen value object enforcing invariants via the canonical admission owner; admit_marketplace_catalog_path is the sole authority for catalog path validity, called from CLI input and persisted reload, guarded by static linter plus behavioral test. Current shape is the simplest correct design at this scope.

DevX UX Expert

  • [recommended] Silent overwrite when --path is used twice on the same repo without --name and manifest names collide. at src/apm_cli/commands/marketplace/__init__.py:580
    add_marketplace replaces any existing entry with the same case-insensitive name. Without --name the name is derived from manifest.name or the repo URL tail. The primary --path use case is several catalogs from one repo; if two catalogs share a manifest name (or lack one), the second add silently replaces the first. The user learns only when an install from the first catalog fails.
    Suggested: When an entry with the resolved name exists and the new registration has a different url or path, warn: Replacing existing marketplace (was: ). Use --name to keep both registrations.
    Proof (missing at): tests/unit/marketplace/test_marketplace_catalog_path.py -- proves: Two --path adds to the same repo with colliding derived names warn instead of silently overwriting the first registration.
  • [nit] Claude Code reference in the --path flag table is tool-specific jargon in user-facing reference docs. at docs/src/content/docs/reference/cli/marketplace.md:123
    Useful as a design note but confusing for non-Claude users in the flag table; CLI help already omits it.
    Suggested: Move the interop note into the prose below the table.

Supply Chain Security Expert

  • [nit] Reject URL metacharacters ? and # at the admission layer for defense-in-depth at src/apm_cli/marketplace/source_identity.py:103
    admit_marketplace_catalog_path admits paths containing ? and #. A path like marketplace.json?ref=evil#x.json passes all current checks. The planned quote(file_path, safe='/') fold neutralises these for the GitHub URL builder, and GitLab/ADO builders already encode. However, ? and # have no legitimate use in a repository-relative JSON file path, and any future URL builder that interpolates file_path without encoding would reintroduce the injection.
    Suggested: After the backslash check, reject '?' and '#' with ValueError.

Doc Writer

  • [recommended] Consumer guide restates discovery order, path rules and the access caveat that the CLI reference already defines at docs/src/content/docs/consumer/installing-from-marketplaces.md:250
    The consumer paragraph repeats the discovery list, the fail-before-registering rule and the access caveat almost word for word from reference/cli/marketplace.md. Repo doc rules say to state a fact once and link to it everywhere else; each extra copy can drift.
    Suggested: Keep the two-command example, the saved-and-reused sentence and the access caveat; replace the discovery-order and path-rules sentences with one link to the marketplace add --path reference.
  • [nit] Missing-file error says 'No marketplace.json found' even when --path names a differently named file at src/apm_cli/commands/marketplace/__init__.py:501
    --path accepts any .json file name (e.g. catalogs/frontend/catalog.json); the error should name what the user passed.
    Suggested: No marketplace catalog found at '' ...
  • [nit] Reference discovery paragraph is wordy and repeats itself at docs/src/content/docs/reference/cli/marketplace.md:129
    Two stacked qualifiers and a repeated 'first one wins' sentence; repo voice rules call for short, direct text.
    Suggested: Without --path, APM checks marketplace.json, .github/plugin/marketplace.json, then .claude-plugin/marketplace.json, and registers the first one found. Drop the 'If several exist' sentence.

Test Coverage Expert

No findings.

Performance Expert

  • [nit] admit_marketplace_catalog_path rescans self.path on every MarketplaceSource construction including registry loads at src/apm_cli/marketplace/models.py:205
    Pure CPU string work, about 1-2us per source, under 0.5% of registry load wall-time; the in-process _registry_cache bounds it to one pass per command. Acceptable for the correctness gain of rejecting hand-edited traversal on reload.
    Suggested: No action needed.

This panel is advisory. It does not block merge. Re-apply the
panel-review label after addressing feedback to re-run.


Generated by autopilot-pr-review-worker. This comment is AI-generated and may contain errors.

A persisted non-string path raised AttributeError and aborted the whole
registry load, and a path carrying '?' or '#' could change the file or ref
the GitHub Contents API request names. Admission now rejects non-string
paths and URL query/fragment characters with ValueError (so registry load
skips the entry), and _github_contents_url percent-encodes the file path.

Addresses Copilot inline on src/apm_cli/marketplace/source_identity.py:103
and :117, and panel follow-up FU-1 (python-architect, supply-chain).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Registering several catalogs from one repository is the main --path use
case; a second add under a name already bound to a different catalog
silently replaced it. Warn with the replaced source and path and suggest
--name. Registrations without --path keep their silent replace. The
missing-file error now names the catalog the user passed rather than a
fixed marketplace.json. Regression tests cover both, plus the admission
and URL-encoding fixes.

Addresses panel follow-ups FU-2 (devx-ux-expert) and FU-4 (doc-writer).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The consumer guide restated the discovery order and path rules from the
CLI reference; link to the reference instead. Tighten the reference
discovery paragraph, move the Claude Code note out of the flag table,
and document the '?'/'#' rule and the replace warning.

Addresses panel follow-ups FU-3 (doc-writer) and FU-5 (devx-ux-expert).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
test_marketplace_source_admission_has_single_owner checked only the URL
parser, so unrouting admit_marketplace_catalog_path from the model left it
green. Assert the single owner definition and both consumer calls so the
architecture test and the static guard fail together on that mutation.

Canonical-owner gate: dual-guardrail architecture assertion for the
marketplace-source-admission owner extension.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Delta panel follow-ups. _github_raw_contents_url (plugin apm.yml fetch
for marketplace audit) still interpolated file_path raw, unlike the fixed
catalog builder; encode it the same way. Pin the replace-warning
cross-case (a bare re-add over a --path registration warns, because it
drops a chosen catalog) and extend the real-CLI rejection sequence with a
'#' path so the URL-metacharacter guard is proven end to end.

Addresses delta follow-ups from supply-chain-security-expert,
devx-ux-expert and test-coverage-expert.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

APM Review Panel: ship_now

All first-advisory and delta-panel findings folded with mutation-break-proven regression tests; marketplace --path delivers single-catalog-fetch precision with comprehensive admission, encoding, and coverage.

panel-mode=delta; personas=python-architect,devx-ux-expert,supply-chain-security-expert,doc-writer,test-coverage-expert,performance-expert,apm-ceo

cc Sergio Sisternes (@sergio-sisternes-epam) -- a fresh advisory pass is ready for your review.

The delta panel reviewed three new findings at ec13bf6 and the merge worker folded all three before the final head 9e5ec14. Combined with the five FU items folded in earlier commits, every finding from both advisory rounds is now addressed in code with regression tests proven by mutation break.

supply-chain-security-expert identified that _github_raw_contents_url still interpolated file_path unencoded, inconsistent with the fixed sibling. Folded: the same quote(file_path, safe='/') pattern is applied, and test_raw_contents_path_is_encoded_and_ref_preserved pins it. devx-ux-expert surfaced an untested cross-case: a bare re-add over a --path registration should warn before dropping the chosen catalog. Folded: test_bare_re_add_over_a_path_registration_warns asserts the warning fires for path-bearing entries while plain default-path re-adds stay silent, preserving existing behavior. test-coverage-expert noted the ?/# admission guard lacked an end-to-end lifecycle rejection. Folded: the lifecycle rejection sequence now includes --path catalogs/a#b.json with returncode 1 and the Invalid --path message.

python-architect, doc-writer, and performance-expert returned clean on the delta diff. Docs match the final behavior across the consumer guide, CLI reference, and ?/# rule. Architecture linter and has_single_owner assertions pass. Lint mirror is clean. The earlier CI failure was a main-branch regression from #3608/#3643, fixed by the now-merged hotfix #3651 -- not a defect in this PR.

Aligned with: Secure by default: Admission gate rejects traversal, backslashes, control chars, percent-encoded bypasses, and URL metacharacters ? and #; both URL builders percent-encode file_path; each guard has a mutation-break-proven regression test. Governed by policy: Static architecture linter enforces single-owner discipline: command and model must call admit_marketplace_catalog_path; architecture tests assert the definition site and both consumer call sites. Portable by manifest: Path persists in the marketplace registration and mirrors Claude Code's native path field. Pragmatic as npm: --path follows standard CLI convention with a discovery-order default for the zero-config common case; the replace warning protects --path users without penalizing default registrations.

Folded in this run

  • (copilot) Reject non-string catalog path with ValueError so registry load skips the entry -- resolved in 104d625.
  • (copilot) Percent-encode file_path in _github_contents_url -- resolved in 104d625.
  • (panel) Reject '?' and '#' in admit_marketplace_catalog_path (python-architect, supply-chain) -- resolved in 104d625.
  • (panel) Warn before --path replaces a different catalog under the same name (devx-ux) -- resolved in 9de9b77.
  • (panel) Missing-file error names the --path catalog, not marketplace.json (doc-writer) -- resolved in 9de9b77.
  • (panel) Consumer guide links to the CLI reference instead of restating rules (doc-writer) -- resolved in a2f71d1.
  • (panel) Move Claude Code interop note out of the --path flag table (devx-ux) -- resolved in a2f71d1.
  • (panel) has_single_owner asserts admit_marketplace_catalog_path definition and both call sites (canonical-owner gate) -- resolved in fae8a41.
  • (panel) Percent-encode file_path in _github_raw_contents_url (supply-chain delta) -- resolved in 9e5ec14.
  • (panel) Pin bare re-add over a --path registration warning cross-case (devx-ux delta) -- resolved in 9e5ec14.
  • (panel) Real-CLI lifecycle rejection of --path catalogs/a#b.json (test-coverage delta) -- resolved in 9e5ec14.

Copilot signals reviewed

  • src/apm_cli/marketplace/source_identity.py:103 -- LEGIT: Non-string persisted path crashed registry load instead of being skipped. (resolved in 104d625; thread replied and resolved).
  • src/apm_cli/marketplace/source_identity.py:117 -- LEGIT: Unencoded file_path in the Contents API URL let '?'/'#' override the file and ref. (resolved in 104d625; thread replied and resolved).

Regression-trap evidence (mutation-break gate)

  • tests/unit/marketplace/test_marketplace_catalog_path.py::TestAdmitMarketplaceCatalogPath::test_rejects_escaping_or_malformed_paths -- deleted '?'/'#' rejection in admit_marketplace_catalog_path; test FAILED as expected; guard restored.
  • tests/unit/marketplace/test_marketplace_catalog_path.py::TestPersistedRegistrationAdmission::test_registry_skips_non_string_path_entry -- deleted non-string ValueError in admit_marketplace_catalog_path; test FAILED as expected; guard restored.
  • tests/unit/marketplace/test_marketplace_catalog_path.py::TestGithubContentsUrl::test_catalog_path_is_encoded_and_ref_preserved -- deleted quote(file_path, safe='/') in _github_contents_url; test FAILED as expected; guard restored.
  • tests/unit/marketplace/test_marketplace_catalog_path.py::TestMarketplaceAddPath::test_replacing_a_different_catalog_under_the_same_name_warns -- deleted _warn_if_replacing_catalog call in marketplace add; test FAILED as expected; guard restored.
  • tests/unit/marketplace/test_marketplace_catalog_path.py::TestGithubContentsUrl::test_raw_contents_path_is_encoded_and_ref_preserved -- deleted quote(file_path, safe='/') in _github_raw_contents_url; test FAILED as expected; guard restored.
  • tests/unit/marketplace/test_marketplace_catalog_path.py::TestMarketplaceAddPath::test_bare_re_add_over_a_path_registration_warns -- deleted existing.path != source.path branch of the replace warning; test FAILED as expected; guard restored.
  • tests/integration/test_architecture_marketplace_source_admission.py::test_marketplace_source_admission_has_single_owner -- deleted admit_marketplace_catalog_path(self.path) in MarketplaceSource.__post_init__; test FAILED as expected; guard restored.

Lint contract

uvx ruff@0.15.12 check + format --check on src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ (All checks passed; 2166 files formatted); pylint R0801 10.00/10; lint-auth-signals clean; lint-architecture-boundaries.sh exit 0 at 9e5ec14; ruff pair on changed files clean after final main merge; CI Lint job green at final head.

CI

Observed green at exact head 021f236 via gh pr checks: 23 pass, 3 skipping, 0 pending/failing. Runs: Deploy Docs 43304389124 success; NOTICE Drift Check 43304389070 success; CodeQL 43304389007 success; Spec conformance 43304388980 success; Merge Gate 43304389066 success; CI 43304389101 success; PR eligibility evidence 43304383414 success. license/cla=SUCCESS. Only prior red was fae8a41 Build & Test Shard 2 tests/unit/test_cli_consistency.py::test_root_help_command_summaries_are_not_truncated, a main regression fixed by hotfix #3651 (merged in). mergeStateStatus BLOCKED = pending required review only. (after 1 CI fix iteration(s)).

Mergeability status

PR head SHA CEO stance iters folds defers Copilot rounds CI mergeable mergeStateStatus notes
#3628 021f236 ship_now 2 11 0 2 green MERGEABLE BLOCKED pending required review

Convergence

2 outer iteration(s); 2 Copilot round(s). Final panel stance: ship_now. Scope: delivers the full apm-scope:v1 record on #3319 (PR body carries Fixes #3319). Ready for maintainer review.

Panel summary

Persona B R N Takeaway
Python Architect 0 0 0 FU-1 correctly hardened: ?/# rejection, non-string guard and URL encoding route through the admit_marketplace_catalog_path single owner; _warn_if_replacing_catalog is command-layer UX, not a second authority.
DevX UX Expert 0 1 0 FU-2 and FU-5 closed; replace-warning cross-case (existing custom path, new bare add) needs a test to document intent.
Supply Chain Security Expert 0 1 0 Admission and encoding fixes are correct; no regression on existing fetch paths. Sibling _github_raw_contents_url still interpolated file_path unencoded.
Doc Writer 0 0 0 FU-3 and FU-4 are closed. Docs match the new ?/# rejection, the replace warning and the new error text, and the consumer-guide link resolves.
Test Coverage Expert 0 1 0 All delta behaviors have regression traps; the ?/# guard should also be proven through the real CLI rejection sequence.
Performance Expert 0 0 0 Delta adds microseconds of CPU (quote, isinstance, cached registry scan) to a network-bound CLI command; no hot-path regression.

B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.

Recommendation

Every finding from both advisory rounds -- five original follow-ups plus three delta items -- has been folded with mutation-break-proven regression tests. Docs, lint, and architecture assertions are current at the final head. No remaining follow-ups. Ready for maintainer merge.


Full per-persona findings

Python Architect

No findings.

DevX UX Expert

  • [recommended] Replace warning also fires on a bare re-add over a --path registration; untested at src/apm_cli/commands/marketplace/__init__.py:679
    The condition explicit_path or existing.path != source.path warns when a no-path add replaces an entry that had a custom path. Arguably better UX (warns before losing a chosen catalog), and such entries only exist because of --path, but only the both-default case is tested.
    Suggested: Add a cross-case test asserting the warning fires.
    Proof (missing at): tests/unit/marketplace/test_marketplace_catalog_path.py -- proves: A bare re-add over a --path registration warns before dropping the chosen catalog.

Supply Chain Security Expert

  • [recommended] _github_raw_contents_url (audit path) does not percent-encode file_path, unlike the now-fixed _github_contents_url at src/apm_cli/marketplace/client.py:1067
    The sibling builder used by fetch_raw for marketplace audit plugin apm.yml fetches interpolated file_path raw. Its input is traversal-checked but not '?'/'#'-checked, so a malicious catalog could inject query parameters into the audit Contents API request. Pre-existing and low incremental risk, but inconsistent with the fixed sibling.
    Suggested: encoded_path = quote(file_path, safe='/') as in _github_contents_url.

Doc Writer

No findings.

Test Coverage Expert

  • [recommended] ?/# path guard has a unit regression trap but no real-binary test at tests/integration/test_marketplace_catalog_path_lifecycle.py
    '?' or '#' in a catalog path could alter the Contents API request, a secure-by-default surface with an integration-with-fixtures floor. The lifecycle test already has a rejection run_sequence; one more tuple proves it end to end.
    Suggested: Add ('marketplace','add',_REMOTE,'--ref','main','--path','catalogs/a#b.json') with returncode 1 and assert the Invalid --path message.
    Proof (missing at): tests/integration/test_marketplace_catalog_path_lifecycle.py -- proves: A catalog path containing ? or # is rejected by the real CLI before any request.

Performance Expert

No findings.

This panel is advisory. It does not block merge. Re-apply the
panel-review label after addressing feedback to re-run.


Generated by autopilot-pr-review-worker. This comment is AI-generated and may contain errors.

…alogs

Resolve conflicts with #3367 (live local apm.yml catalogs, --offline):
- marketplace add: keep both --path (catalog_path) and --offline params
  and both docstring paragraphs.
- reference/cli/marketplace.md synopsis lists both --path and --offline.
- apm-usage commands.md: fold the two auto-merged "marketplace add" rows
  into one row carrying both the --path and the local apm.yml/--offline
  guidance.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…alogs

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…alogs

Move the #3628 CHANGELOG bullet off the Added section edges so later
landings that append to the section do not re-conflict.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@danielmeppiel

Copy link
Copy Markdown
Collaborator Author

Merged current main into this branch (merge-of-main fallback; branch history already contains main merges): 021f236 -> 834157b (merge base a2c63ef).

Conflicting paths resolved (faithful merge of both intents with #3367 live local apm.yml catalogs / --offline):

  • src/apm_cli/commands/marketplace/__init__.py -- add() keeps both catalog_path (--path) and offline (--offline); the docstring keeps both the local apm.yml SOURCE wording and the --path discovery paragraph.
  • docs/src/content/docs/reference/cli/marketplace.md -- synopsis lists both [--path P] and [--offline].

Also folded in the merge commit:

  • packages/apm-guide/.apm/skills/apm-usage/commands.md auto-merged into two apm marketplace add rows; folded into one row carrying both the --path and the local apm.yml / --offline guidance.
  • Moved this PR's CHANGELOG bullet off the edges of the Added section (now second-to-last in the first Added block) so later landings that append there do not re-conflict. No main entries dropped.

No regression-trap test changed in the merge, so the mutation-break gate was not re-run.

Lint contract at 834157b: ruff check and ruff format --check silent; pylint R0801 10.00/10; lint-auth-signals.sh and lint-architecture-boundaries.sh exit 0; no src file over 2100 lines. Owner-touch gate re-detected at the new head (now also reports the live local authoring catalog owner via client.py); its suites and the original functional tests all pass. CI at 834157b: all required checks green.

Post-push mergeability: gh pr view --json mergeStateStatus,mergeable reports BLOCKED / MERGEABLE (BLOCKED = required review / merge queue). Push used git push https://github.com/microsoft/apm.git HEAD:refs/heads/danielmeppiel-issue-3319-multi-marketplace-catalogs (plain fast-forward push of a merge commit; no force).

Ready for maintainer review.


This comment was generated by an AI agent (GitHub Copilot) on behalf of the maintainer. Please verify before acting on it.

@danielmeppiel
Daniel Meppiel (danielmeppiel) merged commit 4782e38 into main Nov 12, 2026
26 checks passed
@danielmeppiel
Daniel Meppiel (danielmeppiel) deleted the danielmeppiel-issue-3319-multi-marketplace-catalogs branch November 12, 2026 23:02
Daniel Meppiel (danielmeppiel) added a commit that referenced this pull request Nov 12, 2026
…-marketplaces

Thread #3628's explicit catalog path through the canonical registration owner: probe_registration(..., explicit_path=) fetches only that file, and MarketplaceCatalogPathError carries the --path errors. marketplace add keeps --path admission and the replace warning; the source-admission owner row keeps both admit_marketplace_catalog_path and parse_registration_source.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] : Support multiple marketplaces (catalogs) in one repository

2 participants