Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cmd/manifest-analyzer/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ func TestRun_ScanJSON(t *testing.T) {

// scanRepoFixture builds a tiny two-folder repo: a plain KRM app folder and a kustomize
// overlay reaching an out-of-subtree base, so scan-repo reports both an accepted plain
// candidate and a refused kustomize-overlay one.
// candidate and an accepted kustomize-overlay one (render-root scoping adopts the overlay).
func scanRepoFixture(t *testing.T) string {
t.Helper()
root := t.TempDir()
Expand Down Expand Up @@ -167,7 +167,7 @@ func TestRun_ScanRepoText(t *testing.T) {
if code := run([]string{"--mode", "scan-repo", scanRepoFixture(t)}, &out, &errBuf); code != 0 {
t.Fatalf("exit = %d, want 0 (stderr=%s)", code, errBuf.String())
}
for _, want := range []string{"candidates:", "kustomize-overlay", "overlay-fan-out-unsupported"} {
for _, want := range []string{"candidates:", "kustomize-overlay", "accepted=2 refused=0"} {
if !strings.Contains(out.String(), want) {
t.Errorf("scan-repo text missing %q:\n%s", want, out.String())
}
Expand Down
5 changes: 3 additions & 2 deletions docs/design/support-boundary/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,9 @@ flowchart LR

The remaining overlay gap is deliberately narrow: creating a **new object** and adding its
`resources:` entry needs a placement/write-path correction. It is planned, not shipped.
`scan-repo` likewise still labels external-base overlays as unsupported while its discovery
classification catches up with the runtime.
`scan-repo` now adopts external-base overlays too, matching the runtime: it reports a
`kustomize-overlay` candidate as accepted and lets its `editable` count show how much the
overlay owns (a pure passthrough overlay is adopted yet `editable: 0`).

## Active design work

Expand Down
10 changes: 6 additions & 4 deletions docs/design/support-boundary/render-root-scoping.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,10 @@ flowchart LR
whole overlay merely by existing.
- A base-owned field is refused rather than written through to the shared base.

The discovery report has not caught up: `scan-repo` still uses
`overlay-fan-out-unsupported` for these layouts. That is a classification follow-up, not a
runtime boundary.
The discovery report has caught up: `scan-repo` runs the same adoption gate over an
external-base overlay's render scope, so a `kustomize-overlay` candidate is now reported
**accepted** (the `overlay-fan-out-unsupported` reason is retired) with its `editable` count
showing how much the overlay owns.

## 2. Why verification, not inversion

Expand Down Expand Up @@ -93,7 +94,8 @@ feature.

1. Correct overlay-local new-object placement, including a deterministic `resources:` entry.
2. Add missing `images:` and `replicas:` declarations only under the same verification proof.
3. Flip discovery classification and add a dedicated cluster end-to-end overlay case.
3. Discovery classification is flipped (an external-base overlay reports accepted); a dedicated
cluster end-to-end overlay case remains.
4. Author a narrow scalar strategic-merge patch for base-owned fields; lists, deletes, and
structural merges require separate decisions.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,14 +136,14 @@ flowchart TD
### Layout classification

Discovery reports the layout and its **scanner classification** as two distinct truths.
The narrow external-base overlay runtime has shipped, but this scanner has not yet flipped
its overlay classification (see below).
The narrow external-base overlay runtime has shipped, and this scanner now matches it: it
runs the same adoption gate over the overlay's render scope rather than blanket-refusing.

| `layout` | Meaning | Scanner output today |
|---|---|---|
| `plain` | Directory of KRM docs, explicit namespaces, no kustomization | ✅ supported today |
| `kustomize-single` | One render root; `namespace` + `resources`/`images`/`replicas` | ✅ supported today |
| `kustomize-overlay` | Base + N overlay roots, one namespace per overlay | ⛔ reports `overlay-fan-out-unsupported`; runtime accepts the narrow existing-document/image/replica slice, and this classification needs to catch up |
| `kustomize-overlay` | Base + N overlay roots, one namespace per overlay | ✅ adopted today; the render scope passes the gate and `editable` shows how much the overlay owns (a pure passthrough overlay is adopted yet `editable: 0`). Base-owned fields and new overlay objects stay write-time-planned |
| `refused-structural` | Helm inflation, generators with hash suffixes, `components`, `namePrefix`/`nameSuffix`, remote bases, `configurations`/`openapi`/`crds` | ⛔ permanent — the support contract |
| `refused-fleet-root` | `clusters/` + `apps/` + `infra/` cluster-root repo (a `GitTarget` points at an app subtree, never a cluster root) | ⛔ out of scope by design |
| `refused-out-of-band` | Namespace/transform injected outside the folder (Flux `postBuild`/`targetNamespace`, Argo Application-level overrides) | ⛔ permanent — round-trip cannot hold |
Expand All @@ -160,11 +160,12 @@ its overlay classification (see below).
> cut therefore ships four candidate layouts: `plain`, `kustomize-single`,
> `kustomize-overlay`, `refused-structural`.

The distinction between `overlay-fan-out-unsupported` and `refused-structural` remains
load-bearing: the first currently identifies a **scanner lag** on a bounded supported
layout, while the second is the **permanent** boundary. Discovery must never collapse them
into one refusal. The classification follow-up should make the scanner match the runtime
without widening either accepted layout or the structural wall.
The distinction between an **adopted** `kustomize-overlay` and a **permanently refused**
`refused-structural` render root stays load-bearing: the first is a bounded supported layout
the operator renders through render-root scoping, the second is the permanent boundary.
Discovery must never collapse them — an overlay must not be reported as a hopeless structural
refusal, and helm inflation must not be reported as an adoptable overlay. The former
`overlay-fan-out-unsupported` reason is retired now that the scanner matches the runtime.

## The report contract

Expand Down Expand Up @@ -283,11 +284,14 @@ refuses `openapi`/`crds` alongside `configurations`, matching the

**Classification findings.**

- The overlay discriminator that trips `overlay-fan-out-unsupported` is precisely **the base
escaping the render root's own subtree** (`../../base` resolves outside the candidate) —
the very fact the operator's hard subtree-scope hits. Fan-out (how many overlay roots
share the base) is carried as informative `detail`, not the trigger. A base nested
*within* the subtree keeps the root `kustomize-single` (accepted today).
- The overlay discriminator that marks a render root `kustomize-overlay` is precisely **the
base escaping the render root's own subtree** (`../../base` resolves outside the candidate).
A base nested *within* the subtree keeps the root `kustomize-single`. Both are adopted; the
distinction is only that an overlay renders — and cannot edit — an out-of-subtree base, so
its `editable` count can be lower than `rendered`. Acceptance of the overlay comes from
running the same gate the live writer runs over the overlay's **render scope** (the overlay
subtree plus the exact base files its graph reaches), so the report cannot drift from the
runtime; the former `overlay-fan-out-unsupported` reason is retired.
- `refused-fleet-root` ships as a repo-summary flag (`fleetRoot: true`), not a
per-candidate layout: the repo root is never itself a candidate, and leaf app folders
still surface normally. `refused-out-of-band` is not detected yet — a structure-only
Expand Down Expand Up @@ -334,10 +338,11 @@ nested base (deduped `rendered`), an overlay base holding parked YAML (excluded
shipped — the acceptance gate, render-root computation
([images/replicas edit-through](finished/images-and-replicas-edit-through.md)),
and namespace inference. Discovery is additive tooling on top of the shared engine.
- **Overlay support changes discovery's *output*, not its code.** Today overlay
candidates still report `acceptedByOperator: false` + `overlay-fan-out-unsupported`.
The pending classification change should report the shipped existing-document and
`images:`/`replicas:` slice, while keeping new overlay resource creation planned.
- **Overlay support changed discovery's *output*, not the operator.** An external-base
overlay candidate now reports `acceptedByOperator: true` (its render scope passes the same
gate the live writer runs) with an `editable` count that shows how much it owns — a pure
passthrough overlay is adopted yet `editable: 0`. New overlay object creation and base-owned
field edits stay write-time-planned; the scan reports folder adoption, not write capability.

## Test plan

Expand Down Expand Up @@ -382,12 +387,10 @@ nested base (deduped `rendered`), an overlay base holding parked YAML (excluded
remains a presentation choice.
4. **Library vs. CLI as the integration point** — _settled:_ library-first
(`ScanRepo`), CLI (`--mode scan-repo`) as the thin wrapper + CI gate.
5. **`overlay-fan-out-unsupported` naming** — the reason code still reads as if *fan-out*
(a base shared by several overlays) is the trigger, but the real trigger is a base
**escaping the render root's own subtree**, true even at fan-out = 1. The first cut
carries the shared-root count as descriptive `detail`, not as the condition. Open:
rename to something like `overlay-base-out-of-subtree`, or keep the published code — it
is part of `pkg/manifestanalyzer`'s surface — and rely on the detail. _Not settled._
5. **`overlay-fan-out-unsupported` naming** — _moot: retired._ Render-root scoping shipped and
the scanner now adopts external-base overlays, so this reason is no longer emitted. The
public `pkg/manifestanalyzer` constant is kept (deprecated) for source compatibility; the
naming debate is closed because the code is gone from the output.
6. **Repo-level `--policy refuse` gate** — the eventual "zero acceptable candidates → exit
1" onboarding gate (see [Exit codes](#the-report-contract)) is **deferred**: the first
cut always exits 0 (or 2 on an I/O error) and `--policy` is not applied in scan-repo
Expand Down
142 changes: 105 additions & 37 deletions internal/manifestanalyzer/scan_repo.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,10 @@ import (
// generation yet, no rename of the existing --mode discovery, and no repo-level
// --policy refuse exit semantics — see the design doc's "explicitly defer" list.

// Layout is the structural shape of a candidate subtree. Layout and acceptedByOperator
// are two distinct truths that diverge while overlays stay unsupported: a kustomize-overlay
// has a well-understood layout yet is not accepted until render-root scoping lands.
// Layout is the structural shape of a candidate subtree. Layout and acceptedByOperator are
// two distinct truths: a kustomize-overlay has a well-understood layout and is now adopted
// when its render scope passes the gate, but its editable count can still be 0 when every
// rendered field is base-owned.
type Layout string

const (
Expand All @@ -46,9 +47,10 @@ const (
LayoutKustomizeSingle Layout = "kustomize-single"
// LayoutKustomizeOverlay is a render root that reaches a base kustomization OUTSIDE
// its own subtree (the classic base/ + overlays/{env} shape reached via ../../base).
// The operator hard-scopes to one subtree and cannot see the base, so it is refused
// today with the forward-looking overlay-fan-out-unsupported reason — it flips to
// accepted if render-root scoping ships.
// Render-root scoping shipped, so the operator adopts it when the adoption gate accepts
// its render scope; the editable count shows how much of the render the overlay owns (a
// pure passthrough overlay is adopted yet editable: 0). It is a distinct layout from
// kustomize-single because it still renders — and cannot edit — an out-of-subtree base.
LayoutKustomizeOverlay Layout = "kustomize-overlay"
// LayoutRefusedStructural is a render root whose kustomization uses a feature the
// contextual-namespace writer cannot map back to editable source (helm inflation,
Expand All @@ -57,14 +59,14 @@ const (
LayoutRefusedStructural Layout = "refused-structural"
)

// Refusal reason codes. The distinction between the two is load-bearing:
// ReasonOverlayFanOutUnsupported is a forward-looking "not yet" that flips to accepted
// if render-root scoping ships; ReasonRefusedStructural is the permanent boundary. Discovery must
// never collapse them into one "refused".
const (
ReasonOverlayFanOutUnsupported = "overlay-fan-out-unsupported"
ReasonRefusedStructural = "refused-structural"
)
// ReasonRefusedStructural is the permanent support boundary: a render root whose kustomization
// uses a construct the writer cannot map back to editable source. It is the only render-root
// refusal reason now that external-base overlays are adopted through render-root scoping (the
// former forward-looking overlay-fan-out-unsupported reason is retired; the public
// pkg/manifestanalyzer constant is kept for consumers pinning the string, but the classifier no
// longer emits it). An overlay refused for a real structural fault carries that fault's own
// gate-issue code instead.
const ReasonRefusedStructural = "refused-structural"

// RefusalReason is one machine-readable reason a candidate is not accepted, with a
// human detail. A candidate carries none when accepted.
Expand Down Expand Up @@ -191,7 +193,7 @@ func scanRepoFS(ctx context.Context, fsys fs.FS) RepoReport {

candidates := make([]RepoCandidate, 0)
for _, rootDir := range renderRoots(kusts) {
candidates = append(candidates, classifyRenderRoot(ctx, fsys, rootDir, kusts, kustContent, store))
candidates = append(candidates, classifyRenderRoot(ctx, fsys, rootDir, scan, kusts, kustContent, store))
}
candidates = append(candidates, plainCandidates(ctx, fsys, store, kusts, ownedFiles)...)

Expand All @@ -205,12 +207,15 @@ func scanRepoFS(ctx context.Context, fsys fs.FS) RepoReport {
}

// classifyRenderRoot classifies one kustomize render root into a candidate: refused
// (unsupported kustomization), a kustomize-overlay reaching an out-of-subtree base, or
// a self-contained kustomize-single.
// (unsupported kustomization), an external-base kustomize-overlay, or a self-contained
// kustomize-single. An overlay is no longer refused merely for reaching an out-of-subtree
// base — render-root scoping shipped, so the operator adopts it — and instead runs the same
// adoption gate over its render scope.
func classifyRenderRoot(
ctx context.Context,
fsys fs.FS,
rootDir string,
scan FolderScan,
kusts map[string]*kustomizationDoc,
kustContent map[string][]byte,
store *ManifestStore,
Expand All @@ -233,13 +238,19 @@ func classifyRenderRoot(

outsideBases := outOfSubtreeBases(rootDir, kusts)
if len(outsideBases) > 0 {
// External-base overlay: the operator now renders it through render-root scoping, so it
// is adopted when the same gate the live writer runs accepts its render scope. The
// editable count still shows how much the overlay owns — a pure passthrough overlay is
// adoptable yet editable: 0, since every field is base-owned. A gate refusal (foreign
// content in the overlay, an unbuildable base, an unsupported nested kustomization)
// surfaces as its own reason rather than a blanket overlay refusal.
c.Layout = LayoutKustomizeOverlay
c.AcceptedByOperator = false
c.ReadScope = outsideBases
c.RefusalReasons = []RefusalReason{{
Code: ReasonOverlayFanOutUnsupported,
Detail: overlayFanOutDetail(outsideBases[0], kusts),
}}
acc := overlayCandidateAcceptance(ctx, rootDir, scan, kusts)
c.AcceptedByOperator = acc.Accepted
if !acc.Accepted {
c.RefusalReasons = issuesToReasons(acc.Issues)
}
c.Resources = countResources(store, rootDir, rendered)
return c
}
Expand Down Expand Up @@ -304,6 +315,78 @@ func plainCandidates(
return out
}

// overlayCandidateAcceptance runs the operator's own adoption gate over an external-base
// overlay's RENDER SCOPE — the overlay subtree PLUS the exact base files its resources/patches
// graph reaches — so the discovery report matches what the live writer's render-root scoping
// (internal/git/render_scope.go) decides. Only the files the graph actually reaches enter the
// scope, never a whole base directory, so parked YAML a base does not reference can never
// refuse the overlay (mirroring the runtime's "read scope is the exact reachable file set").
//
// The scoped store keeps repo-relative paths, so a `../../base` reference resolves within it
// exactly as kustomize resolves it. Acceptance here is folder adoption (GitPathAccepted); the
// write half — editable overlay-local documents and declared images/replicas, but never a
// base-owned field or a new overlay object — is out of scope for a read-only report, and the
// candidate's editable count already reflects how much of the render the overlay owns.
func overlayCandidateAcceptance(
ctx context.Context,
rootDir string,
scan FolderScan,
kusts map[string]*kustomizationDoc,
) Acceptance {
reached := renderScopePaths(rootDir, kusts)
scoped := FolderScan{}
for _, f := range scan.YAMLFiles {
if pathWithin(f.Path, rootDir) || setContains(reached, f.Path) {
scoped.YAMLFiles = append(scoped.YAMLFiles, f)
}
}
// Only the overlay's OWN non-YAML/foreign content bears on its acceptance; a base's loose
// files are never read (the render scope pulls in referenced files only), just as at runtime.
for _, p := range scan.NonYAML {
if pathWithin(p, rootDir) {
scoped.NonYAML = append(scoped.NonYAML, p)
}
}
for _, fe := range scan.Foreign {
if pathWithin(fe.Path, rootDir) {
scoped.Foreign = append(scoped.Foreign, fe)
}
}
store := buildStore(ctx, scoped, nil, WriterAllowlist())
return Accept(store, AcceptancePolicy{Allowlist: WriterAllowlist()})
}

// renderScopePaths returns the files an overlay render root reaches through its resources and
// patches graph: every referenced resource file, plus each reached kustomization's own file
// and strategic-merge patch inputs. These are the exact build inputs kustomize loads, so the
// scoped acceptance store can render `../../base` without importing a base's unreferenced
// content. Paths are scan-root-relative slash paths, matching the store's file keys.
func renderScopePaths(rootDir string, kusts map[string]*kustomizationDoc) map[string]struct{} {
paths := map[string]struct{}{}
for f := range reachedResourceFilesFrom(rootDir, kusts) {
paths[f] = struct{}{}
}
dirs := reachedKustomizationDirs(rootDir, kusts)
dirs[rootDir] = struct{}{}
for dir := range dirs {
doc := kusts[dir]
if doc == nil {
continue
}
paths[doc.path] = struct{}{}
for _, patch := range doc.patches {
paths[patch] = struct{}{}
}
}
return paths
}

// setContains reports membership in a set-like map.
func setContains(set map[string]struct{}, key string) bool {
_, ok := set[key]
return ok
}

// candidateAcceptance runs the structure-only adoption gate over the candidate subtree —
// the exact gate the operator's live writer runs (Scan with WriterAllowlist, which retains
// the kustomize build directives and the operator's .sops.yaml bootstrap config). It
Expand Down Expand Up @@ -425,21 +508,6 @@ func reachedResourceFiles(kusts map[string]*kustomizationDoc) map[string]struct{
return out
}

// overlayFanOutDetail explains why an overlay is unsupported: the shared base and how many
// render roots reach it from outside their subtree.
func overlayFanOutDetail(base string, kusts map[string]*kustomizationDoc) string {
shared := 0
for _, root := range renderRoots(kusts) {
if _, ok := reachedKustomizationDirs(root, kusts)[base]; ok && !pathWithin(base, root) {
shared++
}
}
return fmt.Sprintf(
"base %q is read from outside this folder's subtree and is shared by %d render root(s); "+
"render-root scoping required",
base, shared)
}

// refusedStructuralDetail names the specific unsupported kustomize features so the
// refusal is actionable, not a bare "refused". The features come off the parsed doc, which is
// the same judgement the acceptance gate reads, so the scan and the operator cannot drift.
Expand Down
Loading