fix(attribution)!: partition author facts by declared audit route - #261
Conversation
Every object mirrored through a dedicated in-cluster ClusterProvider was committed as "unknown (attribution unresolved)", while the same actor's writes through "default" attributed correctly. Mirroring stayed correct, so the loss was visible only in the commit author. A kube-apiserver takes one --audit-webhook-config-file, and that file names one server URL, so one physical cluster posts audit under exactly ONE route. Facts were partitioned by the ClusterProvider's name, so every other provider naming that same cluster read a partition nothing writes. The source-namespace feature encourages exactly that shape: allowedNamespaces and allowSourceNamespaceOverride are provider-wide, so two delegation stances on one cluster need two providers. ClusterProvider.spec.attribution.auditRoute now declares the route a cluster's events arrive on, and that value partitions the facts. It defaults to the object's own name, so an install that sets nothing resolves exactly what it resolved before. Several providers share one cluster's facts by declaring one route; providers on different clusters keep separate partitions, which is what still separates an etcd-snapshot clone from its origin (every identity readable from inside a cloned cluster was cloned along with it). Ingestion gets simpler rather than harder, and loses its only Kubernetes read: the /audit-webhook/<route> existence gate, AuditProviderResolver, and the annotation-routing startup rule are gone. A route is a partition name, not a claim about an object, so a fact for a route no provider declares is stored and expires unread. That also stops dropping audit batches in flight while a provider is being created, which the API server does not retry after a 404. The failure is now loud: a route that has never resolved and produces five unresolved events in a row logs once, naming the fix rather than the symptom. PurgeClusterFacts is deleted along with its decision record. It was unwired, and a shared route makes it unsafe: purging on one provider's deletion would drop the facts of every other provider on that route. The reasoning that still binds (no finalizer, because helm uninstall would strand the object in Terminating) moved onto LegacyClusterProviderFinalizer. BREAKING CHANGE: --author-attribution-cluster-annotation-key is renamed to --author-attribution-audit-route-annotation-key, and the chart value attribution.clusterAnnotationKey to attribution.auditRouteAnnotationKey. There is no deprecated alias: an operator still passing the old flag fails to start. The fact-key infix changes from cluster: to route: with no dual-read, so a rolling upgrade has a TTL-bounded window where events commit as the configured committer. Design: docs/design/attribution-fact-identity.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Takes auditRouteFor from 0% to 100% (including the fallback that resolves the provider name when the ClusterProvider cannot be read, which is what keeps a transient read failure from keying the lookup to a partition nothing writes), covers the never-resolves warning through the real resolver rather than the counter alone, and pins which keys each fact shape writes: the rv-only hatch exists only for a fact with no UID, because a UID-bearing fact's rv key would be dead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
The bug
Reported by the gitops-api team: every object mirrored through a dedicated in-cluster
ClusterProviderwas committed asunknown (attribution unresolved), while the same actor's writes throughdefaultattributed correctly. Mirroring stayed correct, no condition changed, and no reconcile failed, so the loss was visible only in the commit author.Their measurement, reproduced twice: 5 objects
absent, all having waited the full grace window, withwritten=66 / matched=12and zero expiries. Facts were being written the whole time, under a key nobody read.Why
A kube-apiserver takes one
--audit-webhook-config-file, and that file uses the kubeconfig format to specify the remote address of the service, singular. So one physical cluster posts audit under exactly one route.Facts were partitioned by the
ClusterProvider's name, so every other provider naming that same cluster read a partition nothing writes.That shape is one the product encourages.
allowedNamespacesandallowSourceNamespaceOverrideare provider-wide, so a platform admin granting the override to one tenant and withholding it from another has no choice but to create two providers on one cluster — which is exactly what the source-namespace e2e does.The fix
ClusterProvider.spec.attribution.auditRoutedeclares the route a cluster's events arrive on, and that value partitions the facts.metadata.name, so an install that sets nothing resolves exactly what it resolved before.metadata.uid, so a design keying facts by object identity alone could not tell them apart, while a human-chosen route can.Ingestion gets simpler, and loses its last Kubernetes read
The
/audit-webhook/<route>existence gate,AuditProviderResolver, itscmd/main.goadapter, and the annotation-routing startup rule are all deleted. Parse the route, record the fact — no API call.A route is a partition name, not a claim about an object, so a fact for a route no provider declares is stored and expires unread. That also stops dropping audit batches in flight while a provider is being created or recreated, which the API server does not retry after a 404.
The failure is now loud
A route that has never resolved and produces five unresolved events in a row logs once, naming the fix rather than the symptom. The threshold is not one because a lone miss is ordinary under audit-batch delay; a run with nothing ever matched is the signature of a route nobody writes to.
Also removed
PurgeClusterFactsand its decision record. It was unwired, and a shared route makes it unsafe: purging on one provider's deletion would drop the facts of every other provider on that route, and of the operator's own cluster, which was never torn down. The reasoning that still binds — no finalizer, becausehelm uninstallwould strand the object inTerminating— moved ontoLegacyClusterProviderFinalizer, where someone about to re-add one will actually read it.Breaking changes
No deprecated aliases and no dual-read, deliberately: this path has no users to protect.
--author-attribution-cluster-annotation-key--author-attribution-audit-route-annotation-keyattribution.clusterAnnotationKeyattribution.auditRouteAnnotationKeycluster:<name>route:<auditRoute>An operator still passing the old flag fails to start, which is the loud outcome. The key-infix change gives a rolling upgrade a TTL-bounded window (minutes) where events commit as the configured committer — the ordinary degradation, and the same rollout behaviour the
v2→v3key bump had.Tests
test/e2e/audit_route_attribution_e2e_test.go): a dedicated in-clusterClusterProviderwhose name is notdefault, declaringauditRoute: default, must commit as the impersonated OIDC identity and specifically not the unresolved placeholder. This fails onmain. A second case reaches an object through arules[].sourceNamespaceoverride, pinning the report's claim that the override was innocent.AuditRoute()defaulting; two providers sharing a route joining one fact while a non-declaring one still misses; the Declare-time capture and its teardown; the warning's four cases; the handler storing a fact for a route no provider declares.task lint,task test, andtask test-e2eall pass (Ran 69 of 91 — 69 Passed, 0 Failed).Stacked on #262
This targets
chore/docs-linting(#262), notmain, so the unrelated markdown-linting work squash-merges as its own commit. Review #262 first; GitHub retargets this tomainautomatically once it merges. The diff shown here is the fix alone.Design:
docs/design/attribution-fact-identity.md🤖 Generated with Claude Code