Changes to webhook.Validator pattern cause version update problems

The webhook.Validator interface is implemented on API types to provide easy webhook creation. However, tying this interface to API types means that API code is coupled with implementation code.

When the webhook.Validator interface is changed, modules cannot import each other across the breaking change boundary. It is difficult to achieve synchronous updates across a single organization maintaining many controllers, let alone the entire community.

For example, in 0.15, the webhookValidate interface was changed to return (admission.Warnings, error) instead of return error. We have updated our controllers to 0.15, but can't import metallb's APIs until they are also at 0.15, because their use of webhook.Validator is incompatible with our module's choice to use 0.15.

Breaking changes in other areas of the code are less problematic because it is common for repos to only import each other for APIs. If the code dependent on the breaking change is not actually used, it will not be included in compilation and it will compile fine, even though a complete compilation would fail.

We were not aware of the danger of this pattern, and only recently started moving away from it for our controllers to reduce the scope of this problem. But we cannot fix it in third party repos, without forking it and fixing it locally ourselves. We could do pull requests, but they will run into their own problems where updating their repo requires them to update all the downstream repos.

Is there a solution for controller-runtime users? If not, is there something that can be done in the project to reduce this issue?

Related: #2327

The issue with metallb is even worse than we thought because metallb imports open policy agent. So, it seems like we can't import metallb until metallb is at 0.15 and metallb can't update to 0.15 until open policy agent is at 0.15.

Edit: open policy agent has a tag which is on controller-runtime 0.15 so we can bump the version locally and resolve that issue.

But it does bring up the issue that if any of our indirect dependencies skipped 0.15, we would be forced to bump up to 0.16 too.

/kind support

API packages should not import anything other than other api packages to avoid precisely this problem. Controller-Runtime does not guarantee compatibility across versions.

@nathanperkins

@nathanperkins with your resolution from the "edit" you had in your comment, does @alvaroaleman's explanation along with the fix you provided settle this issue?

@alvaroaleman

@alvaroaleman

API packages should not import anything other than other api packages to avoid precisely this problem. Controller-Runtime does not guarantee compatibility across versions.

This is true but not possible with the webhook.Validator pattern because is implemented on the API type itself and the methods return admission types which are imported from controller-runtime.

https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.16.3/pkg/webhook/admission#Validator

If this is the suggestion, then the webhook.Validator pattern should be removed in favor of patterns which don't require imports from the API package or it needs to be adjusted so that it can be implemented without API packages having to import controller-runtime.

@troy0820

@nathanperkins with your resolution from the "edit" you had in your comment, does @alvaroaleman's explanation along with the fix you provided settle this issue?

It doesn't settle the issue. It's not possible to implement the webhook.Validator pattern without importing controller-runtime in the API package.

Not sure if you're aware but you can implement CustomValidator & CustomDefaulter outside of the API package instead.

This doesn't help of course if a 3rd party dependency of yours still implements Validator & Defaulter instead. For this it would only help if we drop it in CR and the dependency upgrades to a new CR version without Validator & Defaulter.

I know that cluster-api did a lot to decouple the api types from importing controller-runtime

This PR is what I'm talking about. kubernetes-sigs/cluster-api#9045

It doesn't settle the issue. It's not possible to implement the webhook.Validator pattern without importing controller-runtime in the API package.

Fair, but at that point you should not be doing that and implement it outside of that.

I personally do think that the webhook.Validator pattern is a mistake because of this precise reason, api packages should not import anything other than other api packages.

@alvaroaleman

@alvaroaleman

Fair, but at that point you should not be doing that and implement it outside of that.

Sure but this pattern encourages doing this and it is also described this way in the Kubebuilder book.

https://book.kubebuilder.io/cronjob-tutorial/webhook-implementation

We can only fix this in our projects but the presence of this pattern causes problems across the ecosystem which are out of our control.

I personally do think that the webhook.Validator pattern is a mistake because of this precise reason, api packages should not import anything other than other api packages.

It seems like the only solution for this well-known problem is to not use webhook.Validator. Given that it causes unavoidable cross project upgrade issues, can we remove it or replace it in the next version?

@sbueringer

Not sure if you're aware but you can implement CustomValidator & CustomDefaulter outside of the API package instead.

Now that we are aware of this issue, we use the other patterns in our projects which decouple controller-runtime from our API types. But any project which uses this pattern will cause upgrade issues in the K8s ecosystem.

We should check whether you can implement this pattern outside your API package by embedding the API type, like:

type PodValidator struct {
  corev1.Pod
}

func (v PodValidator) ValidateCreate(...) (...) {
  ...
}

That would at least provide an easy path for projects to shift their existing webhook code and avoid the API dependency issues.

If this works, we could update the Kubebuilder doc to suggest this instead.

edit: on second thought, I doubt this would work. It would require the controller-runtime internals to be unmarshaling JSON into the webhook types, which would be inefficient. They probably use type assertion, which wouldn't work here.

I think with the PR that was linked from cluster-api, it should encourage the kubebuilder scaffold to behave differently when that time comes to migrate from the older scaffold. Albeit a change you are still importing apimachinery/runtime which changes versions with k8s releases as well but not coupled to CR.

I think that's a reasonable ask and maybe something we can settle on to not only provide that as a solution but also adhere to the best practices around api importing.

We can deprecate the current pattern as long as we have a clear better way to create webhooks, a deprecation in the interfaces and code would allow folks to migrate over time

Deprecating the pattern over time sounds good to me.

If you create a replacement interface, it could look something like this:

type Validator interface {
  ValidateCreate(context.Context, client.Object) (webhook.Warnings, error)
  ...
}

Migrating existing code is simple, with a pod := o.(*v1.Pod) at the top.
Having access to the context enables using client queries for validations.

We might be able to do a bit better with a generic interface?

separate webhooks from the api types package

separate webhooks from the api types package (#672)

FYI this bug is not fully fixed, at least not for practical purposes. Even Kubebuilder's own example contains a groupversion_info.go which depends on controller-runtime: https://github.com/kubernetes-sigs/kubebuilder/blob/master/testdata/project-v4/api/v2/groupversion_info.go. So, while the new design is better, any API package, at least any created by Kubebuilder or inspired by it (and let's be honest, next to nobody can do it without Kubebuilder), will depend on controller-runtime directly.

This sounds like a kubebuilder issue.

You could also follow this pattern instead: https://github.com/kubernetes-sigs/cluster-api/blob/main/api/v1beta1/groupversion_info.go#L33

This sounds like a kubebuilder issue

Okay, I will let them know.

You could also follow this pattern instead: https://github.com/kubernetes-sigs/cluster-api/blob/main/api/v1beta1/groupversion_info.go#L33

Thanks, I'll try to apply it!

@sbueringer

@sbueringer hi, sorry to bother you again. Do you plan the same treatment of conversion webhooks? Right now, controller-runtime tests add a dependency on controller-runtime to API, and Kubebuilder does the same.

Yeah conversion is an unsolved problem at the moment. Here's a corresponding issue: #3130

Do you plan the same treatment of conversion webhooks?

In general yes, I personally don't have the bandwith at the moment.

Update K8s libraries to v1.33.1

Update K8s libraries to v1.33.1 (#7232)

Navigation Menu

Repository navigation

Changes to webhook.Validator pattern cause version update problems #2596

31 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Changes to webhook.Validator pattern cause version update problems #2596

Description

Activity

nathanperkins commented on Nov 27, 2023

troy0820 commented on Nov 27, 2023

alvaroaleman commented on Dec 2, 2023

troy0820 commented on Jan 4, 2024

nathanperkins commented on Jan 4, 2024

sbueringer commented on Jan 4, 2024

troy0820 commented on Jan 4, 2024

alvaroaleman commented on Jan 4, 2024

nathanperkins commented on Jan 4, 2024

nathanperkins commented on Jan 4, 2024

troy0820 commented on Jan 4, 2024

vincepri commented on Jan 4, 2024

nathanperkins commented on Jan 4, 2024

vincepri commented on Jan 4, 2024

31 remaining items

dtantsur commented on Mar 21, 2025

sbueringer commented on Mar 21, 2025

dtantsur commented on Mar 21, 2025

dtantsur commented on Mar 25, 2025

sbueringer commented on Mar 25, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions