Repository navigation
Metrics service is not garbage collected #459
Description
I think it would make sense to set an ownerreference to the metrics service to be garbage collected when the operator is gone. One idea I have in mind is to use the downwardAPI to pass the pod name and set the ownerreference directly to it, since it will recreate it anyways. The other idea I have is to create the service from the yaml and just update it if needed from the operator. This way users can know that there is a resource that needs to be removed (helm takes care of it for example).
Activity
pepov commentedon Sep 6, 2018
I can work on this if any of the proposed way makes sense.
bofm commentedon Sep 21, 2018
Maybe an owner reference for the metrics service could help? The owner reference should point to the operator deployment.
pepov commentedon Sep 21, 2018
I agree with the ownerref, but not sure about marking the deployment as the owner because it's not necessarily a deployment that creates the operator. What about looking up the operator pod's ownerref and setting it to whatever has been set there?
shawn-hurley commentedon Sep 21, 2018
I do think this is a change that we will want. I'll defer to @hasbro17 to make sure.
I think that for now, using the downward API, and setting the owner reference as the pod makes sense for now.
hasbro17 commentedon Nov 28, 2018
We turned off the metrics service creation in v0.1.0 as we're currently reworking how to expose both the controller specific metrics(queue length, number of reconciles) and app specific metrics via separate registries and ports #715
But yes we should have the service be owned by the operator so it's garbage collected.
We could have the operator-pod own the service it creates, but we need to then consider what happens if we have 2 replica pods for the operator deployment(now that we support leader election #724 ).
Note that the metrics service selects all operator pods.
https://github.com/operator-framework/operator-sdk/blob/master/pkg/k8sutil/k8sutil.go#L79
Although if we're setting readiness checks then the non-leader pod endpoints would not be selected by the service since they're Unready.
I imagine we could have the following workflow:
@lilic Thoughts. We can probably address this alongside #715
lilic commentedon Nov 29, 2018
Will do this as soon as the initial metrics PR is merged! #786