panic: runtime error: invalid memory address or nil pointer dereference

Your environment

Operator Version: 1.8.0

Connect Server Version: 1.7.2

Kubernetes Version:

❯ kubectl version --output=yaml
clientVersion:
  buildDate: "2023-06-14T09:53:42Z"
  compiler: gc
  gitCommit: 25b4e43193bcda6c7328a6d147b1fb73a33f1598
  gitTreeState: clean
  gitVersion: v1.27.3
  goVersion: go1.20.5
  major: "1"
  minor: "27"
  platform: linux/amd64
kustomizeVersion: v5.0.1
serverVersion:
  buildDate: "2023-07-17T09:25:02Z"
  compiler: gc
  gitCommit: 2ce07bca20da6c3fa49a26fe8e5d53cb8a1e073f
  gitTreeState: clean
  gitVersion: v1.27.3-gke.1700
  goVersion: go1.20.5 X:boringcrypto
  major: "1"
  minor: "27"
  platform: linux/amd64

What happened?

Operator pod panics with the following error after recreating the pod:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x137f5b0]

What did you expect to happen?

The operator doesn't panic.

Steps to reproduce

kubectl delete -n onepassword-connect onepassword-connect-operator-abc123-xyz987

Notes & Logs

❯ kubectl -n onepassword-connect logs -f onepassword-connect-operator-855584b64b-qgh2j
2023-12-04T23:40:27Z    INFO    setup   Operator Version: 1.8.0
2023-12-04T23:40:27Z    INFO    setup   Go Version: go1.20.7
2023-12-04T23:40:27Z    INFO    setup   Go OS/Arch: linux/amd64
2023-12-04T23:40:27Z    INFO    setup   Version of operator-sdk: 1.29.0
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x137f5b0]

goroutine 1 [running]:
k8s.io/client-go/discovery.convertAPIResource(...)
        /workspace/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:114
k8s.io/client-go/discovery.convertAPIGroup({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc00004af78, 0x15}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
        /workspace/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:95 +0x6f0
k8s.io/client-go/discovery.SplitGroupsAndResources({{{0xc000532390, 0x15}, {0xc00004e980, 0x1b}}, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
        /workspace/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:49 +0x125
k8s.io/client-go/discovery.(*DiscoveryClient).downloadAPIs(0xc0001b6c80?)
        /workspace/vendor/k8s.io/client-go/discovery/discovery_client.go:328 +0x3de
k8s.io/client-go/discovery.(*DiscoveryClient).GroupsAndMaybeResources(0xc0001b70b0?)
        /workspace/vendor/k8s.io/client-go/discovery/discovery_client.go:203 +0x65
k8s.io/client-go/discovery.ServerGroupsAndResources({0x1afd738, 0xc0000dc810})
        /workspace/vendor/k8s.io/client-go/discovery/discovery_client.go:413 +0x59
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources.func1()
        /workspace/vendor/k8s.io/client-go/discovery/discovery_client.go:376 +0x25
k8s.io/client-go/discovery.withRetries(0x2, 0xc0005af0c8)
        /workspace/vendor/k8s.io/client-go/discovery/discovery_client.go:651 +0x71
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources(0x0?)
        /workspace/vendor/k8s.io/client-go/discovery/discovery_client.go:375 +0x3a
k8s.io/client-go/restmapper.GetAPIGroupResources({0x1afd738?, 0xc0000dc810?})
        /workspace/vendor/k8s.io/client-go/restmapper/discovery.go:148 +0x42
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper.func1()
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:94 +0x25
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).setStaticMapper(...)
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:130
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper(0xc0001a6360?, {0x0, 0x0, 0xfa3b126c3890da01?})
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:110 +0x182
sigs.k8s.io/controller-runtime/pkg/cluster.setOptionsDefaults.func1(0xc0001df500?)
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/cluster/cluster.go:217 +0x25
sigs.k8s.io/controller-runtime/pkg/cluster.New(0xc000296b40, {0xc0001b7a70, 0x1, 0x0?})
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/cluster/cluster.go:159 +0x18d
sigs.k8s.io/controller-runtime/pkg/manager.New(_, {0xc0001df500, 0x0, 0x0, {{0x1af9e40, 0xc00040b580}, 0x0}, 0x0, {0x0, 0x0}, ...})
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/manager/manager.go:351 +0xf9
main.main()
        /workspace/main.go:144 +0x535

I did some digging and this looks like a problem that would be resolved by updating the dependencies to a version that supports a modern kubernetes version.

I briefly tried to upgrade dependencies to test, however, there are several breaking changes due to deprecations in the package raising the errors.

Since this hasn't seen a new build in 4 months, it is critical that this application is assigned to someone for ongoing maintenance considering how quickly kubernetes moves.

Related:
kubernetes-sigs/controller-runtime#2422

Pinned version: https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.14.5

Breaking versions:

latest version: https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.16.3

There is no compatible release in the v0,14.x tree.

@lhriley

Hey @lhriley! 👋🏻

Thank you for raising this with us and providing so many details. This will help us in narrowing down the problem and find out a solution to this.
We're currently investigating this and we will come back to you when we have any updates. 😄

@lhriley

Hey @lhriley,

We've got the release notes on the latest operator-sdk v1.33.0, which provide some migration steps to upgrade some dependencies and scaffolding of this project. This release came out right after we've replied to you.

This migration also addresses the issue you're facing, as well as updating the project's dependencies to their latest versions.
I'll be working on this migration for the upcoming weeks and will reach out to you once we've reached to a good state with it. 😄

@edif2008

@edif2008 thanks for the update. Looking forward to a new release!

@edif2008

@edif2008 Hey we rolled out latest version and but still fails retrieve 1P records

2024-01-30T10:46:03Z    INFO    setup   Operator Version: 1.8.1
2024-01-30T10:46:03Z    INFO    setup   Go Version: go1.21.6
2024-01-30T10:46:03Z    INFO    setup   Go OS/Arch: linux/amd64
2024-01-30T10:46:03Z    INFO    setup   Version of operator-sdk: 1.33.0
2024-01-30T10:46:03Z    INFO    setup   Automated Connect Management Disabled
2024-01-30T10:46:03Z    INFO    setup   starting manager
2024-01-30T10:46:03Z    INFO    controller-runtime.metrics      Starting metrics server
2024-01-30T10:46:03Z    INFO    controller-runtime.metrics      Serving metrics server  {"bindAddress": ":8080", "secure": false}
2024-01-30T10:46:03Z    INFO    starting server {"kind": "health probe", "addr": "[::]:8081"}
2024-01-30T10:46:03Z    INFO    Starting EventSource    {"controller": "onepassworditem", "controllerGroup": "onepassword.com", "controllerKind": "OnePasswordItem", "source": "kind source: *v1.OnePasswordItem"}
2024-01-30T10:46:03Z    INFO    Starting Controller     {"controller": "onepassworditem", "controllerGroup": "onepassword.com", "controllerKind": "OnePasswordItem"}
2024-01-30T10:46:03Z    INFO    Starting EventSource    {"controller": "deployment", "controllerGroup": "apps", "controllerKind": "Deployment", "source": "kind source: *v1.Deployment"}
2024-01-30T10:46:03Z    INFO    Starting Controller     {"controller": "deployment", "controllerGroup": "apps", "controllerKind": "Deployment"}
2024-01-30T10:46:03Z    INFO    Starting workers        {"controller": "onepassworditem", "controllerGroup": "onepassword.com", "controllerKind": "OnePasswordItem", "worker count": 1}
2024-01-30T10:46:03Z    INFO    Starting workers        {"controller": "deployment", "controllerGroup": "apps", "controllerKind": "Deployment", "worker count": 1}
2024-01-30T10:46:03Z    ERROR   Reconciler error        {"controller": "onepassworditem", "controllerGroup": "onepassword.com", "controllerKind": "OnePasswordItem", "OnePasswordItem": {"name":"*****,"namespace":"default"}, "namespace": "default", "name": "******", "reconcileID": "8e2eb6fe-4d21-43a4-a05b-3771ea6ce9f8", "error": "Failed to retrieve item: status 401: Invalid bearer token"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227
2024-01-30T10:46:03Z    ERROR   Reconciler error        {"controller": "onepassworditem", "controllerGroup": "onepassword.com", "controllerKind": "OnePasswordItem", "OnePasswordItem": {"name":"********","namespace":"default"}, "namespace": "default", "name": "****", "reconcileID": "628a8ea9-53d6-4f5f-a473-9b7169da9b1d", "error": "Failed to retrieve item: status 401: Invalid bearer token"}

@rstreics-lb

Hey @rstreics-lb,

It seems that now the issue is different:

Failed to retrieve item: status 401: Invalid bearer token

This sounds like the Connect token the operator uses is invalid. 🤔

Navigation Menu

Repository navigation

panic: runtime error: invalid memory address or nil pointer dereference #180

Your environment

What happened?

What did you expect to happen?

Steps to reproduce

Notes & Logs

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

panic: runtime error: invalid memory address or nil pointer dereference #180

Description

Your environment

What happened?

What did you expect to happen?

Steps to reproduce

Notes & Logs

Activity

lhriley commented on Dec 5, 2023

edif2008 commented on Dec 18, 2023

edif2008 commented on Jan 2, 2024

lhriley commented on Jan 2, 2024

rstreics-lb commented on Jan 30, 2024

edif2008 commented on Jan 30, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions