Repository navigation
Intermittent 403 authentication issues with EKS #678
Description
Activity
Could this be related to expiration of tokens after a certain amount of time? Is there any under the hood refreshing of the token that happens after a failed request?
And update, still seeing this with 8.0.0.
- changed the title
[-]Intermittent authentication issues with EKS[/-][+]Intermittent 403 authentication issues with EKS[/+]on Dec 19, 2018 kubernetes-sigs/aws-iam-authenticator#157 seems related. Might be an issue with EKS and aws-iam-authenticator. It is correct that this happens with the usage of the role specifically not with the user that created the cluster itself.
Gah, I'm hitting this now too, after I upgraded python via
brew upgradeon my Mac. Was working fine till now. Everything seems to be correct:# Locally. $ aws sts get-caller-identity { "Account": "ACCOUNT_ID", "UserId": "MY_USER_ID", "Arn": "arn:aws:iam::ACCOUNT_ID:user/jeff.geerling" } # On the EKS cluster. $ kubectl describe configmap -n kube-system aws-auth ... Data ==== mapUsers: ---- - userarn: arn:aws:iam::ACCOUNT_ID:user/jeff.geerling groups: - system:mastersBut using Ansible with
kubernetes===7.0.0I get:fatal: [127.0.0.1]: FAILED! => changed=false error: 403 msg: |- Failed to retrieve requested object: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces \"any-namespace-here\" is forbidden: User \"system:anonymous\" cannot get namespaces in the namespace \"any-namespace-here\"","reason":"Forbidden","details":{"name":"any-namespace-here","kind":"namespaces"},"code":403} reason: Forbidden status: 403Oops... I realized 7.0.0 is old. Upgraded
kuberneteswith Pip tokubernetes==8.0.1and I'm back in business. Python environment--Issues go stale after 90d of inactivity.
Mark the issue as fresh with/remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.If this issue is safe to close now please do so with
/close.Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle staleStale issues rot after 30d of inactivity.
Mark the issue as fresh with/remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.If this issue is safe to close now please do so with
/close.Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten@fejta-bot: Closing this issue.
Details
In response to this:
Rotten issues close after 30d of inactivity.
Reopen the issue with/reopen.
Mark the issue as fresh with/remove-lifecycle rotten.Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/closeInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
I had a constant 403 and 401 error using this authentication script. I looked for others and found this https://github.com/peak-ai/eks-token and it worked really well for me.
I'm using the 8.0.0a1 version and I'm doing
with
kube_config_file-and this seems to pass almost always but every now and then I get an error from the python client
which seems to be getting triggered here in the exec_provider in python-base.
The main change that I could think of it being is that I'm using the
-rflag and passing in an access role to use with the authenticator command which I don't see a test for in the exec_provider. Even with the flag the command passes sometimes but fails at other times.This issue only occurs with the Python Client not with using
kubectlsubprocess calls.I'm using EKS with
aws-iam-authenticator.