Repository navigation
userinfo.email scope is needed to work with rbac #54
Description
Activity
- added a commit that references this issue
on Mar 28, 2018 - added a commit that references this issue
on Mar 28, 2018 I believe we are experiencing the exact issue that you are describing when using the Kubernetes Python client. We are finding that when the Python client refreshes the access token in
~/.kube/configwe can't make requests to the Kubernetes API server with a service account that is normally authenticated with RBAC. Though when we usekubectlthe token refresh is fine.Using a
~/.kube/configwith theuserssection of the format:users: - name: application-default-credentials user: auth-provider: config: access-token: <REDACTED_ACCESS_TOKEN> expiry: '2018-11-11T11:11:11Z' name: gcpAnd a GCP service account (not a user service account, but a server service account) with the format:
{ "type": "service_account", "project_id": "example-project-id", "private_key_id": "<REDACTED_PRIVATE_KEY_ID>", "private_key": "<REDACTED_PRIVATE_KEY>", "client_email": "<REDACTED_CLIENT_EMAIL>", "client_id": "REDACTED_CLIENT_ID", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://accounts.google.com/o/oauth2/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/<REDACTED_CLIENT_EMAIL>" }I found that if I used
kubectlwith this configuration and service account to refresh the access token then it would be valid and I'm able to make requests to the Kubernetes API server. However, if
the token is refreshed by the Python client, then the token would be technically valid, but I wouldn't be authenticated to make requests. Instead I would get back an error like:Forbidden: pods is forbidden: User "<REDACTED_CLIENT_ID>" cannot watch pods in the namespace "<REDACTED_NAMESPACE>": Required "container.pods.list" permission.Where the
REDACTED_CLIENT_IDmatches the ID from our service account, which corresponds with what you were saying:But if the userinfo.email scope isn't included that APIServer ends up using the numeric id of service accounts which won't work if RBAC rules are written in terms of
the emails.Using the command
curl https://www.googleapis.com/oauth2/v2/tokeninfo?access_token=<access_token>to get information about the tokens that are generated:`kubectl` { "issued_to": "<REDACTED_ID>", "audience": "<REDACTED_ID>", "expires_in": 2534, "access_type": "offline" "scope": "https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/userinfo.email", "email": "<REDACTED_EMAIL>", "verified_email": true, } Python client: { "issued_to": "<REDACTED_ID>", "audience": "<REDACTED_ID>", "expires_in": 3230, "access_type": "offline", "scope": "https://www.googleapis.com/auth/cloud-platform" }So the token generated by the Python client is technically valid, however it isn't signed by an email and therefore can't be used by the Kubernetes API server to authenticate using RBAC.
I've written a workaround where I wrap the call to
google.auth.defaultso that it appends"https://www.googleapis.com/auth/userinfo.email"to whatever scope get passed in. That seems to have solved the issue for us, however I feel like it would make sense to change the call to google.auth.default so that it passes in this scope like:def _refresh_credentials(): credentials, project_id = google.auth.default(scopes=[ 'https://www.googleapis.com/auth/cloud-platform', 'https://www.googleapis.com/auth/userinfo.email' ]) request = google.auth.transport.requests.Request() credentials.refresh(request) return credentials
I can put up a PR if there's interest, from my experience identifying this as the issue when you're seeing authentication issue can be really obscure. I've also seen a couple other issues of people complaining that the refresh token isn't refreshing so this could potentially help others as well.
I figure I'll just open a PR to get things going if that's alright.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with/remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.If this issue is safe to close now please do so with
/close.Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle staleStale issues rot after 30d of inactivity.
Mark the issue as fresh with/remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.If this issue is safe to close now please do so with
/close.Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten/close
@axelsteingrimsson: Closing this issue.
Details
In response to this:
/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
KubeConfigLoader._refresh_credentials sets the scope to cloud-platform.
I think userinfo.email scope is needed as well.
See kubernetes/kubernetes#58141 for a similar issue with kubectl and a good explanation.
I think we need the userinfo.email scope because RBAC rules can be expressed in terms of the email of service accounts. But if the userinfo.email scope isn't included that APIServer ends up using the numeric id of service accounts which won't work if RBAC rules are written in terms of
the emails.
I haven't confirmed for myself this is an issue (I'm working through a variety of issues with kubectl/kubeconfig/client libs) so I could be wrong.