exec command is parsing output if output looks like json

What happened (please include outputs or screenshots):

Working on creating a scale test where the test orchestrator will exec in each pod and run a curl command to verify the pod is in the expected state. The output of the curl is JSON.

The problem seems to be that the stream client will parse the json as a python dictionary if the output is only json, but return it to the caller as a stringified dictionary. Expecting json, json.loads is failing.

To verify I ran

In [5]: cmd=["echo", '{"example":"json", "with":null, "and":true}']

In [6]: kstream.stream(api_instance.connect_get_namespaced_pod_exec,
   ...:                   name=pod,
   ...:                   namespace=ns,
   ...:                   container=container,
   ...:                   command=cmd,
   ...:                   stderr=True,
   ...:                   stdin=False,
   ...:                   tty=False,
   ...:                   stdout=True)
Out[6]: "{'example': 'json', 'with': None, 'and': True}"

In [7]: cmd=["echo", 'but not actually json {"example":"json", "with":null, "and":true}']

In [8]: kstream.stream(api_instance.connect_get_namespaced_pod_exec,
   ...:                   name=pod,
   ...:                   namespace=ns,
   ...:                   container=container,
   ...:                   command=cmd,
   ...:                   stderr=True,
   ...:                   stdin=False,
   ...:                   tty=False,
   ...:                   stdout=True)
Out[8]: 'but not actually json {"example":"json", "with":null, "and":true}\n'

What you expected to happen:

I really did not expect the response to be different depending on the output of the command being run.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

We are able to get around the issue by adding _preload_content=False to the stream function call and handling waiting for the response and reading stdout/stderr ourselves.

I wasn't sure where the rogue json parsing is happening, maybe in WSResponse class?

Environment:

Kubernetes version (kubectl version):

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-19T15:41:24Z", GoVersion:"go1.12.13", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.8-eks-b8860f", GitCommit:"b8860f6c40640897e52c143f1b9f011a503d6e46", GitTreeState:"clean", BuildDate:"2019-11-25T00:55:38Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

OS (e.g., MacOS 10.13.6):
Ubuntu 19.10
Python version (python --version)

python 3.7.5

Python client version (pip list | grep kubernetes)

kubernetes==10.0.1

@jiahuif

cc @jiahuif

@tkinz27

The json parsing is done by

python/kubernetes/client/api_client.py

Line 242 in ccd3ce4

data = json.loads(response.data)

Essentially what it does is

data = '{"example":"json", "with":null, "and":true}'
data = json.loads(data)
data = str(data)

and the result becomes a string representation of the dict "{'example': 'json', 'with': None, 'and': True}"

This looks like a bug in the upstream code generator. The generated client should not try parsing the response when the payload isn't JSON and the response is of string type.

@tkinz27 could you file an issue in https://github.com/OpenAPITools/openapi-generator?

      "get": {
        "consumes": [
          "*/*"
        ],
        "description": "connect GET requests to exec of Pod",
        "operationId": "connectCoreV1GetNamespacedPodExec",
        "produces": [
          "*/*"
        ],
        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "type": "string"
            }
          },
          "401": {
            "description": "Unauthorized"
          }
        },
        "schemes": [
          "https"
        ],

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

/remove-lifecycle rotten

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

/remove-lifecycle stale

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Fix streaming results from k8s API

No fix or work-around 4 and a half years later?

Fix streaming results from k8s API

Navigation Menu

Repository navigation

exec command is parsing output if output looks like json #1032

15 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

exec command is parsing output if output looks like json #1032

Description

Activity

roycaihw commented on Dec 17, 2019

roycaihw commented on Dec 17, 2019

fejta-bot commented on Mar 16, 2020

mitar commented on Mar 16, 2020

fejta-bot commented on Jun 14, 2020

mitar commented on Jun 14, 2020

fejta-bot commented on Sep 13, 2020

mitar commented on Sep 13, 2020

15 remaining items

mitar commented on Oct 18, 2021

k8s-triage-robot commented on Jan 16, 2022

mitar commented on Jan 16, 2022

k8s-triage-robot commented on Apr 17, 2022

et304383 commented on Jul 10, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions