Skip to content

The documentation of kubectl wait is a bit miss-leading  #754

Closed

Description

@eranreshef

The --timeout option for kubectl wait says:

"The length of time to wait before giving up" .

I personally understand this as the timeout for the entire command but after experimenting with it a bit I realised that the value of this option is per resource. For example, if I am waiting for a set of 10 pods to be Ready and used --timeout=60s, I might end up waiting for 10 minutes before the command exits, and not just 1 minute as I assumed.

So as I see it, there are 2 possible solutions here:

  1. Improve the documentation.
  2. Change the implementation so that the --timeout value will be for the entire command duration (preferable solution in my opinion).

Activity

  1. zhouya0 commented on Nov 7, 2019

    @zhouya0

    Actually the kubectl wait will do the watch request for every single resource and do the request one by one:

    GET https://10.6.192.3:6443/api/v1/namespaces/default/pods?fieldSelector=metadata.name%3Dmy-busybox&resourceVersion=5594455&watch=true
    

    But watch request is actually a unlimited connection so every request has a timeout(default is 30s in wait).

    So the only way to do your second solution is average time for the timeout, but seems not very good to do that.

  2. eranreshef commented on Nov 7, 2019

    @eranreshef
    Author

    I implemented sort of a wrapper for internal use with the following logic:

    1. run kubectl wait <resources> --timeout=0
    2. parse the output from step 1 to get a list of unready_pods
    3. while (command_elapsed_time < command_timeout and len(unready_pods) > 0)
      a. run kubectl wait <unready_pods> --timeout=0
      b. parse the output from step 3.a to get a list of unready_pods
    4. if len(unready_pods) > 0
      a. raise a command_timeout exception.

    IMHO, this logic fits better to the use cases kubernetes users encounter on the day to day work but again, my opinion only. I'm not a go developer so I can't submit a PR with this logic but I would definitely like to hear more opinions about this logic.

  3. fejta-bot commented on Feb 5, 2020

    @fejta-bot

    Issues go stale after 90d of inactivity.
    Mark the issue as fresh with /remove-lifecycle stale.
    Stale issues rot after an additional 30d of inactivity and eventually close.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
    /lifecycle stale

  4. added
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Feb 5, 2020
  5. fejta-bot commented on Mar 6, 2020

    @fejta-bot

    Stale issues rot after 30d of inactivity.
    Mark the issue as fresh with /remove-lifecycle rotten.
    Rotten issues close after an additional 30d of inactivity.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
    /lifecycle rotten

  6. added
    lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.
    and removed
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Mar 6, 2020
  7. pjh commented on Mar 31, 2020

    @pjh

    /remove-lifecycle rotten

    I just experienced exactly the issue described in the OP, and found it very confusing. I noticed these log messages from my e2e test:

    + kubectl wait --for=condition=ready pod -l prepull-test-images=e2e --timeout 30m
    timed out waiting for the condition on pods/prepull-test-containers-gxjhh
    timed out waiting for the condition on pods/prepull-test-containers-mqf4j
    timed out waiting for the condition on pods/prepull-test-containers-r96tp
    + kubectl get pods -o wide
    NAME                            READY   STATUS             RESTARTS   AGE   IP          NODE                                           NOMINATED NODE   READINESS GATES
    prepull-test-containers-gxjhh   14/17   CrashLoopBackOff   85         92m   10.64.2.3   e2e-726b009feb-d872a-windows-node-group-ltb5   <none>           <none>
    prepull-test-containers-mqf4j   14/17   CrashLoopBackOff   84         92m   10.64.1.3   e2e-726b009feb-d872a-windows-node-group-0qdq   <none>           <none>
    prepull-test-containers-r96tp   14/17   CrashLoopBackOff   82         92m   10.64.3.3   e2e-726b009feb-d872a-windows-node-group-lc19   <none>           <none>
    

    The age of the pods is 90 minutes rather than the 30 minutes I expected, because kubectl wait is applying the timeout to each selected pod sequentially. This is very unintuitive to me. If this default behavior can't be changed, perhaps a flag can be added to change the behavior to enforce the timeout across all selected pods/conditions.

  8. removed
    lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.
    on Mar 31, 2020
  9. brianpursley commented on Apr 29, 2020

    @brianpursley
    Member

    /kind bug

  10. seans3 commented on Jul 22, 2020

    @seans3
    Contributor

    /area kubectl
    /sig cli

  11. 5 remaining items

  12. fejta-bot commented on Dec 27, 2020

    @fejta-bot

    Issues go stale after 90d of inactivity.
    Mark the issue as fresh with /remove-lifecycle stale.
    Stale issues rot after an additional 30d of inactivity and eventually close.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
    /lifecycle stale

  13. added
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Dec 27, 2020
  14. eranreshef commented on Dec 27, 2020

    @eranreshef
    Author

    /remove-lifecycle stale

  15. removed
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Dec 27, 2020
  16. fejta-bot commented on Mar 27, 2021

    @fejta-bot

    Issues go stale after 90d of inactivity.
    Mark the issue as fresh with /remove-lifecycle stale.
    Stale issues rot after an additional 30d of inactivity and eventually close.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-contributor-experience at kubernetes/community.
    /lifecycle stale

  17. added
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Mar 27, 2021
  18. fejta-bot commented on Apr 26, 2021

    @fejta-bot

    Stale issues rot after 30d of inactivity.
    Mark the issue as fresh with /remove-lifecycle rotten.
    Rotten issues close after an additional 30d of inactivity.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-contributor-experience at kubernetes/community.
    /lifecycle rotten

  19. added
    lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.
    and removed
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Apr 26, 2021
  20. fejta-bot commented on May 26, 2021

    @fejta-bot

    Rotten issues close after 30d of inactivity.
    Reopen the issue with /reopen.
    Mark the issue as fresh with /remove-lifecycle rotten.

    Send feedback to sig-contributor-experience at kubernetes/community.
    /close

  21. k8s-ci-robot commented on May 26, 2021

    @k8s-ci-robot
    Contributor

    @fejta-bot: Closing this issue.

    Details

    In response to this:

    Rotten issues close after 30d of inactivity.
    Reopen the issue with /reopen.
    Mark the issue as fresh with /remove-lifecycle rotten.

    Send feedback to sig-contributor-experience at kubernetes/community.
    /close

    Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

  22. mgabeler-lee-6rs commented on May 25, 2022

    @mgabeler-lee-6rs

    /reopen

    This seemed to be a confirmed bug, and is definitely still present, quite confusing, and makes it basically impossible to use kubectl wait for more than one resource in any useful fashion

  23. k8s-ci-robot commented on May 25, 2022

    @k8s-ci-robot
    Contributor

    @mgabeler-lee-6rs: You can't reopen an issue/PR unless you authored it or you are a collaborator.

    Details

    In response to this:

    /reopen

    This seemed to be a confirmed bug, and is definitely still present, quite confusing, and makes it basically impossible to use kubectl wait for more than one resource in any useful fashion

    Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

  24. mgabeler-lee-6rs commented on May 25, 2022

    @mgabeler-lee-6rs

    ok, fine, #1219

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/kubectlkind/bugCategorizes issue or PR as related to a bug.lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.priority/backlogHigher priority than priority/awaiting-more-evidence.sig/cliCategorizes an issue or PR as relevant to SIG CLI.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions