Skip to content

V1Pod no longer pickleable with some logging configs #1304

Closed
Closed
V1Pod no longer pickleable with some logging configs#1304
Labels
kind/bugCategorizes issue or PR as related to a bug.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.

Description

@jedcunningham
jedcunningham
opened on Oct 30, 2020

What happened (please include outputs or screenshots):

Starting with 12.0.0, V1Pod now is no longer pickleable depending on how logging is configured.

What you expected to happen:

V1Pod is pickleable.

How to reproduce it (as minimally and precisely as possible):

>>> from kubernetes.client import models as k8s
>>> import logging
>>> import pickle
>>> logging.basicConfig()
>>> p = k8s.V1Pod()
>>> pickle.dumps(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't pickle _thread.RLock objects

Anything else we need to know?:

I believe it is unrelated, but this was initially found in apache/airflow, as it pickles V1Pods.

Environment:

  • Python version: 3.6.12
  • Python client version: 12.0.0

Activity

  1. jedcunningham
    added
    kind/bugCategorizes issue or PR as related to a bug.
    on Oct 30, 2020

  2. ashb commented on Oct 31, 2020

    @ashb
    ashb
    on Oct 31, 2020

    This behaviour came about when b4d11b0 introduced local_vars_configuration, which contains two logger objects.

  3. roycaihw commented on Nov 9, 2020

    @roycaihw
    roycaihw
    on Nov 9, 2020
    Member

    /assign @palnabarun

  4. k8s-ci-robot
    assigned
    palnabarun
    on Nov 9, 2020

  5. palnabarun commented on Nov 14, 2020

    @palnabarun
    palnabarun
    on Nov 14, 2020
    Member

    It looks to me that the change in OpenAPI Generator brought in regression to older Python versions. I can reproduce the above error with Python 3.6.10 but not with Python 3.7.7 or Python 3.8.6. IMHO, it needs further investigation as to why this may be happening.

    $ cat logger_pickle.py 
from kubernetes.client import models as k8s
from kubernetes import __version__

import pickle
import logging

logging.basicConfig()

print("Kubernetes Client version: {}".format(__version__))
print("Pickled V1Pod: {}".format(pickle.dumps(k8s.V1Pod())))

    Python 3.6.10

    (py3.6) $ ipython -i logging_pickle.py
Python 3.6.10 (default, May  7 2020, 12:54:46) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.16.1 -- An enhanced Interactive Python. Type '?' for help.
Kubernetes Client version: 12.0.1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/c/kubernetes-client/python/tmp/logger_pickle.py in <module>
      8 
      9 print("Kubernetes Client version: {}".format(__version__))
---> 10 print("Pickled V1Pod: {}".format(pickle.dumps(k8s.V1Pod())))
     11 

TypeError: can't pickle _thread.RLock objects

    Python 3.7.7

    (py3.7) $ ipython -i logger_pickle.py
Python 3.7.7 (default, May  8 2020, 11:39:39) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.
Kubernetes Client version: 12.0.1
Pickled V1Pod: b'\x80\x03ckubernetes.client.models.v1_pod\nV1Pod\nq\x00)\x81q\x.................'

    Python 3.8.6

    (py3.8) $ ipython -i logger_pickle.py
Python 3.8.6 (default, Sep 30 2020, 04:00:38) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.
Kubernetes Client version: 12.0.1
Pickled V1Pod: b'\x80\x04\x95\xa9\x03\x00\x00\x00\x00\x00\x00\x8c\x1fkubernetes.client.models.v1_pod\......'

    @ashb

    This behaviour came about when b4d11b0 introduced local_vars_configuration, which contains two logger objects.

    I am not very sure if two logger objects are creating the issue since the bug does not surface in Python 3.7 and Python 3.8.

    @jedcunningham @ashb -- is it a hard requirement to use Python 3.6 considering Python 3.6 went into security patch only part of it's lifecycle? In that case, the solution needs to be found in the OpenAPI Generator for this regression.

  6. ashb commented on Nov 14, 2020

    @ashb
    ashb
    on Nov 14, 2020

    Afraid so - as long as 3.6 is not EoL Airflow supports it.

  7. mik-laj commented on Nov 14, 2020

    @mik-laj
    mik-laj
    on Nov 14, 2020

    Python 3.6.8

    $ python --version
Python 3.6.8
$ python -c "import logging; logging.basicConfig(); import pickle; pickle.dumps(logging.getLogger())"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: can't pickle _thread.RLock objects

    Python 3.7.2

    python -c "import logging; logging.basicConfig(); import pickle; pickle.dumps(logging.getLogger())"

    (no problem)

  8. ashb commented on Nov 14, 2020

    @ashb
    ashb
    on Nov 14, 2020

    This could be addressed by adding __getstate__ (I think that's what it's called) to control what gets pickled or not.

    I think not logging the logger objects make sense.

    I'll try and find a time to attempt a pr for this.

  9. fejta-bot commented on Mar 10, 2021

    @fejta-bot
    fejta-bot
    on Mar 10, 2021

    Issues go stale after 90d of inactivity.
    Mark the issue as fresh with /remove-lifecycle stale.
    Stale issues rot after an additional 30d of inactivity and eventually close.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-contributor-experience at kubernetes/community.
    /lifecycle stale

  10. k8s-ci-robot
    added
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Mar 10, 2021

  11. ashb commented on Mar 10, 2021

    @ashb
    ashb
    on Mar 10, 2021

    /remove-lifecycle stale

  12. k8s-ci-robot
    removed
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Mar 10, 2021

  13. fejta-bot commented on Jun 8, 2021

    @fejta-bot
    fejta-bot
    on Jun 8, 2021

    Issues go stale after 90d of inactivity.
    Mark the issue as fresh with /remove-lifecycle stale.
    Stale issues rot after an additional 30d of inactivity and eventually close.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-contributor-experience at kubernetes/community.
    /lifecycle stale

  14. k8s-ci-robot
    added
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Jun 8, 2021

  15. ashb commented on Jun 8, 2021

    @ashb
    ashb
    on Jun 8, 2021

    not stale

  16. fejta-bot commented on Jul 8, 2021

    @fejta-bot
    fejta-bot
    on Jul 8, 2021

    Stale issues rot after 30d of inactivity.
    Mark the issue as fresh with /remove-lifecycle rotten.
    Rotten issues close after an additional 30d of inactivity.

    If this issue is safe to close now please do so with /close.

    Send feedback to sig-contributor-experience at kubernetes/community.
    /lifecycle rotten

  17. k8s-ci-robot
    added
    lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.
    and removed
    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
    on Jul 8, 2021

  18. ashb commented on Jul 9, 2021

    @ashb
    ashb
    on Jul 9, 2021

    /remove-lifecycle rotten

    You abusive bot. Auto closing issues is terrible.

  19. k8s-ci-robot
    removed
    lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.
    on Jul 9, 2021

  20. palnabarun commented on Aug 18, 2021

    @palnabarun
    palnabarun
    on Aug 18, 2021
    Member

    @ashb -- please give the bot some slack. :) It is trying to help to clear out inactive issues.

    /lifecycle frozen

  21. k8s-ci-robot
    added
    lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.
    on Aug 18, 2021

  22. palnabarun commented on Aug 18, 2021

    @palnabarun
    palnabarun
    on Aug 18, 2021
    Member

    /unassign

    (since I'm not working on this and @ashb mentioned that they will be looking at a patch)

  23. k8s-ci-robot
    unassigned
    palnabarun
    on Aug 18, 2021

  24. ashb commented on Oct 7, 2021

    @ashb
    ashb
    on Oct 7, 2021

    I'm not quite sure how to fix this inside this module, particular as the class I might want to edit says "Do Not edit -- autogenreted".

    The fix I have added to Airflow to "workaround" this is to backport the pickle support to Logger objects from 3.7 via the copyreg stdlib module. See apache/airflow#18797

  25. jedcunningham commented on Jan 6, 2022

    @jedcunningham
    jedcunningham
    on Jan 6, 2022
    Author

    And 3.6 is now EOL.

  26. jedcunningham
    closed this as completedon Jan 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions