What happened (please include outputs or screenshots):
Long-running applications using a kubeconfig with certificate-authority-data encounter errors like:
Max retries exceeded with url: /api/v1/pods?fieldSelector=spec.nodeName%3Dip-172-18-66-203.ec2.internal (Caused by SSLError(FileNotFoundError(2, 'No such file or directory'),))
What you expected to happen:
Client should not expect tempfiles to live indefinitely. It is extremely common for servers to reap tempfiles.
Anything else we need to know?:
If you create a client you can easily see that it is using a temp file that cannot go away for the duration of the client:
>>> import kubernetes.config
>>> kubernetes.config.load_kube_config()
>>> api_client = kubernetes.client.ApiClient()
>>> api_client.rest_client.pool_manager.connection_pool_kw["ca_certs"]
'/tmp/tmpqkht2v2g'
You can reproduce the issue by deleting that temp file and attempting to make a request.
Code is here:
|
def _create_temp_file_with_content(content, temp_file_path=None): |
|
if len(_temp_files) == 0: |
|
atexit.register(_cleanup_temp_files) |
|
# Because we may change context several times, try to remember files we |
|
# created and reuse them at a small memory cost. |
|
content_key = str(content) |
|
if content_key in _temp_files: |
|
return _temp_files[content_key] |
|
if temp_file_path and not os.path.isdir(temp_file_path): |
|
os.makedirs(name=temp_file_path) |
|
fd, name = tempfile.mkstemp(dir=temp_file_path) |
|
os.close(fd) |
|
_temp_files[content_key] = name |
|
with open(name, 'wb') as fd: |
|
fd.write(content.encode() if isinstance(content, str) else content) |
|
return name |
In order for this to work reliably for long-running processes on standard linux systems, the temp file really needs to be created for each request rather than a single time at startup.
That said, on linux systems, a potential hack would be to use /proc/self/fd/<fileno> instead of the temfile path since that would share the lifecycle of the process.
Environment:
- Python version (
python --version): 3.6
- Python client version (
pip list | grep kubernetes): 21.7.0
What happened (please include outputs or screenshots):
Long-running applications using a kubeconfig with
certificate-authority-dataencounter errors like:What you expected to happen:
Client should not expect tempfiles to live indefinitely. It is extremely common for servers to reap tempfiles.
Anything else we need to know?:
If you create a client you can easily see that it is using a temp file that cannot go away for the duration of the client:
You can reproduce the issue by deleting that temp file and attempting to make a request.
Code is here:
python/kubernetes/base/config/kube_config.py
Lines 63 to 78 in 1271465
In order for this to work reliably for long-running processes on standard linux systems, the temp file really needs to be created for each request rather than a single time at startup.
That said, on linux systems, a potential hack would be to use
/proc/self/fd/<fileno>instead of the temfile path since that would share the lifecycle of the process.Environment:
python --version): 3.6pip list | grep kubernetes): 21.7.0