Skip to content

Poetry doesn't find a compatible version of python, even though one is available #7772

New issue
New issue
Closed
#7771
Closed
Poetry doesn't find a compatible version of python, even though one is available#7772
#7771
Labels
kind/bugSomething isn't working as expectedstatus/triageThis issue needs to be triaged

Description

@tungol
tungol
opened on Apr 6, 2023
  • Poetry version: Introduced in poetry 1.4.0
  • Python version: Python 3.7, 3.10, 3.11 are all involved
  • OS version and name: MacOS and Debian
  • pyproject.toml:
  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Running with -vvv did not produce additional output.

Issue

In my project, I have the python dependency python = "~=3.10". I happen to have python 3.11 available on my development laptop, so I use that there. As part of our build pipeline, the pyproject.toml and poetry.lock files are copied into a Debian docker image which has a default python of 3.7, and has 3.10 but not 3.11 available. Prior to Poetry 1.4.0 this worked fine - Poetry would note that 3.7 wasn't supported by the project and then discover and use 3.10 automatically:

#24 0.801 The currently activated Python version 3.7.4 is not supported by the project (~=3.10).
#24 0.801 Trying to find and use a compatible version. 
#24 0.833 Using python3.10 (3.10.8)

On Poetry 1.4.0 and higher, that doesn't happen. Instead, I get an error because 3.11 isn't available:

#23 0.844 The currently activated Python version 3.7.4 is not supported by the project (~=3.10).
#23 0.844 Trying to find and use a compatible version. 
#23 0.885 
#23 0.885 [Errno 2] No such file or directory: 'python3.11': 'python3.11'

I believe this behavior was introduced by #7221 , although I haven't read through the associated MR in detail yet.

Activity

  1. tungol
    added
    kind/bugSomething isn't working as expected
    status/triageThis issue needs to be triaged
    on Apr 6, 2023

  2. dimbleby commented on Apr 6, 2023

    @dimbleby
    dimbleby
    on Apr 6, 2023
    Contributor

    poetry isn't simply inventing python3.11 out of thin air, there must be something in your docker image that makes it think that's a good idea.

    perhaps you have copied an unwanted envs.toml into the container?

    Anyway, if you can't figure it out yourself and are hoping that someone else will help - please provide a way to reproduce the issue.

  3. tungol commented on Apr 7, 2023

    @tungol
    tungol
    on Apr 7, 2023
    Author

    Hm, I made the assumption that 3.11 was recorded in the lock file somewhere, but I just made a minimal reproduction and that's definitely not the case so I'm really not sure where it's coming from. Here's what I have to reproduce:

    pyproject.toml

    [tool.poetry]
name = "test_case"
version = "1.0"
description = ""
authors = []
	
[tool.poetry.dependencies]
python = "~=3.10"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

    Dockerfile

    FROM ubuntu:focal

RUN apt-get update
RUN apt-get install -y --no-install-recommends software-properties-common

RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt update
RUN apt install -y --no-install-recommends python3.9 python3.9-venv python3.10 python3.10-venv

RUN python3.9 -m ensurepip

ARG POETRY_VERSION=1.4.2

ENV PATH=/root/.local/bin:$PATH
RUN --mount=type=cache,target=/root/.cache/pip pip3.9 install poetry==$POETRY_VERSION
COPY pyproject.toml ./
RUN poetry -vvv run pip list

    'pip list' there isn't special. I was originally getting it with a different, more useful pip command there.

    Running docker build . gives this:

     => ERROR [stage-0 10/10] RUN poetry -vvv run pip list                                                                                                                                                      1.5s
------                                                                                                                                                                                                           
 > [stage-0 10/10] RUN poetry -vvv run pip list:                                                                                                                                                                 
#14 0.431 /usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.15) or chardet (3.0.4) doesn't match a supported version!                                             
#14 0.431   warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "                                                                                                                              
#14 0.642 The currently activated Python version 3.9.16 is not supported by the project (~=3.10).                                                                                                                
#14 0.642 Trying to find and use a compatible version. 
#14 0.642 Trying python3
#14 0.657 Trying python3.11
#14 0.763 
#14 0.763   Stack trace:
#14 0.763 
#14 0.763   12  /usr/local/lib/python3.9/dist-packages/cleo/application.py:327 in run
#14 0.798        325│ 
#14 0.798        326│             try:
#14 0.799      → 327│                 exit_code = self._run(io)
#14 0.799        328│             except BrokenPipeError:
#14 0.799        329│                 # If we are piped to another process, it may close early and send a
#14 0.799 
#14 0.800   11  /usr/local/lib/python3.9/dist-packages/poetry/console/application.py:190 in _run
#14 0.802        188│         self._load_plugins(io)
#14 0.802        189│ 
#14 0.802      → 190│         exit_code: int = super()._run(io)
#14 0.802        191│         return exit_code
#14 0.802        192│ 
#14 0.802 
#14 0.803   10  /usr/local/lib/python3.9/dist-packages/cleo/application.py:431 in _run
#14 0.837        429│             io.input.interactive(interactive)
#14 0.837        430│ 
#14 0.837      → 431│         exit_code = self._run_command(command, io)
#14 0.837        432│         self._running_command = None
#14 0.837        433│ 
#14 0.837 
#14 0.837    9  /usr/local/lib/python3.9/dist-packages/cleo/application.py:473 in _run_command
#14 0.869        471│ 
#14 0.869        472│         if error is not None:
#14 0.869      → 473│             raise error
#14 0.869        474│ 
#14 0.869        475│         return terminate_event.exit_code
#14 0.869 
#14 0.869    8  /usr/local/lib/python3.9/dist-packages/cleo/application.py:454 in _run_command
#14 0.902        452│ 
#14 0.902        453│         try:
#14 0.902      → 454│             self._event_dispatcher.dispatch(command_event, COMMAND)
#14 0.902        455│ 
#14 0.902        456│             if command_event.command_should_run():
#14 0.902 
#14 0.902    7  /usr/local/lib/python3.9/dist-packages/cleo/events/event_dispatcher.py:26 in dispatch
#14 0.908         24│ 
#14 0.908         25│         if listeners:
#14 0.908      →  26│             self._do_dispatch(listeners, event_name, event)
#14 0.908         27│ 
#14 0.908         28│         return event
#14 0.908 
#14 0.908    6  /usr/local/lib/python3.9/dist-packages/cleo/events/event_dispatcher.py:89 in _do_dispatch
#14 0.914         87│                 break
#14 0.914         88│ 
#14 0.914      →  89│             listener(event, event_name, self)
#14 0.914         90│ 
#14 0.914         91│     def _sort_listeners(self, event_name: str) -> None:
#14 0.914 
#14 0.915    5  /usr/local/lib/python3.9/dist-packages/poetry/console/application.py:299 in configure_env
#14 0.934        297│ 
#14 0.934        298│         env_manager = EnvManager(poetry, io=io)
#14 0.934      → 299│         env = env_manager.create_venv()
#14 0.934        300│ 
#14 0.935        301│         if env.is_venv() and io.is_verbose():
#14 0.935 
#14 0.935    4  /usr/local/lib/python3.9/dist-packages/poetry/utils/env.py:977 in create_venv
#14 1.033         975│                 try:
#14 1.033         976│                     python_patch = decode(
#14 1.033      →  977│                         subprocess.check_output(
#14 1.033         978│                             [python, "-c", GET_PYTHON_VERSION_ONELINER],
#14 1.033         979│                             stderr=subprocess.STDOUT,
#14 1.033 
#14 1.034    3  /usr/lib/python3.9/subprocess.py:424 in check_output
#14 1.125         422│         kwargs['input'] = empty
#14 1.125         423│ 
#14 1.125      →  424│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
#14 1.125         425│                **kwargs).stdout
#14 1.125         426│ 
#14 1.125 
#14 1.125    2  /usr/lib/python3.9/subprocess.py:505 in run
#14 1.215         503│         kwargs['stderr'] = PIPE
#14 1.215         504│ 
#14 1.215      →  505│     with Popen(*popenargs, **kwargs) as process:
#14 1.215         506│         try:
#14 1.215         507│             stdout, stderr = process.communicate(input, timeout=timeout)
#14 1.216 
#14 1.216    1  /usr/lib/python3.9/subprocess.py:951 in __init__
#14 1.311         949│                             encoding=encoding, errors=errors)
#14 1.312         950│ 
#14 1.312      →  951│             self._execute_child(args, executable, preexec_fn, close_fds,
#14 1.312         952│                                 pass_fds, cwd, env,
#14 1.312         953│                                 startupinfo, creationflags, shell,
#14 1.312 
#14 1.312   FileNotFoundError
#14 1.312 
#14 1.313   [Errno 2] No such file or directory: 'python3.11'
#14 1.313 
#14 1.313   at /usr/lib/python3.9/subprocess.py:1821 in _execute_child
#14 1.406       1817│                     else:
#14 1.406       1818│                         err_filename = orig_executable
#14 1.406       1819│                     if errno_num != 0:
#14 1.406       1820│                         err_msg = os.strerror(errno_num)
#14 1.407     → 1821│                     raise child_exception_type(errno_num, err_msg, err_filename)
#14 1.407       1822│                 raise child_exception_type(err_msg)
#14 1.407       1823│ 
#14 1.407       1824│ 
#14 1.407       1825│         def _handle_exitstatus(self, sts,
------
executor failed running [/bin/sh -c poetry -vvv run pip list]: exit code: 1

    while running docker build --build-arg POETRY_VERSION=1.3.2 . completes successfully.

  4. tungol commented on Apr 7, 2023

    @tungol
    tungol
    on Apr 7, 2023
    Author

    The issue is avoided by adding the line RUN poetry env use python3.10, but it seems wrong that that's needed.

  5. radoering commented on Apr 8, 2023

    @radoering
    radoering
    on Apr 8, 2023
    Member

    So the issue is:

    >>> subprocess.check_output(["python3.11", "-c", "print('test')"])
FileNotFoundError: ...

    >>> subprocess.check_output("python3.11 -c print('test')", shell=True)
subprocess.CalledProcessError: ...

    And we are catching CalledProcessError but not FileNotFoundError.

  6. dimbleby commented on Apr 8, 2023

    @dimbleby
    dimbleby
    on Apr 8, 2023
    Contributor

    this is fixed in #7771 (by first doing shutil.which(), and only proceeding on finding something)

  7. radoering
    closed this as completedin #7771on Apr 8, 2023

  8. github-actions commented on Feb 29, 2024

    @github-actions
    github-actionsBot
    on Feb 29, 2024 – with GitHub Actions

    This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

  9. github-actions
    locked as resolved and limited conversation to collaborators on Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working as expectedstatus/triageThis issue needs to be triaged

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions