Skip to content

Poetry does not handle full Python version markers correctly #2480

Closed

Description

@cjolowicz

Poetry does not install dependencies into an environment if the dependency has a marker with a full three-digit Python version, even though the environment matches the marker.

For example, the pyproject.toml below is for a project requiring Python 3.6.1 and higher. It also uses the backport of dataclasses to Python 3.6.

(This is not an uncommon situation. Packages like pre-commit explicitly exclude 3.6.0 due to a broken NamedTuple implementation. If such packages are added as dependency, our package also needs to exclude it.)

[tool.poetry]
name = "example"
version = "0.1.0"
description = ""
authors = ["user <user@example.com>"]

[tool.poetry.dependencies]
python = "^3.6.1"
dataclasses = {version = "^0.7", python = ">= 3.6.1, < 3.7"}

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

If we create an environment with the latest Python 3.6, dataclasses is not installed:

$ poetry env use 3.6
Creating virtualenv example-TsH6FKJq-py3.6 in ~/Library/Caches/pypoetry/virtualenvs
Using virtualenv: ~/Library/Caches/pypoetry/virtualenvs/example-TsH6FKJq-py3.6

$ poetry run python -V
Python 3.6.10

$ poetry run pip freeze | grep dataclasses

$ poetry export --format=requirements.txt | grep dataclasses
dataclasses==0.7; python_version >= "3.6.1" and python_version < "3.7" \

Note that the generated requirements.txt is invalid. The python_version marker cannot contain the full three-digit version. Instead python_full_version should be used. See PEP 508.

Activity

  1. added
    kind/bugSomething isn't working as expected
    area/solverRelated to the dependency resolver
    status/triageThis issue needs to be triaged
    on Jun 2, 2020
  2. RauliRuohonen commented on Nov 4, 2020

    @RauliRuohonen

    [...] if the dependency has a marker with a full three-digit Python version [...]

    The python_version marker cannot contain the full three-digit version. Instead python_full_version should be used.

    This focuses on three-digit versions, but the requirements.txt issue at least can occur with two-digit versions as well:

    pyenv local 3.6.1
    poetry new foo
    cd foo
    perl -pe 's/python = "\^3.6"/python = "~3.6.1"/' -i pyproject.toml
    poetry add fsspec==0.8.4
    poetry export -f requirements.txt -o requirements.txt
    pip install -r requirements.txt
    

    says

    Ignoring fsspec: markers 'python_version > "3.6"' don't match your environment
    

    This is wrong, even though the requirement python_version > "3.6" has only a two-digit version number in it. The reason is the strict comparison: it basically says "the next version after 3.6 or newer", but "the next version" is different for python_version (3.7) and python_full_version (3.6.1). The latter is what is meant, but not what is said.

    The original requirement for fsspec 0.8.4 is Requires-Python: >3.6 in its METADATA file. Although this has two digits, it's basically the same thing as >=3.6.1: if you run pip install fsspec==0.8.4 on Python 3.6.0 it fails, but on 3.6.1 it works. Since >3.6 is basically the same requirement as >=3.6.1, it has the same problem: a python_version constraint cannot express it.

    Using python_full_version fixes this, too.

  3. dimbleby commented on May 29, 2022

    @dimbleby
    Contributor

    results of export today for the two cases described are

    dataclasses==0.7 ; python_full_version >= "3.6.1" and python_version < "3.7"
    

    and

    fsspec==0.8.4 ; python_full_version >= "3.6.1" and python_full_version < "3.7.0"
    

    which both use python_full_version as desired.

  4. github-actions commented on Mar 1, 2024

    @github-actions

    This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

  5. locked as resolved and limited conversation to collaborators on Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/solverRelated to the dependency resolverkind/bugSomething isn't working as expected

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions