Repository navigation
Does Poetry support recursive extras? #3369
Description
- I have searched the issues of this repo and believe that this is not a duplicate.
- I have searched the documentation and believe that my question is not covered.
Issue
Does Poetry support recursive (for lack of a better term, it's early) extras best exemplified by this pyproject.toml snippet?
[tool.poetry.dependencies]
attrs = "~19.3"
pandas = "~1.1"
# optional
quandl = { version = "~3.5.2", optional = true }
matplotlib = { version = "~3.3.0", optional = true }
networkx = { version = "~2.4.0", optional = true }
scikit-learn = { version = "~0.22.0", optional = true }
[tool.poetry.extras]
quandl = ["quandl"]
viz = ["matplotlib", "networkx"]
ml = ["viz", "scikit-learn"]
all = ["ml", "quandl"]
Activity
sinoroc commentedon Nov 16, 2020
I don't think so. Did you try it? What was the result?
alexifm commentedon Nov 16, 2020
Did not work. Can you suggest a location in the codebase to look for where I could potentially add this contribution if it's considered desirable? (I have yet to reliably figure out the poetry codebase)
alexifm commentedon Nov 16, 2020
Seems like it could be here?
poetry/poetry/utils/extras.py
Line 10 in 8b8b97a
Specifically modify this list creation to look up packages if the name is an extra:
poetry/poetry/utils/extras.py
Line 30 in 8b8b97a
n8sty commentedon Nov 16, 2020
This might be too pedantic of a concern but you should make sure to guard against situations where an extra and a package could have the same name. In your example file, whatever algorithm you implement would need to know that the
quandlreferenced in theallsection is different than the "quandl" referenced in thequandlextras line.I'm not sure the increased ease of nesting optional dependencies is worth the potential brittleness of solving for the above.
alexifm commentedon Nov 16, 2020
@n8sty Yes, that is something I avoid but unfortunately didn't pay close attention to in that example.
As for an implementation, I think the simplest thing would be to just fail and have the error suggest picking a different name that doesn't create a conflict.
Another thing is that it seems like
poetry-coreis where package building is handled and it's much less clear to me how it resolves the packages in an extras definition than this function I identified earlier.ETA: Looks like this is where
poetry-corecould handle the recursive extras:https://github.com/python-poetry/poetry-core/blob/71f35e82188b85ea690735f8ee554b9f5a74486e/poetry/core/factory.py#L96-L109
sinoroc commentedon Nov 17, 2020
Personally I am not convinced there is a need for such a feature with such a notation. Seems to be adding more confusion. You may want to give the UX a second look
See if it would make sense in the context of the dependencies groups feature (planned for v1.2).
If I were to change extras in order to avoid repetition for dependencies that are in multiple extras (in a way this is what you are trying to solve, right?), this is what I would aim for, but I think the dependencies groups feature already covers this (I have not read the full discussion on this, so I might be wrong):
finswimmer commentedon Nov 18, 2020
Is this something that is defined in any PEP? If so,
poetryshould support it. If not, then probably the corresponding wheel/sdist will not be interpreted correct by other tools andpoetryshould not support it as well.sinoroc commentedon Nov 19, 2020
@finswimmer I think they want enhancements to the notation in
pyproject.toml, that would avoid some repetition of dependencies in that file. And poetry should take care to expand the notation to make it standard-conform in the distribution's metadata.sinoroc commentedon Nov 25, 2020
For comparison, this is how it can be done with setuptools
setup.cfg: https://stackoverflow.com/a/61610341/alexifm commentedon Dec 2, 2020
@sinoroc I'm okay with dropping this in favor of the groups approach mentioned for the upcoming release. Not worth it to support two approaches, even if it's not that far off what setuptools allows.
smheidrich commentedon Mar 11, 2023
Probably worth pointing out that since the last comment here, recent versions of setuptools+Pip have started supporting recursive extras specified without any hacks (i.e. not like in that StackOverflow answer), by simply having dependencies on the current package itself with
[extra1,extra2,...]appended.As far as I can tell that wouldn't work with how extras and dependencies are currently specified in Poetry, as dependencies can only be listed once and so there is no way to have e.g. extra A depend on
currentpackage[X, Y]and extra B depend oncurrentpackage[Z]. So that's a shame.And by the way, I don't see how groups solve this issue at all - they are only meant for development of the package itself, aren't they? Whereas extras are user-facing. Reopen?
github-actions commentedon Feb 29, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.