Repository navigation
Install removes dependencies of non-installed extras #6229
Description
- [x ] I am on the latest Poetry version.
- [x ] I have searched the issues of this repo and believe that this is not a duplicate.
- [ x] If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- Debian bullseye:
- poetry 1.2.0rc1:
- Link of a Gist with the contents of your pyproject.toml file: pyproject.toml
Issue
I've been testing 1.2.0rc1 and it seems poetry is removing dependencies of non-installed extras. My understanding of #4336 and #4242 is that poetry should only ever remove packages during an install if they're superseded by the locked versions that need to be installed, or if the --sync option is used. A workaround is to rewrite the extras as optional groups, which seems to solve the problem. If nothing else, perhaps some documentation updates are needed to mark extras as deprecated?
In my pyproject.toml, setuptools is a derived dependency of gunicorn (an optional package grouped under the web extra).
Running poetry install --no-root -vvv removes setuptools:
#8 2.645 + poetry lock --check
#8 3.115 poetry.lock is consistent with pyproject.toml.
#8 3.159 + poetry install --no-root -vvv
#8 3.515 Using virtualenv: /code/.venv
#8 3.555 Project environment contains an empty path in sys_path, ignoring.
#8 3.591 Installing dependencies from lock file
#8 3.635
#8 3.635 Finding the necessary packages for the current system
#8 3.647
#8 3.647 Package operations: 0 installs, 0 updates, 1 removal, 2 skipped
#8 3.647
#8 3.648 • Removing setuptools (65.3.0)
#8 4.726 • Updating setuptools (63.2.0 -> 65.3.0): Skipped for the following reason: Not required
#8 4.727 • Installing gunicorn (20.1.0): Skipped for the following reason: Not required
#8 DONE 4.9sI also tested re-writing the extras as groups instead, and did not get the same result. Making these changes fixed the problem:
#gunicorn = {optional = true, version = "*"}
#
#[tool.poetry.extras]
#web = [
# "gunicorn",
#]
[tool.poetry.group.web]
optional=true
[tool.poetry.group.web.dependencies]
gunicorn = "*"
Activity
dimbleby commentedon Aug 24, 2022
I don't see that #4336 ever had anything to say about extras, and #4242 is a different thing altogether (relating to poetry 1.1's handling of poetry 1.2 lockfiles).
Described behaviour seems correct to me: you've asked to install a package without extras, and poetry has made that happen.
chimaerase commentedon Aug 24, 2022
IMO poetry's behavior with extras should parallel what's happening with
--without devflag as documented in #4336. From the PR description: "Any locked packages will be kept in the environment regardless of what is asked via the install command unless the--syncoption is specified...Overall, this should be less surprising for end users makes working with dependency groups easier."Also, IMO whether
pyproject.tomlis written using an optional group or using extras, the end result should be consistent. I think this ticket shows it isn't right now.dimbleby commentedon Aug 24, 2022
groups and extras are absolutely not interchangeable and you should not expect them to give identical results - them not being "consistent" here is not an issue.
chimaerase commentedon Aug 25, 2022
Ok, thanks for your reply.
I submit that the distinction between when to use extras and when to use optional groups isn't clear in the documentation yet (especially to me, mostly a 1.1 user, who only had the option to use extras in that version). I think this important distinction re: whether dependencies get removed when the extra / optional group isn't selected is important to end users and should be clearly / prominently documented.
dimbleby commentedon Aug 25, 2022
I see that #6230 has just gone in hopefully helping to clarify the distinction, I expect further MRs improving the docs would be welcome.
chimaerase commentedon Aug 25, 2022
Thanks, that is a good start!
Unfortunately, I'm asking for documentation largely because I'm still unclear on the motivations for making optional groups/extras behave differently when not selected for install. I don't think clarifying that reasoning is something I can contribute to.
Clarify removal for non-installed extras & optional groups (python-po…
dimbleby commentedon Sep 12, 2022
duplicate #5895
Clarify removal for non-installed extras & optional groups (#6229)
Clarify removal for non-installed extras & optional groups (#6229)
Clarify removal for non-installed extras & optional groups (#6229)
Clarify removal for non-installed extras & optional groups (#6229)
github-actions commentedon Mar 1, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.