Repository navigation
Poetry fails when two different dependency types are given for the same dependency #6215
Description
-
I am on the latest Poetry version.
-
I have searched the issues of this repo and believe that this is not a duplicate.
-
If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption). -
OS version and name: MAc OS Darwin Kernel Version 21.3.0
-
Poetry version: Poetry 1.1.14
Issue
poetry update consistently crashes when I try running it with a pyproject.toml file that contains the following:
pyscf = [
{ url = {my_url.pyscf-2.0.1-cp38-cp38-macosx_12_0_arm64.whl}, markers="platform_machine == 'arm64'"},
{ version="^2.0.1", markers= "platform_machine != 'arm64'" }
]
I chose to configure poetry in this way because the package that I need installed (pyscf) hasn't officially been generated for MacOS with the M1 chip. This way I am able to have devices that can get the pyscf package from pypi doing so, and M1 macs (and any other arm64) get an instance of the package I compiled locally and that I am storing on a server. However when configured like this poetry crashes before attempting to download or install any package.
The error that I receive is:
9 ~/venvs/venv1/lib/python3.8/site-packages/clikit/console_application.py:131 in run
129│ parsed_args = resolved_command.args
130│
→ 131│ status_code = command.handle(parsed_args, io)
132│ except KeyboardInterrupt:
133│ status_code = 1
8 ~/venvs/venv1/lib/python3.8/site-packages/clikit/api/command/command.py:120 in handle
118│ def handle(self, args, io): # type: (Args, IO) -> int
119│ try:
→ 120│ status_code = self._do_handle(args, io)
121│ except KeyboardInterrupt:
122│ if io.is_debug():
7 ~/venvs/venv1/lib/python3.8/site-packages/clikit/api/command/command.py:171 in _do_handle
169│ handler_method = self._config.handler_method
170│
→ 171│ return getattr(handler, handler_method)(args, io, self)
172│
173│ def __repr__(self): # type: () -> str
6 ~/venvs/venv1/lib/python3.8/site-packages/cleo/commands/command.py:92 in wrap_handle
90│ self._command = command
91│
→ 92│ return self.handle()
93│
94│ def handle(self): # type: () -> Optional[int]
5 ~/venvs/venv1/lib/python3.8/site-packages/poetry/console/commands/update.py:47 in handle
45│ self._installer.update(True)
46│
→ 47│ return self._installer.run()
48│
4 ~/venvs/venv1/lib/python3.8/site-packages/poetry/installation/installer.py:103 in run
101│ local_repo = Repository()
102│
→ 103│ return self._do_install(local_repo)
104│
105│ def dry_run(self, dry_run=True): # type: (bool) -> Installer
3 ~/venvs/venv1/lib/python3.8/site-packages/poetry/installation/installer.py:235 in _do_install
233│ )
234│
→ 235│ ops = solver.solve(use_latest=self._whitelist)
236│ else:
237│ self._io.write_line("Installing dependencies from lock file")
2 ~/venvs/venv1/lib/python3.8/site-packages/poetry/puzzle/solver.py:65 in solve
63│ with self._provider.progress():
64│ start = time.time()
→ 65│ packages, depths = self._solve(use_latest=use_latest)
66│ end = time.time()
67│
1 ~/venvs/venv1/lib/python3.8/site-packages/poetry/puzzle/solver.py:239 in _solve
237│ packages = result.packages
238│ except OverrideNeeded as e:
→ 239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
240│ except SolveFailure as e:
241│ raise SolverProblemError(e)
ValueError
Package('pyscf', '2.0.1') is not in list
at ~/venvs/venv1/lib/python3.8/site-packages/poetry/puzzle/solver.py:214 in solve_in_compatibility_mode
210│ packages.append(package)
211│ depths.append(_depths[index])
212│ continue
213│ else:
→ 214│ idx = packages.index(package)
215│ pkg = packages[idx]
216│ depths[idx] = max(depths[idx], _depths[index])
217│
218│ for dep in package.requires:
After some debugging I believe that this occurs because of two different instances of the class Package that are supposed to represent pyscf and compared while running the program. This all happens in the function "solve_in_compatibility_mode" in solver.py. In my case the list "packages" contains an instance of package as follows:
Package('pyscf', '2.0.1', source_type='url', source_url='{my_url}/pyscf-2.0.1-cp38-cp38-macosx_12_0_arm64.whl')
and the "package" variable is:
Package('pyscf', '2.0.1')
In this case the if statement "package not in packages" will return False, because the comparison sees them as the same instance. However later (line 214 in solver.py) the list index function is called and it scans the list and compares the hash of each object in the list to the hash of the object who's index it is looking for. In this case a ValueError is raised because the seperate instances of Package are different and therefore have different hashes.
I believe that this is unintentional and can be fixed.
Thanks
packages = []
depths = []
for override in overrides:
self._provider.debug(
"<comment>Retrying dependency resolution "
"with the following overrides ({}).</comment>".format(override)
)
self._provider.set_overrides(override)
_packages, _depths = self._solve(use_latest=use_latest)
for index, package in enumerate(_packages):
if package not in packages:
packages.append(package)
depths.append(_depths[index])
continue
else:
**idx = packages.index(package)**
pkg = packages[idx]
depths[idx] = max(depths[idx], _depths[index])
Activity
dimbleby commentedon Aug 23, 2022
Try 1.2.0rc1.
I don't know whether this does better or not because you haven't shared a repro that anyone else can run, but it's definitely worth a try.
If it doesn't do better, please provide a repro that others can work with.
roiefclassiq commentedon Aug 24, 2022
Adding the following to your pyproject.toml file will cause the same behavior
pyscf = [
{ git = "https://github.com/pyscf/pyscf", tag = "v2.0.1", markers= "platform_machine == 'arm64'"},
{ version="^2.0.1", markers= "platform_machine != 'arm64'" }
]
dimbleby commentedon Aug 24, 2022
and did you try with 1.2.0rc1?
ralbertazzi commentedon May 20, 2023
Closing as stale and duplicate of #7722
github-actions commentedon Feb 29, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.