Repository navigation
Potential thread safety issue under certain environments: Incorrectly nested style tag found #3010
Description
In certain environments, specifically CI environments, this issue occurs randomly. It would seem that when using the new executor for installation the io being used is not thread safe and pastel fails attempting to pop the wrong stack.
• Installing glob2 (0.6)
ValueError
Incorrectly nested style tag found.
• Installing httpretty (0.9.7)Reacted by Machiel Groeneveld, Vikash, Michal Szelag, Kudryavets, Jan Skrle, phyng, Aleksei, Matthieu Devlin, Patrick Decat, Jiri Daněk and 1 more
Reacted by Öykü and Rui Rei
Activity
PetterS commentedon Oct 5, 2020
There are also other issues with the parallel installation: see #2658.
Test workflow: pin Poetry to 1.0.10
machielg commentedon Oct 15, 2020
I also have this problem when doing poetry (1.1.2) install in gitlab. The workaround is to disable the new installer:
poetry config experimental.new-installer falseit seems to fail at a random point or sometimes not at all.
PetterS commentedon Oct 15, 2020
Yeah, this should be relatively easy to fix. Just wrap a lot of calls to
self._iowith a lock in the executor. Alteratively, makeIOfrom clikit safe to call from multiple threads.vikigenius commentedon Dec 12, 2020
Yeah, I am also facing this issue on aws, when using the user-data script to install poetry packages
PetterS commentedon Dec 14, 2020
For automation or when reliability is needed, the new installer should definitely be disabled.
We have a config file in the repo disabling it by default. If anyone wants to use it (e.g. when setting up a new env), they can do so manually.
michalszelagsonos commentedon Dec 17, 2020
I hit this couple times now, also in CI. If I keep running into this, I may disable new installer which is a bummer since it significantly speeds up the install in my project with 100+ dependencies.
sdispater commentedon Dec 18, 2020
@Michal-Szelag-Sonos Which Python version are you using when the error occurs?
michalszelagsonos commentedon Dec 18, 2020
I am seeing this on 2.7
PetterS commentedon Dec 18, 2020
Should happen on any Python version since
IOis not thread-safe. This particular issue is easy to fix in the executor, though. There are other concurrency issues that are more tricky.sdispater commentedon Dec 18, 2020
Yes, from experience I only have seen this on Python 2.7 and not on other Python versions, I am not sure why. I don't know if it's a coincidence or not though.
It may also be caused by the
futurespackage used on Python 2.7.Note that every write in the
Executorclass occurs after having acquired a lock so even though theIOis not thread safe, everything should work as expected.michalszelagsonos commentedon Dec 18, 2020
So, silly question, can I use python3 poetry to manage a python2.7 project, assuming I would install poetry using
pip3? It seems that this should work but double checking. I could test the2.7vs3.xassertion by using python3 env for my toolchain which manages python 2.7 project and give it a few days to bake in.27 remaining items
jeffwidman commentedon Aug 8, 2022
Looks like there will also be a
installer.parallel falseoption that can be used in place ofinstaller.max-workers 1.However, neither is available with
poetry<1.2, (as I just discovered!) so AFAICT,experimental.new-installer falseis the only viable workaround for now... plus once1.2drops, these workarounds shouldn't even be needed.Workaround thread safety issue by limiting worker count
github-actions commentedon Mar 1, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.