Repository navigation
[Discussion] Poetry to provide a decent hook-based plugin system #693
Description
It's a follow-up for #672 (comment).
- I have searched the issues of this repo and believe that this is not a duplicate.
Issue
So setuptools have this nice hook points system where they allow replacing certain utils by some third-party implementations.
It can catch up with what's available by iterating through a list of entrypoints with names corresponding to hooks.
This is a fairly popular technique among setuptools-based dists. For example, pytest and tox support having third-party dists being automatically activated and used when installed given they "advertise" their entry points as required by the plugin system. These two tools use https://github.com/pytest-dev/pluggy to achieve this.
Back to Poetry, I think lots of people agree that providing ability to extend such a great tool is a widely wanted feature.
This would allow adding custom dist/package format generators, custom metadata generators etc. They could be used to retrieve long description from some esoteric formats or places converting that into supported markups. This would also make possible having a custom version constructor like popular plugins to setuptools facilitating version sourcing from Git/Hg (#140/#672). Or, another example, exporting a pinned env and direct dependencies into pip-friendly formatted requirements. And who knows, maybe someone will even try out having another resolver replacement?
Any thoughts on how this might look like?
Activity
jgirardet commentedon Dec 1, 2018
Hi,
At the time I was searching a way to specify python version at install, I thought a bit about it.
What does we want ?
add some custom command:
the easiest I think. It just needs to look for new commands in the setup tools entry point and add it in console/application.py
add some flags/option to existing command:
already more tricky. should it be added for all commands ? I see something like it : a do_plugin_task() a start of
handlewhich would apply some change before the builtin code. it isn't something hard except that poetry runs via cleo which define the option in the class docstring. so if you want to add an option to an existing command, it has to be changed at instantiation time. It means parsing the docstring change it and instantiate the "new" command. You need to have a clear logic in it since many plugins could change the same command.What is different in poetry for doing this ?
Poetry is not installed via pip so using the standard way of entry point maybe not be ok (I'm not sure about it). It should come from a poetry command. someting like :
poetry plugin install some_pluginand the plugin could be in pypi.Should we try to implement this ?
I would be ok, maybe others too but my feeling is that @sdispater since the beginning of the project, likes to implement himself the new core features of poetry. It's me feeling, I'm not sure about sebastien's mind about it.
Anyway, the idea of a plugin system is itself a great thing !!!
sdispater commentedon Dec 3, 2018
Just for information: I am currently rewriting the core of Cleo to make it more flexible and pluggable.
Regarding the actual plugin system, I am not sure what it will look like just yet. But the general idea is to be able to add commands and command options to Poetry like @jgirardet said. I also think being able to hook into the
Poetryobject creation would be good since it would allow things like #672.Now, i will likely implement the plugin system myself and make a PR to get feedback.
This is something I had in my mind from the start but there were more pressing matters than this but since Poetry is on the path of stabilization I think it's a good time to start experimenting with it.
I can't guarantee that it will make it to the
1.0release but I will try.webknjaz commentedon Dec 3, 2018
Cool, thanks for the info.
webknjaz commentedon Mar 14, 2019
Hey @sdispater, is https://poetry.eustace.io/docs/plugins/ something that implements what you wanted? Is it already possible to hook into that?
webknjaz commentedon May 31, 2019
@sdispater any updates?
funkyfuture commentedon Oct 11, 2019
while @webknjaz inital post points to uses that would extend
poetry's functionality as a packaging tool and kinda venv manager, the idea to allow custom commands points to a different path. from the discussions that i came across, they all aimed at either extendingpoetryto a task runner or to interact with version information stored in the config file. for these cases,poetrywould only act as a framework to implement cli commands around apyproject.toml. at that pointpoetrywould be two different things. in a better, possible worldpoetryand these other command implementations would share a common framework.vBLFTePebWNi6c commentedon Oct 17, 2019
Hi, @sdispater ! Our team is looking for pipenv's alternatives and poetry seems great, if it'd have plugin system, which is mandatory for our development process. I've seen that related PR is in progress -- can you give us eta for this feature?
mrh1997 commentedon Jan 3, 2020
Another use case for hooks would be calling preprocessors like cython or template precompilers.
Ideally these libraries could provide a plugin, that is called automaticially before the "build" command. The plugins could retrieve the details which files to process and how from a custom section of the
pyproject.toml.One important thing is that this requires the plugin interface living within the projects python virtual environment, while the plugin interface discussed above lives in the python interpreter where poetry is installed.
The big advantage of a plugin interface living in the project interpreter would be, that it not only solves the preprocessing-requirement, but also the other requirements of this issue on a per project basis. This would mean that adding a plugin is as simple as adding a development dependency to
pyproject.toml. It will ensure that all developers working on the project are using the same plugin and plugin-version.gsemet commentedon Feb 11, 2020
That would be really useful if plugins could be installed inside the virtualenv of the package, not system/user wide!
6 remaining items
gsemet commentedon Jul 5, 2020
I wonder why this feature would not be provided by Poetry out-of-the-box, of course on a voluntary basis.
Conchylicultor commentedon Jan 28, 2021
Is there any update on this ? We deploy our project in two pip packages (a
project-nightlyreleased every day andprojectcontaining the stable version). There is currently no way to support this in poetry, which is a blocker for us.@abn , it seems @sdispater isn't contributing anymore. Will someone take over the issue ?
Moved the metadata into setup.cfg from setup.py.
miigotu commentedon Apr 6, 2021
Example (simple but useful initial concept) project utilizing the new plugin system is at https://pypi.org/project/poetry-date-version-plugin/ and https://github.com/miigotu/poetry-date-version-plugin
#3733 makes the possibilities endless
It requires installing poetry from master
pip install git+https://github.com/python-poetry/poetryin order to work until next poetry release.gsemet commentedon Apr 6, 2021
Does this means the plug-in can be located inside the virtual env and not on the same installation than poetry ?
miigotu commentedon Apr 8, 2021
Yes, it can be installed next to poetry itself with pip even
gsemet commentedon Apr 8, 2021
What I would expect is not to pollute the system install and everything works from the user’s virtualenv
miigotu commentedon Apr 9, 2021
It works either way. If poetry is inside a venv, so are the plugins.
bkhl commentedon Apr 9, 2021
I was hoping as well that it would be possible to enable a plugins specific to a project.
However, it seems difficult to allow them to run within the virtualenv that you created with Poetry, since that might be running an entirely different Python version than the one that Poetry is running in.
miigotu commentedon Apr 11, 2021
That's when you run
poetry plugin addhave you tried the feature?takeda commentedon Apr 12, 2021
@miigotu, @gsemet meant whether you can include a plugin in your project dependencies that will be invoked within your project's virtualenv. Kind of like with setuptools, you could include setuptools_scm in setup_install to get project version from git.
Anyway, I looked at the PR and the answer unfortunately is that plugin needs to be installed in the same environment from which poetry runs. Then your pyproject.toml needs to declared it wants to use it. So ultimately is similar what existing unofficial plugins were doing, just that now there's an official way to do it. This kind of sucks for reproducibility (another developer will need to also install this plugin), but it is still something.
I'm hoping that maybe in the future when you run
poetry installfrom your project, poetry would display something like:gsemet commentedon Apr 12, 2021
If you want this plugin system to work you need to treat this plugin like a “dev-package”, ie, just install it within the virtualenv.
Installing it in the same place than poetry is pretty much useless.
My use case is auto versioning. I just want this feature for the moment it needs to be installed on the user environnement (poetry-dynamic-versioning). So I have a ugly bootstrap script that installs it with a fixed version, like the old days where no lock files existed...
the plugin dependency should be declared and controlled in the lock file, so that two different project could use two different versions of the same plugin.
github-actions commentedon Mar 2, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.