Repository navigation
provide binary wheels for Linux compatible with lxml #316
Description
Activity
@nijel The workaround solution would be to force lxml not to use the pre-compiled wheel:
pip install lxml xmlsec --no-binary=lxml
We already have pre-compile wheels for linux and macos (see release note: https://github.com/xmlsec/python-xmlsec/releases/tag/1.3.14) but I didn't upload them in pypi because of this version checking that was added in the latest release.
I'm gonna note this issue for the next release, meanwhile can you provide the following information:- OS distribution and version
- xmlsec1 library version
- libxml2 version
- python version
I know the workaround, the problem is that there is no way to tell this inside dependencies. Instead, you need to tell users to not use
pip install <package>, butpip install --no-binary=lxml <package>, what half of them won't do and complain that things do not work then :-).I'm on Debian 12 (Python 3.11, libxml2 2.9.14, xmlsec1 1.2.37), it happens on GitHub CI environment with Ubuntu 22.04 (any Python version installed by actions/setup-python, libxml 2.9.13, xmlsec1 1.2.33).
PS: I'm happy user of https://github.com/pypa/cibuildwheel and https://github.com/pypa/gh-action-pypi-publish to automate all the wheel builds and publishing.
@mxamin i like the idea of providing Linux wheels for the next release as well. We are already making the wheels with cibuildwheel, so it's just a matter of distributing them.
I can work on a PR to automate attaching the wheels to the release in GitHub and uploading them to PyPI if you would like.
@mxamin i like the idea of providing Linux wheels for the next release as well. We are already making the wheels with cibuildwheel, so it's just a matter of distributing them.
I can work on a PR to automate attaching the wheels to the release in GitHub and uploading them to PyPI if you would like.
I could have do it manually (simply downloading the artificats and uploading them to pypi) but I decided not to do it because of libxml2 version check. Assume that lxml wheel is build with version 2.9.x and python-xmlsec with 2.11.x, we would end up with the mismatch error which is not good. So the real problem here is to fix the lxml/python-xmlsec compatibility issue for real :)
Making the integration with lxml safer, like you are suggesting, would be a win for sure.
Even if we do that though, I think there is still a strong argument for providing wheels for Linux. It makes things much easier for end users of the package, especially when installing inside a container where you don't necessarily have all the build tooling available.
Definitely it would help. I wonder if the created wheels would help in reported cases. I probably check that later.
I was thinking if it is possible to build a private xmlsec1 against libxml2 from lxml and ship it inside the wheel.
@nijel would you be able to manually install one of the Linux wheels attached the the release in GitHub https://github.com/xmlsec/python-xmlsec/releases/tag/1.3.14 and see if it solves your issue?
The wheels are statically linked against libxml2 and libxmlsec, so I believe it will sort your issue. Those are the wheels we are discussing uploading to pypi.
- added 2 commits that reference this issue
on Apr 19, 2024 3 remaining items
Is it possible that after this change, I get the
lxml & xmlsec libxml2 library version mismatcherror withlxmlbuilt locally? The CI started to fail for us meanwhile, so this might be the reason.It looks like maybe you were installing
lxmlwith no-binary, but installing the binary wheel forxmlsec. Now that we have wheels uploaded bothlxmlandxmlsecneed to be built the same way. Either installed from wheels or built with no-binary.Maybe the version check probably can be skipped in case xmlsec is linked statically with the libxml?
Because
xmlsecpasses data intolxmlboth need to be built with the same version oflibxml2. This situation is unfortunate, and we'd like to fix it, so this requirement doesn't exist. But for now it does.This means that either they both need to be installed from wheels, which includes same version of
libxml2or they both need to be compiled using no-binary so they both get the system version oflibxml2. If you mix and match these approaches you will seeversion mismatcherror.Thanks for explanation, it now works for me. Publishing binary wheels while I was testing this in CI made results for me quite unpredictable, but now with wheels-only setup, everything works just fine. Thanks for your effort to make this work!
- added 2 commits that reference this issue
on May 28, 2024 - added 2 commits that reference this issue
on May 20, 2025 - added 2 commits that reference this issue
on May 28, 2025 - added 3 commits that reference this issue
on Jun 25, 2025
I know this error has been added intentionally to catch other breakages, but is there a way to provide binary wheels that will work with current lxml? So that
pip install lxml xmlsecwill bring working combo.Presently, it doesn't:
The problem is that lxml wheels are built with internal libxml while xmlsec is using system one.
Rebuilding lxml against system libxml does address the issue, but working combo out of the box would be better.