Repository navigation
lxml 6.0.0 is not supported #263
Description
Activity
Hi, following up with this, lxml@5.4.0 has been marked to have Stack-based Buffer Overflow by Checkmarx (CVE-2025-6170). According to it, the remediate to this vulnerability is on 6.0.2 fyi
Reacted by Ramona TWhat was the problem that
lxml >= 6.0was causing?As far as I can tell,
lxmlis only used withinredshift_connectoras the specified parser for BeautifulSoup4 when parsing SAML responses within the Active Directory Federation Services Identity Provider plugin inredshift_connecter/plugin/adfs_credentials_provider.pyThe modified dependency with the upper bound makes it impossible to install
redshift_connectoron Python 3.14 (without buildinglxmlfrom source).If the upper bound is left in place instead of fixing the underlying problem, then
redshift_connectoris effectively ending support at Python 3.13Reacted by Gordon Myers and Stuart BertramWhat was the problem that
lxml >= 6.0was causing?It looks like it may have been related to this behavior change noted earlier this year:
When you build a BeautifulSoup from a plain text, that text was getting wrapped into a
<p>tag when the installed lxml version is earlier than 6, but isn’t any more with the new version of lxml.
-- Luc SaffreOr perhaps:
Fixed an unhandled exception when parsing invalid markup that contains the { character
when using lxml==6.0.0. [bug=2116306]
-- bs4 changelog for 4.13.5 (20250824)It'd be a good idea to bump the version of beautifulsoup4 and remove the lxml upper bound to see if the problem has been fixed in bs4.
The fix to increase upper bound for lxml to support python 3.14 and bump the bs4 version as well are implemented in 2.1.13 release
https://pypi.org/project/lxml/#history
A new version was released by lxml and our requirements.txt needs version lxml>=4.6.5
However with the latest version connector is failing