Repository navigation
soup: [attr*=], [attr^=], [attr$=] (and ~=, |=) are dropped, silently widening the match #181
Description
Summary
select() accepts any [...] token but only understands [attr] and [attr="value"]. Every other CSS attribute operator is tokenized, fails to parse as a constraint, and is dropped. The compound then degrades to its tag name and the match silently widens, so a caller cannot tell a working selector from a widened one.
Reproduction
Three forms, chosen so a dropped constraint is visible instead of a near miss:
from soup import Soup
html = '''<form id="form_a" action="/process_consent"><button type="submit-alpha">A</button></form>
<form id="form_b" action="/account/process_login"><button type="submit">B</button></form>
<form><button type="submit">C</button></form>'''
s = Soup(html)
s.select('form[action*="process_login"] button') # ['A', 'B', 'C'], expected ['B']
s.select('form[action^="/account"] button') # ['A', 'B', 'C'], expected ['B']
s.select('form[action$="process_login"] button') # ['A', 'B', 'C'], expected ['B']
s.select('button[type*="alpha"]') # ['A', 'B', 'C'], expected ['A']A broken selector returns exactly what its tag-only form returns, which is the tell:
form[action*="process_login"] button -> ['A', 'B', 'C'] == form button -> ['A', 'B', 'C']
button[type*="alpha"] -> ['A', 'B', 'C'] == button -> ['A', 'B', 'C']
Measured on v2026.9.28 (46a0426) against beautifulsoup4 4.14.3 / soupsieve 2.10:
| selector | soup | soupsieve |
|---|---|---|
form[action*="process_login"] button |
['A', 'B', 'C'] |
['B'] |
form[action^="/account"] button |
['A', 'B', 'C'] |
['B'] |
form[action$="process_login"] button |
['A', 'B', 'C'] |
['B'] |
button[type*="alpha"] |
['A', 'B', 'C'] |
['A'] |
button[type$="alpha"] |
['A', 'B', 'C'] |
['A'] |
button[type~="submit-alpha"] |
['A', 'B', 'C'] |
['A'] |
form[id*="form_"] button |
['A', 'B', 'C'] |
['A', 'B'] |
button[type|="submit"] |
['A', 'B', 'C'] |
['A', 'B', 'C'] |
The gap is every operator except plain =: ~=, |=, ^=, $=, *=, in both the ancestor and the target position. The last row shows the trap for a test author: |= agrees with soupsieve on this fixture while being wrong for the wrong reason (the constraint is dropped, and all three buttons match either way). A discriminating case such as button[type|="submit-"] separates them ([] when parsed, ['A', 'B', 'C'] when dropped).
Root cause
_SIMPLE_RE (soup/soup.py:589) matches \[[^\]]+\] as an attribute token, while _ATTR_RE (soup/soup.py:601) only accepts [name] and [name="value"]. _apply_regex_match (soup/soup.py:642) drops the token when _ATTR_RE does not match:
am = _ATTR_RE.match(m.group("attr"))
if am:
compound.setdefault("attrs", []).append(...)For form[action*="x"] the compound keeps only tag="form", which is the widened selector.
Impact
Measured against the real Tello login page (the selector shape behind Oaklight/Tello-Renewal#10):
form[action*="process_login"] button[type="submit"] n=7 # identical to: form button[type="submit"]
Seven submit buttons match; three of them are in forms posting to process_login. The rest belong to the cookie consent banner, the social sign-in form and the newsletter form. Chromium counts 3.
Suggested fix
Either implement the operators, or raise on attribute syntax the parser does not implement. select's docstring documents only [attr] and [attr="value"], so callers reasonably assume anything else either works or fails loudly. The silent widening is the part that costs time.
Environment: zerodep v2026.9.28 (46a0426), Python 3.11.
Activity
fix(soup): support the CSS attribute operators in selectors