Skip to content

soup: [attr*=], [attr^=], [attr$=] (and ~=, |=) are dropped, silently widening the match #181

Closed

Description

@clementine-oaklight

Summary

select() accepts any [...] token but only understands [attr] and [attr="value"]. Every other CSS attribute operator is tokenized, fails to parse as a constraint, and is dropped. The compound then degrades to its tag name and the match silently widens, so a caller cannot tell a working selector from a widened one.

Reproduction

Three forms, chosen so a dropped constraint is visible instead of a near miss:

from soup import Soup

html = '''<form id="form_a" action="/process_consent"><button type="submit-alpha">A</button></form>
<form id="form_b" action="/account/process_login"><button type="submit">B</button></form>
<form><button type="submit">C</button></form>'''

s = Soup(html)
s.select('form[action*="process_login"] button')   # ['A', 'B', 'C'], expected ['B']
s.select('form[action^="/account"] button')        # ['A', 'B', 'C'], expected ['B']
s.select('form[action$="process_login"] button')   # ['A', 'B', 'C'], expected ['B']
s.select('button[type*="alpha"]')                  # ['A', 'B', 'C'], expected ['A']

A broken selector returns exactly what its tag-only form returns, which is the tell:

form[action*="process_login"] button  -> ['A', 'B', 'C']  ==  form button  -> ['A', 'B', 'C']
button[type*="alpha"]                 -> ['A', 'B', 'C']  ==  button       -> ['A', 'B', 'C']

Measured on v2026.9.28 (46a0426) against beautifulsoup4 4.14.3 / soupsieve 2.10:

selector soup soupsieve
form[action*="process_login"] button ['A', 'B', 'C'] ['B']
form[action^="/account"] button ['A', 'B', 'C'] ['B']
form[action$="process_login"] button ['A', 'B', 'C'] ['B']
button[type*="alpha"] ['A', 'B', 'C'] ['A']
button[type$="alpha"] ['A', 'B', 'C'] ['A']
button[type~="submit-alpha"] ['A', 'B', 'C'] ['A']
form[id*="form_"] button ['A', 'B', 'C'] ['A', 'B']
button[type|="submit"] ['A', 'B', 'C'] ['A', 'B', 'C']

The gap is every operator except plain =: ~=, |=, ^=, $=, *=, in both the ancestor and the target position. The last row shows the trap for a test author: |= agrees with soupsieve on this fixture while being wrong for the wrong reason (the constraint is dropped, and all three buttons match either way). A discriminating case such as button[type|="submit-"] separates them ([] when parsed, ['A', 'B', 'C'] when dropped).

Root cause

_SIMPLE_RE (soup/soup.py:589) matches \[[^\]]+\] as an attribute token, while _ATTR_RE (soup/soup.py:601) only accepts [name] and [name="value"]. _apply_regex_match (soup/soup.py:642) drops the token when _ATTR_RE does not match:

am = _ATTR_RE.match(m.group("attr"))
if am:
    compound.setdefault("attrs", []).append(...)

For form[action*="x"] the compound keeps only tag="form", which is the widened selector.

Impact

Measured against the real Tello login page (the selector shape behind Oaklight/Tello-Renewal#10):

form[action*="process_login"] button[type="submit"]   n=7   # identical to: form button[type="submit"]

Seven submit buttons match; three of them are in forms posting to process_login. The rest belong to the cookie consent banner, the social sign-in form and the newsletter form. Chromium counts 3.

Suggested fix

Either implement the operators, or raise on attribute syntax the parser does not implement. select's docstring documents only [attr] and [attr="value"], so callers reasonably assume anything else either works or fails loudly. The silent widening is the part that costs time.

Environment: zerodep v2026.9.28 (46a0426), Python 3.11.

Activity

  1. added a commit that references this issue on Oct 13, 2026
    f71a9f0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions