Skip to content

feat: recognize Division/Subdivision heading markers in hierarchy inference - #4473

Open
aImErYbArRaUlT wants to merge 1 commit into
docling-project:mainfrom
aImErYbArRaUlT:feat/heading-hierarchy-division-subdivision
Open

aImErYbArRaUlT wants to merge 1 commit into
docling-project:mainfrom
aImErYbArRaUlT:feat/heading-hierarchy-division-subdivision

Conversation

@aImErYbArRaUlT

@aImErYbArRaUlT aImErYbArRaUlT commented Oct 1, 2026 •

Copy link
Copy Markdown

Resolves #4472

Summary

The heading-hierarchy numbering recognizer handled Part/Chapter/Article but not Division/Subdivision, so legal-statute structure (Part > Division > Subdivision > section) stayed flat even with HeadingHierarchyOptions(enabled=True). This teaches the parser the two keywords. Because Division/Subdivision are ordinary words ("Division of Powers", "Division du congé"), they count only when a real enumerator follows; multi-letter tokens are validated with the existing _is_roman, consistent with the file's own rule in _classify_letter. docs/usage/heading_levels.md is updated with the new markers and default order.

Changes

  • _DEFAULT_FAMILY_ORDER: add division/subdivision between part and the section-level families.
  • _parse_marker: recognize the two keywords via _KW_DIVISION/_KW_SUBDIVISION, gated by a new _keyword_enumerator helper that requires an Arabic index, a single letter, or a valid Roman numeral (_is_roman). The enumerator must be followed by whitespace, end of string, or separator punctuation, so French elisions like Division d'appel are not treated as markers.
  • _LEADING_MARKER: add the keywords so bookmark/title matching strips them in sync.
  • pipeline_options.py: document the families in the numbering_schemes field description.
  • docs/usage/heading_levels.md: add the markers and the default order.
  • Tests: positives (including separator-terminated enumerators), negatives (a Roman-letter word and French elisions, straight and curly apostrophe), and a custom-scheme lowest-rank case.

Evidence

Parser (_parse_marker):

DIVISION I / Division 1 / DIVISION A      -> division
SUBDIVISION A / Subdivision a             -> subdivision
Division of Property / Division du congé  -> None   (no enumerator)
Division Civil Remedies                   -> None   (word of Roman letters; blocked by _is_roman)
Division d'appel / Division d'emploi      -> None   (French elision; apostrophe is not a terminator)

Corpus (20 Canadian federal + Alberta statutes): 372 headings begin with Division/Subdivision; 341 classified structural, 31 left unclassified (plain-word titles), 0 false negatives.

End-to-end (Alberta Corporate Tax Act, enabled=True): all 59 Division headings move ## -> ###, nested under the 20 Part headings; total heading count unchanged (359).

Side effect: inserting two families above the section rank pushes the pre-existing (N)-subsection headings one level deeper. In this document one reaches level 6, which docling-core renders as seven # (beyond CommonMark's six-level cap), so it displays as text rather than a heading. That is the existing max_level clamp interacting with docling-core's level+1 hash mapping, not new behavior; I am happy to file it in docling-core separately.

Behavior change for custom numbering_schemes: a list that omits division/subdivision previously left those headings unnumbered; they are now numbered at the lowest rank, below every listed family. The default order is unaffected. Covered by test_omitted_scheme_family_ranks_lowest. Users who want them ranked higher can add the two families to their list.

Known limitation (follow-up): compound enumerators such as Division 1A / Subdivision 6AA (common in Australian statutes) are not yet recognized and fall through as unnumbered. They can be added later without changing the default behavior. The default order also places part/title above division; documents that nest Division above Title can reorder via numbering_schemes (documented in heading_levels.md).

Testing

tests/test_heading_hierarchy.py - 45 passed. make validate clean.

Checklist

  • Documentation has been updated (docs/usage/heading_levels.md).
  • Examples have been added, if necessary. (N/A - docs/examples/heading_levels.py already covers usage.)
  • Tests have been added.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @aImErYbArRaUlT, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@codecov

codecov Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…erence

Signed-off-by: Aimery Barrault <aimery@barratec.com>
@aImErYbArRaUlT
aImErYbArRaUlT force-pushed the feat/heading-hierarchy-division-subdivision branch from 92ce259 to 69afd9a Compare October 2, 2026 08:29
@aImErYbArRaUlT

Copy link
Copy Markdown
Author

Heads-up on the red CI, in case it's useful: the only failing check here is code-checks / lint (3.10) (with code-checks / check and ci-status failing only as its aggregates). It's a single ruff error:

C901 `_handle_list` is too complex (31 > 30)
  --> docling/backend/html_backend.py:2894

That's in html_backend.py, which this PR doesn't modify. It appears to have come in with #4390, and ruff reports the same error on main itself, so it looks unrelated to these changes. Tests, codecov/patch, and DCO are green.

Happy to rebase once main's lint is green, or to adjust anything on my side if you'd prefer.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Heading hierarchy: recognize Division/Subdivision numbering markers

1 participant